Multi-modal dialog scene detection using hidden Markov models for content-based multimedia indexing

Document Type

Article

Publication Date

6-1-2001

Abstract

A class of audio-visual data (fiction entertainment: movies, TV series) is segmented into scenes, which contain dialogs, using a novel hidden Markov model-based (HMM) method. Each shot is classified using both audio track (via classification of speech, silence and music) and visual content (face and location information). The result of this shot-based classification is an audio-visual token to be used by the HMM state diagram to achieve scene analysis. After simulations with circular and left-to-right HMM topologies, it is observed that both are performing very good with multi-modal inputs. Moreover, for circular topology, the comparisons between different training and observation sets show that audio and face information together gives the most consistent results among different observation sets.

Identifier

0035368101 (Scopus)

Publication Title

Multimedia Tools and Applications

External Full Text Location

https://doi.org/10.1023/A:1011395131992

ISSN

13807501

First Page

137

Last Page

151

Issue

2

Volume

14

This document is currently not available here.

Share

COinS