The Holy Grail of Content-Based Media Analysis

Tools and systems for content-based access to multimedia and-image., video, audio, graphics, text, and any number of combinations-has increased in the last decade. We've seen a common theme of developing automatic analysis techniques for deriving metadata (data describing information in the content at both syntactic and semantic levels). Such metadata facilitates developing innovative tools and systems for multimedia information retrieval, summarization, delivery, and manipulation. Many interesting demonstrations of potential applications and services have emerged-finding images visually similar to a chosen picture (or sketch); summarizing videos with thumbnails of keyframes; finding video clips of a specific event, story, or person; and producing a two-minute skim of an hour-long program. In order to evaluate content-based research methodologies, the article considers intended users and whether alternative solutions exist and areas of research