Semantic Video Search Engines (Dutch Research)

A couple of weeks ago I was invited to a series of presentations by several Dutch research projects in the area of semantic media retrieval. The projects CHOICE and MUNCH presented their preliminary results and next steps. Guus Schreiber presented CHOICE together with Luit Gazendam and Hennie Brugman. Arnold Smeulders presented MUNCH with a demo by Cees Snoek: MediaMill semantic video search engine
MediaMill
The MediaMill semantic video search engine is bridging the gap between research and applications. It integrates the state-of-the-art techniques developed at the Intelligent Systems Lab Amsterdam of the University of Amsterdam and applies it to realistic problems in video indexing. The techniques employed in MediaMill originate from various disciplines such as image and video processing, computer vision, language technology, machine learning and information visualization. To ensure state-of-the-art competitiveness, MediaMill participates in the yearly TRECVID benchmark.
CHOICE A Thesaurus Browser at Sound and Vision
Over the last few years people at the Netherlands Institute for Sound and Vision spent much effort designing a state of the art thesaurus to support their documentation work, and to populate this thesaurus on basis of several previously existing thesauri and term lists. This thesaurus (the GTAA - Gemeenschappelijke Thesaurus Audiovisuele Archieven) has approximately 160.000 terms in six facets:

1. Subjects,
2. Genres,
3. Person Names,
4. Names,
5. Makers, and,
6. Locations.

Some facets are hierarchically structured, others are flat lists of terms. Some facets in the thesaurus are updated regularly, others are more stable over time. A relational database is used for storage.
The GTAA plays a central role in research and software development within the CHOICE-project. As a pilot project for the CHOICE research program a web application for browsing and searching for relevant terms in the GTAA was built. The user interface was designed to exploit all structure and information present in the thesaurus, or added by CHOICE, in a useful and user friendly manner. Examples are: are broader/narrower term hierarchies, thematic classification of terms, associative relations between terms and cross-facet links. When searching for terms, the use of synonyms and spelling variations provide with additional entry points for searches.
MuNCH
In the MuNCH project MediaMill aims to deliver a working video retrieval system to the Sound and Vision national (video)archive. A first version is planned for at the end of 2006 making use of the engines for video analysis and interaction. In the MuNCH, MediaMill cooperates with Maarten de Rijke on natural language processing, with Guus Schreiber on ontologies of video material, and of course with Sound and Vision, Annemieke de Jong providing the basic material.