The main objective of EUMSSI is developing technologies for identifying and aggregating data presented as unstructured information in sources of very different nature (video, image, audio, speech, text and social context), including both online (e.g., YouTube) and traditional media (e.g. audiovisual repositories), and for dealing with information of very different degrees of granularity. The multimodal analytics will help organize, classify and cluster cross-media streams, by enriching its associated metadata. A core idea is that the process of integrating content from different media sources is carried out in an interactive manner, so that the data resulting from one media helps reinforce the aggregation of information from other media, in a cross-modal interoperable semantic representation framework. This will be accomplished thanks to the integration in a multimodal platform of state-of-the-art information extraction and analysis techniques from the different fields involved. Interoperability and interactive reinforcement of the data aggregation and a high-level semantic, conceptual and eventive representation will distinguish this proposal from others that incorporate multimodal search.
The resulting platform will be potentially useful for any application in need of cross-media data analysis and interpretation, such as intelligent content management systems, personalized recommendation, real time event tracking, content filtering, etc. The project brings together 5 universities and research centres, a public service broadcaster and a SME providing solutions for the media industry. The real-world necessities of the 2 user partners motivate two strong user cases that have immediate market applicability. We also expect EUMSSI, which covers English, German, Spanish and French, to promote interaction and mutual knowledge among the diverse linguistic communities within Europe.