Project at a glance

inEvent is a 36 month project, comprising 358 person-months of effort, with a total budget of 4.369 million Euros (and a requested EC contribution of 2.928 million Euros). 92% of the budget (345 person-months) is devoted to RTD, 3.8% (6 person-months) to dissemination and technology transfer, and 4.3% (7 person-months) to management. The effort is balanced over the duration of the project.

The following characteristics are reflected in inEvent :

  • Multi-disciplinarity: inEvent is built on a multidisciplinary team, and the final success of the project will rest on the integration of the team. Actually, several inEvent group subsets already worked together in the past in collaborative projects, which should make integration easier, while richer. Finally, there are strong interdependencies between the different tasks and work packages, making the whole much greater than the sum of its parts.
  • Hard science: Besides a compelling and original vision (complex objects structured as “hyper-events”; indexing and relationships between those hyper-events, and presentation of those hyper-events) coupled with integration and user-based evaluation aspects, inEvent is founded on strong research and scientific components in multimedia data mining, intelligent multi-sensory processing, social interaction (in meetings, lectures, videoconferencing, etc) and user-driven design, integration and evaluation.
  • Strong user focus: There are many intriguing ways in which complex objects and “hyper-events” could be developed. In order to guide our research, and to maximize its impact, inEvent will have a strong, user-led focus, reflected in the identified members of its Community of Users, which is expected to expand in the course of the project.
  • Expert-led: inEvent consists of a team with track records and reputations in specific fields; there is mutual trust in one another’s abilities, coupled with an understanding that this respect is earned.
  • Excellent project management: The management structure has been designed to be effective, flexible, and transparent; particular attention is paid to agility and to risk management (as further discussed below).
  • Diversity: Cultural, national, and professional differences in style and approach will be present in the consortium, and we embrace them as a strength and a learning opportunity. This diversity is also strongly reflected by the members of the Community of Users, really coming from different communities.

Project goals

The main goal of inEvent is to develop new means to structure, retrieve, and share large archives of networked, and dynamically changing, multimedia recordings, mainly consisting here of meetings, video-conferences, and lectures.

Several partners of the inEvent consortium have indeed access to (and continuously generate) such large multimedia repositories, which keep being enriched everyday by new recordings, as well as social network data. The resulting resources often share common or related information, or are highly complementary, but also come from different sources, in different formats, and different types of metadata information (if any). Hence, it is still impossible to properly search across those very rich multimedia resources simply based on metadata.

Exploiting, and going beyond, the current state-of-the-art in audio, video, and multimedia processing and indexing, the present project proposes research and development towards a system that addresses the above problem by breaking our multimedia recordings into interconnected "hyper-events" (as opposed to hypertext) consisting of a particular structure of simpler "facets" which are easier to search, retrieve and share. Building and adaptively linking such "hyper-events", as a means to search and link networked multimedia archives, will result in more efficient search system, in which information can be retrieved based on "insights" and "experiences" (in addition to the usual metadata).

Reaching the aforementioned goal requires challenging RTD efforts going much beyond current state-of-the- art in the fields of knowledge representation, audio processing, video analysis, semantics of information, and exploitation of social network information. Ultimately, the main goal of inEvent could thus be summarized as developing new ways to replace the usual "hypertext" links (linking "information" bits) by multi-faceted "hyper-events" (linking different "experiences/insights" related to dynamic multimedia recordings).

Keywords: Networked multimedia events; indexing and searching, multimedia indexing, retrieval, searching, and sharing, social network data exploitation, hyper-events.

Work packages structure

To carry out the project, we have divided the work into six Work Packages (WPs). The relations between the workpackages are illustrated here below.

inEvent is built up of 4 different Research and Technological Development (RTD) WPs plus a WP for Management and a WP for Dissemination and Technology Transfer.
Each work package will be led by one partner (in collaboration with an adjunct leader), selected based on experience and expertise in the area.

The technical components needed for achieving an archiving and retrieval system which is capable of preserving multimedia events are part of WP1 to WP4. WP4 will also assemble the technical components into a single application and make an evaluation of this prototype. All technological components will be developed based on guidance provided by WP4 as user requirements and scenarios. The dissemination work package (WP5) will make sure that the achievements of the different working packages will be exploited and disseminated. WP4 and WP5 will also strongly benefit from the involvement of a inEvent Community of Users, providing usercentric evaluation and feedbacks. The cost of running the Community of Users (workshops, etc), will be taken from the global Dissemination and Technology Transfer. Finally, the management work package (WP6) will oversee the progress of all these work packages and will steer the direction of the individual work packages where needed.

The six Work Packages (WPs) are the followings:

inEventWps

Work package objectives

WP1 - Data Capture and Storage

The objectives of WP1 are first to select, define and enhance multimedia capturing system in the context of
events; second to analyze and select a data storage method and system; and third to define the interface between
the data capturing system and the data storage system, including formats, compression protocols and
resolutions.

WP2 - Analysis and Metadata

The overall aim of WP2 is to automatically analyse the multiple streams of information collected from recordings
of meetings, lectures, videoconferences and symposia, to create higher level metadata that will be integrated
into multimodal hyper-events. The information streams to be analysed including audio & video sequences,
slides, pictures & handwritten notes, and social & web information. To achieve this aim, we have four
sub-objectives:

  • Automatic analysis of audio and video streams, and the development of information fusion approaches;
  • Automatic speech processing: who spoke when (diarization), who said what (recognition), how was it
  • said (prosody);
  • Automatic video processing: visual events, person detection, human motion analysis, feature detection
  • and tracking;
  • Analysis of online social interactions and user-generated (meta)data and annotations;
  • Metadata integration to enable semantic descriptions of hyper-events.

WP3 - Assimilation and Unified Indexing

The objective of this work package is to carry out the research and development of all the middleware components that comprise archiving part of the project, both assimilating original media and auto-generated data, indexing of the events data using federated approach, preparing data for immediate retrieval and supporting

users interaction. This includes scientific innovation to solve the challenges posed by each components objectives, as well as the implementation work required for the delivery of a concrete, functional middleware system. The main package objectives are:

    • Building a scalable system for archiving and maintaning complex digital media content, particularly
    • archiving a complex objects captured in WP1 and constructed in WP2
    • Fusion of all the media and metadata, and construction of complex event internal connections
    • Effective facet-based, federated search and retrieval of complex events
    • Insuring a long-term maintenance and liveness of the archive, supporting users input and collaboration
    • around it.

WP4 - User Requirements, Interaction and Applications

This WP will focus on the user side of the system - user requirements, user interaction and user evaluation. It will begin by generating a set of appropriate user scenarios that can guide the development of the system. Based on these use-cases, it will define the system requirements from the user’s perspective. Part of the system requirement definition will be defining the interaction scenarios of the user with the system. As a result of this work, an easy-to-access, ecient and user oriented ”portal” will be developed, as the user interface of the system. Once the system is complete, it will be evaluated based on these requirements and user feedback. This will assist in optimizing the system, in terms of functionality, usability and effectiveness.

WP5 - Dissemination and Technology Transfer

WP5 is concerned with dissemination, technology transfer and IPR strategy. The main objective of WP5 is to disseminate, exploit and communicate the achievements of inEven.

WP6 - Management

  • Planning, organising, directing and controlling the integrated eort to achieve all the inEvent objectives,
    including all the RTD, exploitation, and training objectives described in the present document.
  • Optimisation of the impact of the project given the constraints of time schedule and budget.
EU Project

Networked Media and Search Systems
(ICT-2011-1.5)

European flag

“This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 287872”.