Cognitive Perspectives of Multimodal Signal and Event Processing

Cognimuse Project
Greek Secretariat of Research & Technology, European Funds for Cohesion Policy (ARISTEIA I call)
December 2013 - December 2015

Motivated by the grand challenge to endow computers with human-level abilities for multimodal and multisensory information processing, COGNIMUSE undertakes basic research in modeling multisensory and sensory-semantic integration via a synergy between system theory, computational algorithms and experimental research on human cognition. The project focuses on integrating three modalities (audio, vision and text) at both the signal and semantics level for detecting salient perceptual events in hollywood movies. The objective is the creation of a movie-trailer generation algorithm which will incorporate salient events of the movie as indicated through the integration of linguistic and perceptual information and its experimental validation through cognitive experiments on saliency with high level (complex) stimuli.

Consortium Members:

  • National Technical University of Athens, Department of Computer Science, Computer Vision, Speech Communication and Signal Processing Lab

In collaboration with:

  • Cognitive Systems Research Institute (Greece),
  • Institute for Language and Speech Processing/Athena R.C.