Krešimir Ćosić

Marko Horvat, Nikola Bogunović, Krešimir Ćosić

Affective multimedia documents such as images, sounds or videos elicit emotional responses in exposed human subjects. These stimuli are stored in affective multimedia databases and successfully used for a wide variety of research in psychology and neuroscience in areas related to attention and emotion processing. Although important all affective multimedia databases have numerous deficiencies which impair their applicability. These problems, which are brought forward in the paper, result in low recall and precision of multimedia stimuli retrieval which makes creating emotion elicitation procedures difficult and labor-intensive. To address these issues a new core ontology STIMONT is introduced. The STIMONT is written in OWL-DL formalism and extends W3C EmotionML format with an expressive and formal representation of affective concepts, high-level semantics, stimuli document metadata and the elicited physiology. The advantages of ontology in description of affective multimedia stimuli are demonstrated in a document retrieval experiment and compared against contemporary keyword-based querying methods. Also, a software tool Intelligent Stimulus Generator for retrieval of affective multimedia and construction of stimuli sequences is presented.

Marko Horvat, Siniša Popović, Krešimir Ćosić

Multimedia documents such as images, sounds or videos can be used to elicit emotional responses in exposed human subjects. These stimuli are stored in affective multimedia databases and successfully used for a wide variety of research in affective computing, human-computer interaction and cognitive sciences. Affective multimedia databases are simple repositories of multimedia documents with annotated high-level semantics and affective content. Although important all affective multimedia databases have numerous deficiencies which impair their applicability. To establish a better understanding of how experts use affective multimedia databases an online survey was conducted into the subject. The survey results are statistically significant and indicate that contemporary databases lack stimuli with rich semantic and emotional content. 73.33% of survey participants find the databases lacking at least some important semantic or emotion content. Most of the participants consider stimuli descriptions to be inadequate. Overall, 1-2h or more than 24h are generally needed to construct a single stimulation sequence. Almost 84% of the survey participants would like to use real-life videos in their research. Experts unequivocally recognize the need for an intelligent stimuli retrieval application that would assist them in experimentation. Almost all experts agree such applications could be useful in their work.