Rainer Goebel

Maria Heinze, Lars Hausfeld, Rainer Goebel et al.

An acoustic stimulus, e.g., a musical harmony, is transformed in a highly non-linear way during the hearing process in ear and brain. We study this by comparing the frequency spectrum of an input stimulus and its response spectrum in the auditory processing stream using the frequency following response (FFR). Using electroencephalography (EEG), we investigate whether the periodicity pitches of complex harmonies (which are related to their missing fundamentals) are added in the auditory brainstem by analyzing the FFR. While other experiments focus on common musical harmonies like the major and the minor triad and dyads, we also consider the suspended chord. The suspended chord causes tension foreign to the common triads and therefore holds a special role among the triads. While watching a muted nature documentary, the participants hear synthesized classic piano triads and single tones with a duration of 300ms for the stimulus and 100ms interstimulus interval. We acquired EEG data of 64 electrodes with a sampling rate of 5kHz to get a detailed enough resolution of the perception process in the human brain. Applying a fast Fourier transformation (FFT) on the EEG response, starting 50ms after stimulus onset, the evaluation of the frequency spectra shows that the periodicity pitch frequencies calculated beforehand +/-3Hz occur with some accuracy. However, jitter turned out as a problem here. Note that the sought-for periodicity pitch frequencies do not physically exist in the frequency spectra of the stimuli.

Alexander Kroner, Mario Senden, Kurt Driessens et al.

Predicting salient regions in natural images requires the detection of objects that are present in a scene. To develop robust representations for this challenging task, high-level visual features at multiple spatial scales must be extracted and augmented with contextual information. However, existing models aimed at explaining human fixation maps do not incorporate such a mechanism explicitly. Here we propose an approach based on a convolutional neural network pre-trained on a large-scale image classification task. The architecture forms an encoder-decoder structure and includes a module with multiple convolutional layers at different dilation rates to capture multi-scale features in parallel. Moreover, we combine the resulting representations with global scene information for accurately predicting visual saliency. Our model achieves competitive and consistent results across multiple evaluation metrics on two public saliency benchmarks and we demonstrate the effectiveness of the suggested approach on five datasets and selected examples. Compared to state of the art approaches, the network is based on a lightweight image classification backbone and hence presents a suitable choice for applications with limited computational resources, such as (virtual) robotic systems, to estimate human fixations across complex natural scenes.

Michele Svanera, Sergio Benini, Gal Raz et al.

Deep neural networks have been developed drawing inspiration from the brain visual pathway, implementing an end-to-end approach: from image data to video object classes. However building an fMRI decoder with the typical structure of Convolutional Neural Network (CNN), i.e. learning multiple level of representations, seems impractical due to lack of brain data. As a possible solution, this work presents the first hybrid fMRI and deep features decoding approach: collected fMRI and deep learnt representations of video object classes are linked together by means of Kernel Canonical Correlation Analysis. In decoding, this allows exploiting the discriminatory power of CNN by relating the fMRI representation to the last layer of CNN (fc7). We show the effectiveness of embedding fMRI data onto a subspace related to deep features in distinguishing semantic visual categories based solely on brain imaging data.