Uni Fakultäten Fk. V SFB/TR 31 Groups

Groups

A1 Spatial Auditory Scene Analysis in Mammals

Prof. Dr. Georg Klump
(Georg.Klump@uni-oldenburg.de)

In auditory scene analysis, signals originating from different sources are analyzed separately. Both the representation of the spatial position of the signal source (in the where-pathway) and the representation of structural features of the signal (in the what-pathway) must be linked. How this linkage is achieved is studied in gerbils. Using both psychophysical and neurophysiological methods the responses to compound sounds are investigated in which components originate from the same of different spatial positions. Especially the relationship between spatial position and other possible cues used for auditory grouping, such as harmonicity and the temporal coherence of component sounds, is investigated. Understanding the mechanisms for integration of information for auditory object formation in the what- and the where-pathway will provide for a new approach to segregate signals from background noise in technical systems.

A2 Functional organization of the periodotopic map in auditorx cortex and its role in object recognition

PD Dr. Holger Schulze
(holger.schulze@ifn-magdeburg.de)

The project aims at an understanding of the cortical representation of a number of different stimuli in hearing-situations with two concurring sound objects. In this context, different sound objects may be defined by variations in position in space, stimulus onsets, and spectral content of the sound sources. In a first behavioral approach (in cooperation with project A1, Klump), we will investigate how different two stimuli have to be with respect to one of these parameters to be separated into two sound objects by the auditory system. In a second step, we will try to find neurophysiological (cortical) correlates of these perceptual thresholds. Finally, attentional (top-down) influences of higher cortical areas onto these correlates will be investigated.

A3 Interaction of bottom-up and top-down processes in cortical processing of frequency-modulated signals

PD Dr. Frank Ohl, Dr. Eike Budinger
(frank.ohl@ifn-magdeburg.de), (budinger@ifn-magdeburg.de)

An act of perception recruits several sub-processes, like the constitution of a perceptual object from features or the attentional selection of an object or of features. These are examples of what frequently are called "bottom-up" or "top-down" processes, respectively. Sensory systems have to simultaneously serve both types of functions. The aim of this project is to uncover neural mechanisms underlying the interaction of bottom-up and top-down processes during perception of frequency-modulated tones, a stimulus class for which auditory cortex has been demonstrated to operate as a critical interface between both processes. Using a combination of anatomical (neuronal tract tracing, immunohistochemistry), physiological (single cell recording, parallel multichannel single cell recording, pharmacological manipulation) and behavioral (stimulus detection and discrimination, feature selection) approaches we aim at establishing a model for the interaction between bottom-up and top-down processes which is explicit with relation to (a) its anatomical substrate, (b) the physiological mechanisms, and (c) the computational principles implemented.

A4 Neural correlates of streaming in the auditory cortex of humans and macaque monkeys

Dr. André Brechmann, Prof. Dr. Henning Scheich
(brechman@ifn-magdeburg.de), (henning.scheich@ifn-magdeburg.de)

Speech, music, and many environmental sounds consist of characteristic sequences of discrete acoustic elements. When two or more of such sequences co-occur, the auditory system is faced with the problem not only to segregate simultaneous acoustic elements but also to sequentially bind acoustic elements generated by one sound source. This process is called streaming. The aim of this project is to study the neural mechanisms of streaming in the auditory cortex by using a converging approach with fMRI experiments in humans and electrophysiological experiments in the macaque monkey. As a general paradigm, these experiments will use sequences of alternating tones (ABAB). The influence of variations in spectral properties of the elements, temporal structure of the sequence in combination with specific tasks on neural activity in auditory cortex will be analysed to identify the neural correlates of stream formation in trained monkeys and to clarify the pattern-independent involvement of the left hemisphere in humans shown in previous fMRI-studies.

A5 Physiology of Object Formation in the Bird Forebrain

Prof. Dr. Georg Klump
(Georg.Klump@uni-oldenburg.de)

Acoustic signals provide the most important means for birds to communicate. The songbirds have evolved a complex set of communication signals that is learned in a similar way to our speech learning. To communicate in the cacophony of the early morning dawn chorus birds must be able to achieve a performance in auditory scene analysis that is similar to the performance of humans. Many parallels in the perception of sounds in auditory scene analysis tasks have been found between birds and humans in psychoacoustic studies. The aim of the current project is to investigate whether this is due to similar neural processing mechanisms. Auditory object formation will be studied with harmonic tone complexes and in the integration of sequential signals into auditory streams comprised of the sounds originating from one source.

A6 Processing and recognition of the temporal pattern of acoustic signals in the auditory system

PD Dr. Peter Heil
(peter.heil@ifn-magdeburg.de)

The temporal envelope of acoustic signals constitutes one of the main features used for auditory object formation. The project aims to examine mechanisms of the recognition and classification of auditory objects which are based on an analysis of temporal envelopes. The recognition performance is presumably relatively independent of overall sound level, since due to source- and context-variation the sound levels from the same object can vary at the receiver. Psychoacoustic and neurophysiological techniques will be used to characterise neural mechanisms underlying the recognition of auditory temporal patterns and their possible level independence.

A7 Temporospatial imaging of auditory selection and audio-visual integration during language comprehension

Prof. Dr. Thomas Münte
(muente@helios.med.uni-magdeburg.de)

The selection of an auditory message from many competing messages is a complicated problem for the auditory system. The goal of our project is to delineate the temporal and spatial aspects of the neural correlates of information selection in multi-speaker-situations. To this end multimodal neuroimaging (fMRI; event-related brain potentials) will be used. In a second part of the project, we will use imaging techniques to characterize the neural correlates of audiovisual integration during speech perception, as it is known that lip-movements are used even by normal listeners in noisy environments.
A specific methodological aspect of the first part of the project is the use of a probe-stimulus technique: In addition to attended and unattended speech messages coming from different positions in space task irrelevant probe-stimuli will be presented that are manipulated to share different features with the speech messages. Thus, we hope to determine which features of a speech message are used for selection in a complex auditory scene. In the second part we use brief video-segments that are manipulated for congruency of the auditory and visual message.

A8 Neural correlates of audiovisual temporal integration

Dr. Tömme Noesselt, Prof. Dr. Hans-Joachim Heinze
(toemme@helios.med.uni-magdeburg.de), (heinze@neuro2.med.uni-magdeburg.de)

This project investigates the cognitive and neural mechanisms underlying the perception of audiovisual synchrony. Behavioural and psychophysical measures are combined with both high temporal resolution (Magnetoencephalography) and high spatial resolution (functional magnetic resonance) brain imaging techniques. First we attempt to identify the temporal neural dynamics and neuroanatomical substrates of the cognitive processes underlying audiovisual integration. Second, we will investigate the functional properties of these areas, determining those which compute audiovisual synchrony automatically, and those which can be modulated by adaptation. Third, we attempt to determine how the manipulation of simple stimulus parameters (e.g. brightness) modifies the neural processes underlying audiovisual integration. For example, since brightness changes alter the arrival times of visual information in the isocortex, brightness manipulations may reveal how the brain integrates information across the senses despite changing cortical arrival times. Together, the results of this project will significantly broaden our understanding of the cognitive and neural mechanisms of multisensory temporal integration.

A9 Top-down modulation of auditory brain activity via memory representations in humans

Prof. Dr. Christoph Herrmann
(Christoph.Herrmann@Nat.Uni-Magdeburg.de)

The processing of auditory stimuli within the human brain not only depends upon physical properties of these stimuli but also upon cognitive processes which represent top-down control. In this project we plan to investigate how the existence of memory representations influences auditory processing. In addition to event-related potentials we will analyse 40 Hz oscillations which have been demonstrated to be important for memory processes. We will also test whether memory representations might be used for frequency-selective attention. In addition, we want to investigate whether resonance frequencies could be the mechanism underlying such top-down modulations.

A10 Auditory Memory and Sound Pattern Recognition in Songbirds

Dr. Ulrike Langemann
(Ulrike.Langemann@uni-oldenburg.de)

Acoustic events leave memory traces that may be recalled immediately, e.g., in order to compare recently perceived sounds with preceding sounds. This kind of "echoic" representation may fade away or may be overwritten rather quickly. Alternatively, memory traces may persist for rather long time periods, enabling the auditory system to recognize and remember sound patterns (e.g., communication signals) acquired and stored as "templates" earlier in life. The representation of sound patterns in auditory memory will therefore constantly change and will influence the processing of newly arriving acoustic signals and the analysis of auditory scenes. In the current project, songbirds provide an animal model to evaluate how processing of acoustic signals relies on auditory "echoic memory" or "template memory".

B1 Modelling the signal processing in the auditory scene analysis of normal-hearing and hearing-impaired people as well as of cochlea-implant users

Prof. Dr. Dr. Birger Kollmeier
(birger.kollmeier@uni-oldenburg.de)

The interpretation of auditory scenes in a real acoustical environment is a complex, poorly understood ability of our auditory system, which is significantly disturbed in hearing-impaired people. In particular, it is unclear how much the "distortion" effect of the (peripheral) auditory transformation (which constitutes the "bottom-up" process direction) contributes in relation to the defective (central) cognitive processing (which constitutes the "top-down" process direction). As a long-term goal of this project, psychoacoustic and physiological (EEG) experiments as well as quantitative models of the "effective" signal processing of the hearing process shall quantify and model the auditory processing deficits that lead to an altered scene analysis/object perception in hearing-impaired people. On one hand, the influence of the "distortion" component (e.g. recruitment phenomenon in hearing-impaired people) on object recognition will be studied (i.e., the consequence of impaired auditory functions on bottom-up processing). On the other hand, the influence of a (defective) cognitive interpretation of the internal representation of these acoustic patterns will be studied (i.e., consequences of impaired top-down processes). A better understanding of auditory processing deficit in hearing-impaired people will help to clarify the role of these processing principles in acoustic scene analysis and will also help to improve "intelligent" hearing aids.

B2 Models of sound source localization and source separation based on the statistics of monaural and binaural signal features

Dr. Volker Hohmann
(volker.hohmann@uni-oldenburg.de)

For the localization, identification and separation of sound objects, the human auditory binds several monaural and binaural signal-related feature streams in a largely unknown way to form a consistent image of the acoustical environment. The aim of the project is to clarify and model this process by using statistical methods of source coding and separation in combination with psychoacoustically and physiologically motivated signal preprocessing stages. Binaural hearing in realistic listening environments including noise and reverberation is particularly considered. The specific assumption is that a-priori knowledge on possible sound sources is necessary to solve the separation problem in these conditions characterized by ambiguous and/or partially masked information. Possible applications of the models are speech processing in hearing aids and automatic speech recognition.

B3 Psychoacoustical modelling of auditory object perception in humans

Jun.Prof. Dr. Jesko Verhey, Prof. Dr. Torsten Dau
(jesko.verhey@uni-oldenburg.de), (tda@oersted.dtu.dk)

We are continuously bombarded with a mixture of sounds from different sound sources. The first step of the auditory system is the analysis of the frequency content of the sound with a bank of overlapping bandpass filters. However, since the spectra of different sound sources usually overlap it is impossible to separate them by the analysis of the spectrum alone. Across-frequency processes are required to bind the frequency components into different auditory objects. The characterization of the mechanisms underlying object formation with psychoacoustical experiments and modelling is the main goal of the current project. One aim of the project is to characterize the processes related the binding of comodulated frequency components, i.e., which show similar level fluctuations over time, into an object. It will be investigated over which frequency range this process can be observed using the comodulation masking release (CMR) paradigm. The interaction of CMR and other object binding mechanisms such as pitch or binaural cues are measured in order to quantify the contributions of active (top-down) and passive (bottom-up) processes. Our long-term objective is to develop a model of a realistic internal representation of the features of objects in complex acoustical environments On the basis of the experimental results.

B4 Subjective dimensions of audiovisual object formation

Prof. Dr. Hans Colonius
(hans.colonius@uni-oldenburg.de)

Detection, discrimination, and identification of objects (for example, letters or words) presented in several modalities at the same time are often faster, or more accurate, than for objects presented in a single modality. A first hypothesis to be tested is whether there exist crossmodal features, that is, features that only occur when an object is presented in more than a single modality and whether these features may underlie the observed superior processing of crossmodal objects. Investigation will first be confined to the formation of visual-auditory objects, in particular, written or spoken letters. Empirical testing of the hypothesis is based on methods in the theory of dissimilarity developed by Colonius in collaboration with E. N. Dzhafarov (Purdue U.) that allow construction of subjective metrics on arbitrary stimulus spaces from different data sets, for example, from matrices of confusion frequencies. Moreover, the time course of visual-auditory object formation will be studied through the analysis and modeling of saccadic responses in various discrimination and identification tasks, in collaboration with A. Diederich (International University Bremen).

C1 Scalable Compression of Audio- and Speech Signals using auditory measures

Prof. Dr. Alfred Mertins, Prof. Dr. Dr. Birger Kollmeier
(alfred.mertins@uni-oldenburg.de), (birger.kollmeier@uni-oldenburg.de)

The aim of this project is to incorporate new findings and models for the hearing system into algorithms for the efficient coding of audio and speech signals. In doing this, on the one hand, the properties of efficient coding techniques have to be considered in view of scalability, progressiveness and usefulness in future heterogeneous networks. On the other hand, novel experimental results and models from other projects within the SFB have to be considered and included in the coding algorithm. In particular, objective measures for audio and speech quality that stem from actual psychoacoustic experiments are to be optimized, evaluated and used actively during coding.

C2 Complex Auditory Features for Robust Automatic Speech Recognition and for Modelling Human Speech Perception

Prof. Dr. Dr. Birger Kollmeier, Prof. Dr. Alfred Mertins
(birger.kollmeier@uni-oldenburg.de), (alfred.mertins@uni-oldenburg.de)

Despite of technological advances and an enormous increase of computational power, there is still a large gap in performance between normal-hearing native listeners and artificial, automatic speech recognition (ASR) systems. This is most drastically encountered in adverse acoustic conditions and prohibits automatic speech recognition (ASR) technology from being widely used. The project targets at an enhancement of robustness of ASR systems by improving the preprocessing and representation of acoustic input information, which should closely resemble the corresponding signal processing stages in the human auditory system. Furthermore, relevant models of auditory signal processing will be evaluated by integrating them in ASR applications. Thereby, hypothesis for these models based on perception experiments with humans and animals can be verified. A close interaction between experiment, development of models and technical applications is planned.