Auditory and Visual Sensations

Temporal and spatial sensations. Temporal auditory sensations include auditory qualities such as pitch, timbre, loudness, and duration, which are modeled using features extracted from the temporal autocorrelation function (ACF) of the sound. Neuronal responses related to these qualities are mainly associated with left hemisphere auditory regions of cerebral cortex. Spatial auditory sensations include the perceived direction of sound, its apparent size (apparent source width, ASW) and its subjective diffuseness deeply related to envelopment. These spatial aspects of sound perception are extracted from the interaural crosscorrelation function (IACF), which takes into account the differences in the sounds that reach the two ears. Cortical neuronal responses related to spatial hearing are mainly associated with auditory regions in the right hemisphere.

Thus, the "primary sensation" evoked by a sound in some space can be divided into two categories: temporal sensations and spatial sensations. Any other subjective responses of the source sound signal and the sound field can be described in terms of temporal and spatial factors. Part I of the book describes experimental results from basic research in acoustics, psychoacoustics, the psychology of auditory preferences, and auditory neurophysiology. Applications include optimal design of concert hall and opera house acoustics, architectural acoustics of speech intelligibility and musical expression, as well as the perception of noise and its annoyance.

In Part II, temporal and spatial sensations in vision are described in terms of similar kinds of correlation-based representations. A seamless, general theory can be established for temporal and spatial aspects of vision that includes subjective preferences for visual environments. Many audio-visual analogies are drawn. A typical temporal sensation of vision is the "pitch" of a flickering light, for which the missing fundamental is perceived in a manner highly analogous to its acoustic counterpart. Visual spatial sensations include contrast, regularity and coarseness. A spatial "vibrato" in a drawing is demonstrated as an application of the theory in the visual arts.

The reader is given a highly fleshed out strategy for architectural acoustical design that is grounded both in psychology and neuroscience. Few other works are as ambitious in attempting to understand and explain such a wide range of perceptual phenomena. This human-centered design approach can be applied to any design practice in which the goal is to optimize the experience of the design product by its users. The proposed theory shows how a unified science of human perceptual experience and preference might be both possible and useful.