Abstract
Augmented audio reality (AAR) is a mixture of virtual reality and the natural environment. AAR enhances the acoustics of a real world environment with additional audio information. A hearing aid is a simple example of AAR. We describe an experiment in an indoor environment with up to 16 simultaneous voices (such as in a crowded room) and up to 48 microphones in arbitrary configurations. We develop array signal processing algorithms to train the system, and separate, localize, track, enhance and zoom in on the voices. Effective audio zoom algorithms and natural gesture control will be useful in many contexts in of themselves. Their combination will enable a “super” hearing aid which can be applied to restore impaired hearing, guide visually impaired people, as well as enhance natural hearing. The longer-term objective is to develop algorithms to control the location on which the array is focused using head and eye gestures. The listener can simply look at the location from where s/he wants to hear the audio, thus controlling the audio zoom via head and eye gestures.