Hearing It Like It Was

 

Recorded sounds don't often give you much sense of "being there." That's because your ears not only tell you what you're hearing, but also a lot about where you're hearing it. A new recording and playback method developed at the University of California, Davis, puts your head back in the mix, so you can hear it like it really was.

The patent-pending technique, which the researchers have named "motion-tracked binaural" (MTB) sound, captures both the 3-D position of sound sources and the subtleties of natural, ambient sound that other systems don't. On top of that, the system makes it easy to record 3-D sound with off-the-shelf equipment that won't break the bank.

"Conventional audio playback doesn't reflect how you hear in real life," said Ralph Algazi, director of the Interface Laboratory in the UC Davis Center for Image Processing and Integrated Computing (CIPIC). "Your body, the shape and motion of your head and the room acoustics all affect how you hear."

MTB sound could find its way into applications including teleconferences, home theater presentations, training simulators, video games and museum exhibits. Algazi, Richard Duda and Dennis Thompson presented the research leading to the MTB method at the 116th convention of the Audio Engineering Society in Berlin, Germany. The work is supported as part of two awards from the National Science Foundation.

"MTB captures the dynamic cues of head motion with reasonable computational requirements and affordable equipment for recording and playback," Duda said. "For groups of listeners, MTB sound has the advantage over the computational demands of existing commercial systems."

The MTB sound process starts with microphones -- eight for voice, 16 for music -- attached around a ball or cylinder standing in for a human head. During the recording process, the microphones capture the distinctive sounds at each point around the dummy head.

For playback, a listener wears headphones that have a small head tracker attached. The head tracker determines the position of the listener's ears relative to the microphones on the dummy head, and the MTB software computes signals that combine sounds from the two microphones closest to each ear. When you turn your head, MTB repositions the virtual ears and recalculates the audio feeds.

The system can capture the sound of instruments much more fully than a conventional single microphone," Algazi said. "It captures changes in sound and the effects of a room in a way that is much closer to reality."

A laptop computer can support an audience of several dozen listeners who can independently move their heads to locate a sound source, turn to "face" a person speaking and tell when sound sources are nearby or far away. In addition, MTB captures the ambient sounds of the location, so you recognize the echoes in a church or the confines of a conference room.

NSF program officer Ephraim Glinert has experienced an MTB sound demo first hand. "I put on headphones, listened, turned my head in various directions, and so on," he said. "The results were really quite amazing -- some of the best and most innovative work I've seen in the field."

Home-theater surround sound also imitates a 3-D sound experience, but making these recordings is often more art than science. Engineers experiment with microphone locations, multiple audio tracks and a manual editing process to produce the final mapping of sounds to speakers.

Another spatial sound technique, binaural recording, uses two microphones embedded in a dummy head to record sound and does a fair job of reproducing sounds to the left and right. However, it doesn't allow for movement, Duda said, so sounds to the front appear to come from immediately in front of or behind your head.

With MTB sound, the researchers have created a systematic process that extends binaural recording to allow for head movement. The engineers have made sample recordings with musicians from the UC Davis music department and visiting classical and bluegrass musicians.

"I think they have something really wonderful," said Pablo Ortiz, chair of the Department of Music at UC Davis. "The thing that's interesting to me is the way that it records space."

William Beck, a composer of electronic music and lecturer at the music department, said that live recordings could be a major application.

"The 'being there' feel is something people would really like," Beck said.
-- David Hart