SFC Guest Blog: W, X, Y, and Z
W, X, Y, and Z
Greetings, music lovers! As both creators and capturers of music, we are always striving for either the most realistic experience for the listener or the most surrealistic from our recordings. Heck, sometimes we may want both. For many years, surround sound has been pursued as a way to replicate the sounds that engulf us from all directions in real life. Reproducing that enveloping environment has usually been exclusive to surround sound delivered by multiple speakers in a listening room that was specially designed for that purpose. However, there are some recent advances in one area of audio that I believe may revolutionize how we all experience and create music. One of those areas of advancement in particular is Virtual Surround, particularly for headphones. There are several names and approaches associated with this experience such as spatial audio, 3d audio, VR audio, Ambisonics, and more, many referencing specific techniques of recording and mixing audio.
Like many of you, I have been thrilled to go into movie theaters and experience audio via advances in surround sound technology. Currently, as effects, dialog, and sometimes music come at me from all directions, I enjoy a much more immersive experience. However, many of you are possibly producers, engineers, or recording artists that capture or create music more than you work on “production sound” (i.e. dialog, sound effects for film or tv, and beyond). So, let’s talk about surround for music.
In past years, my personal view of traditional surround sound, like many, was as more of a "specialized" delivery format. This was because experiencing our creations in surround required specific surround speaker systems, room design, and enough space to plop down in the very middle of it all to properly get the effect. To me, this has been the most limiting factor for having some sort of surround sound experience as a regular routine for listening to music. This delivery method via surround speaker systems seems to still be the best scenario if the listener can sit in the right spot in the right room. Imagine if you could get a similar experience wearing standard or multi-speaker headphones! Headphones have recently become commonplace in many parts of modern society. Cell phones used the world over often come bundled with earbuds for listening to audio. With headphones, you can be facing any direction and sitting in any place and experience sound in front of you, to the left or right of you, behind you, or above you! Virtual Surround opens up a similar experience to that of traditional surround sound to a MUCH larger audience.
Recently, in many sectors of the audio industry, things have come a LONG way in the research and delivery of psychoacoustic information to our ears and brain. There are new advances in how we understand and recreate how audio cues trigger decisions that make us think that sound is in front of, above, below, left or right, or behind us. When you think about it, we do have but two “ear holes” for capturing this data to send to our brain for interpretation. We do not have a surround sound set of holes in our head that listen for a sound in each direction and report what it sounds like in different parts of a space. Our brain seems to be receiving actual directional source information into those two left-and-right-only funnels and re-interpreting it into directional info again. That said, I have learned that things like our ear shape, the dimensions of our head, our shoulders, and other anatomical structure can greatly contribute to our sense of sound localization (from where we think a sound originates in our environment). Scientists have been studying HRTF (head related transfer function) measurements to find profiles that will help deliver the most realistic experience.
Multi-channel audio can be interpolated and processed even for stereo delivery into even common phone-based listening devices that make the listener feel that sounds are all around them. One way this is possible is by using dedicated software or plug-ins for encoding and decoding recordings such as ambisonic B-format audio that uses four channel WXYZ recordings. In simple terms, WXYZ describes audio that captures four specific directions of audio instead of just XY, or only left-right. This is often done with four microphone capsules that capture audio happening in a space from one spot, but facing four different directions, often built into one microphone. There are many other tools currently available with different approaches and they are gaining traction fast. If you think about it, when a sound reverberates around a room, we often hear reflections of the sound all over the room, including behind us and from the ceiling above us. In traditional stereo mixing, we may add artificial reverb to a sound via hardware or plug-ins, but miss the reality of how a real room moves sound ALL AROUND US, not just left or right or in a virtual center. I compare this to how we can watch a movie on a small screen in front of us and are constantly aware that we are not actually in the scene because we can plainly see where the movie environment ends and the room we are watching it in begins at the edge of the screen.
At one time, monophonic recording and playback was the norm. Luckily, some forward-thinking engineers started recording in stereo anyway, believing that people would eventually have left and right speakers in their homes and automobiles. Those records that were first released in mono, later had the ability to be remixed and mastered for stereo listening because stereo recordings were captured of the performance. This forethought opened up a new experience closer to that of the performance in the room, regardless of the way people were listening to music at the time. I believe there is a good reason why so many companies like Dolby, DTS, Waves, Harpex, Core Sound, Sennheiser, and others are pushing forward on the mass availability of the Virtual Surround listening experience. I believe it is something to explore and get behind as a deeper way to create content and a demand for that "more immersive experience." It may have the impact that stereo recording and playback did for recorded music.
It has to be said that listening levels in headphones truly have to be monitored for the hearing health of everyone listening in headphones, but that is part of another awareness initiative being developed by audio professionals in the Recording Academy. Virtual Surround technology can deliver a spatial experience that, while not quite as realistic as a surround speaker array, is still deep and wide in ways perhaps beyond standard stereo panning, delays, reverbs, and phase manipulation techniques used in most common mixing approaches. Recent Virtual Surround listening experiences have really changed my opinion of the viability of surround for the enjoyment of music by most everyone. In my opinion, the progress of surround IN HEADPHONES or even stereo speakers could be a new chapter in the way consumers can experience, enjoy, and even demand virtual surround mixes of music and audio that we create. I highly recommend you check it out and start making your own surround recordings and/or binaural mixes!
McKay Garner is an engineer, producer, and relentless explorer of sound and music, specializing in part in hacking existing instruments and musical interfaces to make new ways to create or control sound through countless trips to the hardware store. McKay is a Voting Member of The Recording Academy, a member of the San Francisco Chapter Board, and Chair of the SF Chapter P&E Wing Committee. Get to know him at http://www.mckaygarner.com