Holophonic Room Acoustics
The goal of this session was to test concepts and theories regarding capturing ambient room acoustics for XR (holophonic) listening environments. A jazz trio was recorded in NYU’s Lowe theatre, a medium sized hall with a nice dark timbre.
This research was completed with help and knowledge from colleagues Parichat Songmuang and Ying-Ying Zhang.
Two systems (based on a Hamasaki Square) were used to capture holophonic ambience:
Hamasaki VR array with Sennheiser Twins set to figure-8
Hamasaki VR array with Sennheiser Ambeo ambisonic mics
If you want to just experience the recording, follow this link to download.
Drum Set
The drum set was set up as a typical 2nd degree MtH Array. Although this time I wanted to put more focus on the side-to back image. Which is why the rear microphones (KM-81s) are to the sides of the drum set rather than directly behind their back.
The thought process behind the rear mics: If I were to think of the back of the drummer as an entire face of radiation, I’d want to capture that face at an angle so that the spread can be wider when rippled 180º in unity. So instead of pointing a single mic, directly perpendicular to the drummers back, it’s better to place two at some kind of angle that will fan out into a spread when placed in Unity.
2nd Degree MtH Array on a Drum Set
Top of Guitar Amp
Front of Guitar Amp
Guitar Cabinet
You can see a similar thought process here when miking the front of the guitar cabinet. Instead of miking it with a single mic head on, I’m giving the audio object more presence in Unity by capturing its spread at an angle.
If the front of the amp were captured with a single mic, then the listener would hear little difference in the timbre of the amp walking back and forth in front of it in unity. In my opinion, this experience is unrealistic and not 3D enough to be called a 3D recording.
By capturing the front of the amp at an angle with two mics, it achieves the necessary width to have a realistic sonic presence in Unity.
Saxophone
To capture the saxophone, an inverse-tetra was utilized. It was supplemented with a spot mic that was attached to the sax, just in case the player decided to leave their spot (Which we gave them freedom to do later in the session as a means of testing the ambience capture systems).
Inverse Tetra Array for Saxophone
Holophonic Ambience Capture Systems
Below, is an image of the two arrays employed for testing capturing 3D Ambience. The main difference between the two systems is the Ambisonic system is capturing ceiling and floor reflections while the system made up of Twins are only capturing the wall reflections. The use of these arrays in unity are to mask the possibility of mis-localized reflections of instruments (this happened last time I recorded in a reverberant space), as well as provide a more realistic sense of space in the recording.
Photo of Ambience Arrays
A Hamasaki square was formed around the players, two mics in the audience and two in the rear of the stage. We created two squares on top of each other, with the mics placed as coincident to each other as possible. One square was made out of Sennheiser Ambeo Ambisonic microphones while the other was made up of dual capsule Sennheiser MKH-800 Twins.
To the right are diagrams that depict the ways the arrays were to be implemented in Unity. Click on an image to expand it.
Image depicting the implementation of the Sennheiser Twins in Unity (Click to Expand)
Image depicting the implementation of the Sennheiser Ambeos in Unity (Click to Expand)
As of this moment, only one of the Stereo Mix is available to Listen to below. More Atmos Mixes are being created right now.
Below is a video that explains each array in detail.
Unity Scene
Drum Set
Here you can see the placement of the directional Steam Audio objects on top of the 3D Model of a Drum Set. Notice how the rear facing image is made up of two objects at an angle instead of a single object facing backwards.
Unity World with Drum Set Audio Objects
Guitar Audio Objects
Guitar
The placement of the guitar amp’s audio objects were done similarly to the Bass and Harmonica’s from the Holo-Jazz Session. In a tight, triangular cluster.
Saxophone
The saxophone audio objects were placed in a tetra formation on top of the 3D saxophone model. Read more about this in the Violin & Viola Session.
The Saxophone’s Inverse-Tetra placed in Tetra configuration in Unity
Twin Ambience Array Unity Placement
Hamasaki VR Array (TWINS)
This is the Unity application of the Twin array implementation diagram shown earlier.
Hamasaki VR Array (Ambisonic)
This is the Unity application of the Ambeo array implementation diagram shown earlier.
After experiencing the two arrays in Unity, it is clear that the ambisonic approach to recording 3D room ambience is superior to the Bi-Directional array. The addition of the height and floor reflections add so much to the realism of the rooms acoustics.
Ambisonic Ambience Array Unity Placement