Holo-Jazz_1
A Holophonic recording experiment with a jazz quartet
The goal of this session was to test all the knowledge I’ve done on holographic recordings up until this point by recording a jazz quartet. I wanted to set-up the session so that I’d be able to create Stereo, Atmos, and Holophonic mixes. The instrumentation allowed me to further develop techniques for holographically micing the piano and drum set, as well as experiment with micing bass and harmonica! I’ll break down the session in this page.
Go to my itch page to download the app and listen to the holophonic recording!
I want to also thank the amazing musicians who performed/composed the pieces in these recordings;
Anton Kot - Drum Set/Composer of “Here in a Room”
Sora Kim - Piano/Composer of “Waiting for Spring”
Cy Leo - Harmonica/Composer of “A Glimpse Of”
Warren Louie - Bass/Composer for “In the Stu”
A bit about the approach
Before getting into the session, I’d like to talk a little bit about the technique/theory I’ve been using to capture these sound sources, holophonically.
To decide which array to use, I’ve come up with a way to classify sound sources based on the way they radiate sound. There are three classes:
Radiates a “Single Timbre” Omni-Directionally
Violin, Viola, Bassoon, Voice, most smaller acoustic sound sources.
Radiates a “Single Timbre” Multi-Directionally
Piano, B3 Organ, Acoustic Guitar, Amplifiers, sound sources that are big enough to occlude the most important radiating side at some angle.
Radiates Multiple Timbres Omni-Directionally
Drum Set, percussion ensemble, sound sources that are made up of multiple important parts that have little change based on the relative position of the listener.
When I say “Single Timbre” I mean that there is one thing that is a “main timbre creator” and the directionality is created by the shape of the instrument. There’s a bit of an overlap between the first and second classes (You’ll see that in this session). Depending on how detailed you want to be, a violin’s radiation does change based on the listeners angle to the F-hole. However, due to the smaller size of the instrument the directionality is more of a sphere with slight variation. Whereas a piano has a huge lid that separates one side from the other, putting into the second class with a Multi-Directional Radiation pattern.
As far as which array to use, most of the sound sources in the first class can be captured well enough with some kind of inverse-tetra array, like in the Violin/Viola Session from before. The second class uses a Faces of Radiation approach which will be explained more in the session below. The third class suggests the use of a Multi-timbral Holophone array (MtH).
The Session Set-Up
Piano
To the right is a diagram of the piano micing. For the stereo and Atmos mixes, an Schoeps ORTF was placed above middle-C, with L facing the hammers and R facing the foot of the piano.
For the Holophonic micing array, I chose to use a Faces of Radiation approach for the piano.
The Cyan rectangle/rhombus in figures 1 & 1.1 displays the pianos Faces of Radiation. The idea for an instrument like the piano (Radiates a “single timbre” with multi-directional characteristics) is to mic each face in an interesting way (based on their importance/dynamic range) for a binaural reproduction in unity.
In figure 1, you can see the arrays used on each face. The most important faces are the open lid side and the performers side, the other two faces are important to capture so there are no holes in the holophonic image.
For the open lid, a Mic-Line Array was used. The Mic-Line Array was first tested in the Marimba Drum Set Duo session and proved to be an excellent way to capture a detailed dynamic range of an instrument. Due to multiple microphones being pointed at the same face, multiple audio objects were used in Unity. This let the face project a little louder than the others in Unity, the open lid is the loudest audio projection for the piano.
For the performers perspective, a “Holo-ORTF” was used. Which is a way to create a fanned out radiation pattern in Unity by reversing the application of an ORTF in the capture process. Everything in Unity will be rotated 180º related to the real world capture.
The other two faces were each captured with a single mic.
(Figure 1) Piano Micing Diagram
(Figure 1.1) Diagram of a piano’s Faces of Radiation, Marked with the importance of each face.
(Figure 2.1) Drum Micing Diagram: MtH Array
(Figure 2.2) Small Room Ambience Capture System: Ride Cymbal Side
Drum Set
A drum set has multiple timbres that each radiate omni-directionally. For this kind of sound source, an MtH Array was used. The first step in creating a MtH Array is to define the Acoustical Points of Interest of the sound source. The points for this drum set are shown in the image to the left next to each circled microphone.
Each microphone is placed equidistant from a center point and pointed towards their corresponding points of interest.
For more information on this array, check out the MtH Drum Set Session on my site!
Another system was attempted in the drum room as a means of capturing small room ambience. Two Holo-ORTFs were used respectively on either side of the drum set, facing the wall. The idea is that the sound objects in Unity will be placed in an ORTF fashion to give more life to the drum set. You can see the two DPA 4011s in figure 2.2
This system ended up not creating too much of a difference in the holophonic reproduction in Unity.
Bass & Harmonica
This is a good example of how the first and second defined classes above can overlap. I think, for the bass and harmonica, an inverse-tetra could be used, but might sound too distant/ambient for a “studio-style recording”. For classical music, where the acoustics of the room are as much of a part of the performance as the instrument, player, and piece, I believe an inverse tetra is the right choice. For a produced Jazz quartet, something a bit more detailed is needed.
The arrays ended up being kind of a mix of MtH and Faces of Radiation, I’m looking for acoustical points of interest as well as the importance of the faces of radiation each instrument has. The front faces for both instruments are more important than the back so two mics were used in the front and one in the back.
For the Bass, The F holes were treated as points of interest, each one is mixed accordingly. For the harmonica, the whole front side of the instrument was treated as an important face and a Holo-ORTF was used.
Also, thank you to Julian for being my model for the diagrams! He’s a fantastic Engineer who has helped me a lot.
(Figure 3) Bass Micing Diagram: Faces of Radiation
(Figure 4) Harmonica Micing Diagram: Faces of Radiation
This session was recorded so that a Stereo and Atmos mix could be derived from the stems. They are available to listen to on this page below.
Microphone Diagram
Below are videos that go through the design of each array.
Unity Scene
After Mixing the sessions in Stereo and Atmos, I exported the stems with some processing on them as well as individual tracks for FX like Reverb and Delay. I created a scene in Unity with 3D models of the four instruments and placed the sound objects accordingly.
The main difference between this spatialization and the ones from the sessions before is that now I am using “Steam Audio” as the 3D audio engine in Unity. It’s a great free software that does occlusion and directional sound objects. Before, Google Resonance was used, it seems like Google has stopped supporting that project, so the change was made.
Piano
These imaged depict the placement of the Pianos Faces of Radiation Array as well as some piano reverb/FX placed in an ORTF fashion above. The polar patterns of each object are shown as red spheres.
Piano in Unity, Angled View
Piano in Unity, Top View
Piano in Unity, Side View
2nd Degree MtH-Array on Drum Set, Angled View
Drum set/Small Room Ambience Capture
Here you can see how the MtH and Small Room Capture Arrays were placed in Unity.
2nd Degree MtH Array on Drum Set, Top View
Drum Mics + Room Mics, Top View
Bass
Here you can see how the three mic array used on the bass was implemented in Unity.
1st Order MtH Array on Bass, Angled View
1st Order MtH Array on Bass, Top View
1st Order MtH-Array in Unity on Harmonica + Harmonica Verb in ORTF, Angled View
Harmonica
These three images show the three mic array used on the harmonica, implemented in Unity. It also shows the FX Objects placed in an ORTF fashion above the space harmonica, similarly to the piano.
1st Order MtH-Array in Unity, Top View
Harmonica Reverb in ORTF, Top View