Summary

2 part project: 

Part 1 30 minute Soundscape AudioBook experience called Grianstad - using Dolby Atmos

Part 2: Mixing Project: 6-8 mixes spanning different musical genres/stylistic approaches from jazz to pop and metal. 

20 mins sampler of different musical genres.

1. Pop - ‘Bad Tonight’

2. Beats ‘ Brave New World’

3. Acoustic/Indi - ‘Becoming Nobody’

4. Jazz Trio - Jesper Buhls Trio

5. Techno - ‘Gone’

Here is a link to all the audio files - these files are all rendered in 3rd order AmbiX (16 channels). It is recommended to listen to these in a room with multichannel speaker configuration, whilst binaural d will work, it kind of defeats the purpose of it :) the binaural version of Grianstad lives on my bandcamp page for everyone to download for free.

It is also worth mentioning that these are unmastered. I purposefully left these without final adjustments to consider the mix in its raw state. Thus there are some slight discrepancies in the levels between the songs

SET UP:

When working in ambisonic the routing is often the most confusing thing in the first stage. It is advisable to establish a template and use this as your starting point. I worked off a template built by the set up advice of IEM in their blog post.

I also utilised the template given to me by Thomas Koch from UDK/Fraunhofer HHI Timelab (thanks!)

MONITORING:

Headphones v Speakers - realities practicalities. 

Although the practicality of spatial mixing in binaural and headphones a reality, I feel pretty strongly that as much as possible the mixing should be performed in a room with speakers. My approach was to perform the basic set up and routing in binaural, which doesn’t really required much listening. Then continue with the bulk of the mix in a purpose built immersive mixing room. I also checked the mixes on different speaker configurations. Final adjustments and checks were made again binaurally before I rendered out the mixes.

Decoders - binaural IEM vs Sparta

The Sparta decoder clearly is the better sounding decoder. However these are of course not the only ones. A potential consideration for future research is to investigate the range of decoders, and explore with more detail the field of HRTF.

Beyond selecting an encoder however, there are a few important considerations that I would like to discuss. Firstly it is really important that you monitor in the ambisonic order that you are going to render. The issue at play is the tonal changes that occur between different ambisonic orders. Which makes sense when we consider that the higher orders. are becoming sensitive to more details and the more subtle details of a sound relate to their harmonic content. I also suspect that the nuance of movement is enhanced, which would relate to the resolution of the amplitude variations? Similar to the manner of bit depth? I have not researched this so this is just a hunch - indeed this needs further research.

Secondly it certainly helps to take the time to audition and find the sofa files that work best. This is best done whilst mixing with speakers and transition to binaural and back to find the sofa file that gives the most similar representation.

Sub matters

Perhaps this is very obvious but I found that mixing with and without a sub really impacted my mixing decisions. When I first started this project I was mixing in my room which has a quadraphonic set up and no sub speaker. Once I was able to check these in a room with a sub, it really impacted my mix decisions. In particular as I establish ‘foundation’ in my mix many of my decisions were impacted. Mainly of course it related to those sources with low frequency information, but it also had a knock on effect in that I made specific decisions about frequency content that then impacted my frequency approach to low mid sounds and so on.

KEY PLUGINS USED:

I purposefully wanted to limit my plugin use. It is very easy to get caught constantly trying out new plugins, and although this is also useful, I wanted to focus on mixing workflows and spatialisation, so after experimenting initially with all of the different packages I found a combination that worked well for me and more or less stuck to this throughout the mixing. Different mixes utilise different ambiX plugins and explore features differently and I will speak about these more specifically, but the core that were always present in my mixes are listed below.

• IEM for encoding

• IEM for master ambiX bus processing limiter, eq, multiband compressorand visualiser

• Sparta decoder 

• Reverb, IEM fdn reverbv and Noisemakers Ambi HD

• Delays: IEM Dual Delay and Blue Ripple Spatial Delay

• I restricted myself to equalisation and compression so as not to get into ‘remixing and also limited which I could use.

• I explored also the room encoder and directivity shaper, although I found that in the context of the ‘songs’ that i was working with these tools were less useful.

Workflow

Workflow is key for me. I feel as though I understand something once I feel a stable workflow emerging.

Below is an overview of my basic workflow:

1. Load up the ambiX template - which is based on the IEM suggestion for routing

2. Import audio stems from the stereo mix export

3. Consider the groupings of the stems - i.e. Drums, Perc, - what could be grouped together in a multi-encoder?

4. Batch change the channel count (script!)

5. Batch routing to main bus

6. Decide on ambiX order and check routing

7. Set up encoders

8. Consider the spatial placement of individual items- this is a BIG part of the spatial mixing that shouldn’t be underemphasised. Simple in action but important for laying out the space and starting to get meaningful interactions across space.

9. At this point quite often I realise there are some elements that might need further adjustments to source character through frequency and dynamic profile adjustment.

10. Consider the spatial planning and set up motion/gesture characteristics on specific parts (which means automations around the space).

11. Consider the use of reverberant spaces - set up different rooms to further create a solid sense of space and spatial placement - combination with using stereo reverbs and delays to create specific experiences of sounds 

12. Address the bass management and ensure that only sounds with sub  intention have low frequencies (go into more detail about approaches to LFE management) 

13. Consider the intention of the song and pay attention to details - transitions lead moments and ensure that these are not sacrificed due to the spatial approach. 

14. Apply master bus processing and start to finalise the mix. 

15. Render: being careful of the Routing settings and channels - I always render ambiX as well as binaural in my session.

16. Check the render!

SPATIALISATION - STATIC POSITIONING

Source positioning in spatial mixing are perhaps the most important decisions that I make in spatial mixing. Similarly in stereo mixing when you begin to consider the balances of the elements and create a sense of their positioning largely through panning and fader level. In an ambisonic mix the panning is far more nuanced than a stereo pan, and becomes for me a primary component in the spatial mix.

First of all the mix has to have stability and foundation just as any mix. Making decision on those element which should stay centred, and whether the centring is front or is above or below is one of my first important decisions.

Considering to use a source as stereo or mono is another very important decison. In particular in the current trends in popular music clients increasingly deliver me stems to mix that are all stereo, even kick drums. This is due to the popularity of Ableton as production software (all stems get rendered through the master bus which is stereo. Exporting mono is a separate and time consuming process. So when in the immersive mix, it is really important to mono out mono sources first of all. After this it is really important to consider whether to use sources as mono or stereo as discussed in AES Immersive Sound chapter on immersive mixing (below). When using stereo sources, or a stereo sub-mix bus these stereo sources can create phantom image which may create phase cancellations as the listener walks around the room.

From that point I next consider the elements which will be spatially positioned but will remain static. In this moment it becomes really important to consider the balance between the elements. Interesting inferred movement and rhythms can be created through the relationship between well positioned static elements. An example of this can be heard in the track Brave New World (in the audio files folder ). The percussive elements mid way through the song are static but create a nice sense of movement as your attention is drawn from one to the other.

In working with more traditional or purist formats the spatial positioning is really fundamental to creating a realistic 3D sound stage. The jazz trio mix from the Jesper Bohls trio (also found in the audio files folder ) is a simple mix from the perspective of automations and movement. I find that holding back can actually be one of the hardest things to do when mixing. In this mix I spent quite some time getting the positioning feeling right to me. You can see in the images below the relative positioning of the three main elements in the mix.

The piano is the lead in the trio and so wanted to be clearly out the front. In this mix perhaps it could be argued that the piano is sitting a bit ‘high’ I did experiment with putting it lower but it either felt to wide or it lost its upper transient information that helped it to sit out and feel exciting. So this is what I compromised with. It really gives me the feeling that I am inside the trio.

There was some small adjustments to the frequency of the parts some mid energy increase in the bass, and some low mid clearance in the piano. The drums required a bit more work. i tried to focus the kick and snare with equalisation and some compression. Perhaps the kick is too powerful for jazz purists, but hey I’m not a jazz purist and i love a more impactful thud to the kick.

SPATIALISING - MOVEMENT

Movement is of course an exciting feature in spatial mixing.

“Immersive audio provides a much larger range of possible motion for sound sources. With the inclusion of overhead loudspeakers in particular, the ability for sound to perform “flyovers” creates an enticing new effect, particularly for sound designers.”

— (Roginska and Geluso 2018, pp. 353)

However, at least in my opinion, it is also something that is overused and often misused. At first it is exciting but this excitement is short lasting if it is too much and particularly if the movement has no concept or reason it quickly becomes uninteresting and works against the overall momentum of the piece.

So in saying that, I think very carefully before creating movements. Not all pieces require movement (as demonstrated in the previous post on static positioning).

But lets talk about the fun stuff of movement ! In Reaper there are some great ways to create interesting movement. In particular, due to the really flexible routing capacities of the program, (it means you can route anything to anything) it includes sending an audio signal as a control signal and using that envelope to force a plugin parameter to react. In the context of spatial mixing it means that you can make sounds push each other around in the spatial field. Below is a short video explaining an example of how I used it and the set up

I also wanted to discuss the importance of balancing movement with foundation or stability. In the below video I go into some detail discussing how I considered this as I built the mix for Brave New World. This track needs power and urgency that will require a strong foundation, but also benefits from the use of motion to create some sense of instability. It can be a hard line to walk effectively - hopefully I did ok in this mix!

SPATIALISATION - Depth & Space, Phase & Delay

One of the key elements to creating an engaging mix is space. This is the same for stereo as it is for spatial mixes. One of the attractions to Dolby atmos is the access to more high end plugins such as Logic’s reverbs and delays. They remain working in the channel format however, so when looking for an ambisonic reverb which requires 16 - 64 channels it becomes harder. The IEM fdnreverb is good to a point it works up to 7th order, but it lacks in creating a sense of realism. The noisemakers Ambi Verb (recommended to me by Thomas Koch) was an excellent solution it is limited to 3rd order (16 channels) but this wasn’t a problem for me as my project was to be rendered in 3rd order. The preset IR are very realistic and the range of spaces to chose from is quite diverse. You can also import your own IR’s which is something to try for the future. I found that the combination of the IEM and Noisemakers reverbs gave me flexibility to create interesting spaces.

Considering delay in a 3D environment is quite fun, and I enjoyed exploring some more creative use of the Blue Ripple O3A spatial delay in the below reverb. The format of techno was a great place to play with things a bit more as demonstrated below.

SCRIPTS IN REAPER = Time savers

I was getting annoyed at having to manually set up every source channel as i was setting up my mixes and thought ‘there has to be an easier way’. Then I took myself to the internet and stumbled into the wild world of Reaper action scrip community. Wow! So there certainly is easier way (but perhaps only easier for some people. If coding makes your head hurt then you might not consider this easier. Even though I am by no means a coder, i was able to find some pre-existing code and get it set up.)

The main one I used was the action script to set the channel count from 2 - 64 or how ever many channels. I also used a script to automatically create a send for all selected tracks which was really useful also.

Next of course I’m already thinking - More scripts! What would I want?

The two main scripts that I will look for next are:

• turn of route to main

• automatically put multi encoder on selected tracks  (perhaps this is achievable first by creating a template?)

The key processes of mixing are STILL Frequency & Dynamics

Frequency and using equalisation is still a key component of mixing just as in stereo mixing. I have mentioned various decisions for equalisation and dynamic processing in earlier blog posts, but I think it is important enough to spend more focussed time on this topic. If you think that spatial mixing is just using a spatial panner then i feel that the mixes will be lacking. That said, of my rules in this process was to do as little re-mixing as possible - meaning that I should honour the character of the stems. That said, I still found it necessary to perform compression and equalisation to different elements in order to get them to sit correctly in the new spatial mix. In order not to get to carried away with this i limited myself to 4 plugins - ssl channel strip, dynamic EQ, fet compressor, optical compressor.

These were of course performed before the encoding. One of the main equalisation that I found myself doing was low mid reduction - perhaps with more space there is less masking? also another thought I had was that due to the need for more precise positioning the low mid which dull the directional perception became more problematic.

Low Frequency & SUB

Thinking about low frequency is really important in spatial mixing. It is actually more critical in ambisonic mixing as opposed to channel based& atmos, because in ambisonic the low frequency is calculated and sent to the sub - so if you haven’t cleaned up the low frequency it can create problems. In the traditional channel based format you would create an LFE bus and this made it very clear what information is being sent there. Im not sure that I like the ambisonic way, and am still considering the best approach to working with this.

For these mixes my approach was to be very thorough and cut out everything low that wasn’t specifically intended to carry low frequency information (probably good practice anyway.

Dynamics

One of the reasons that I love spatial mixing is that the ‘loudness wars’ debate becomes redundant. With so many speakers you don’t have to worry about loudness. In fact if you push it too much it really spoils the mix. I have heard spatial mixes that were mixed according to the stereo loudness approach and I had to leave the room it was awful. So the renewed importance for dynamics is exciting for me. When considering the dynamics however it does also rely on the stems that you receive. If the stems have already been over processed and have dynamics greatly reduced you will encounter challenges in the mix.

Reflections & Next Steps

Key Takeaways

• Set up the monitoring as you are going to bounce. When listening in binaural the decoder really matters. The ambisonic order also really make a tonal difference

• Using templates and action scripts really helps for a speedy set up , which is interesting to me because I normally don’t like using templates in stereo mixing.

• Things which are more spacious in design (less stems!) work well and conversly the more detailed and full the arrangement is the harder it is to create a sense of space because all ‘spaces’ are filled with sound’ there is not much negative space to gain a spatial perspective.

I performed more than 5 mixes, some were less successful than others (perhaps I should include them as well in the audio folder) those mixes which were really busy and really compressed before bringing into the spatial mix performed poorly.

Some more key takewayas relating to the spatial presentation:

• More sparse tracks like bad tonight, and jazz trio relied quite heavily on a static approach to spatialisation. Too much movement, perhaps any movement, would destroy the intention of the music. 

• Establishing stability with parts that don’t move, in particular drums and bass is quite important for me. It leaves opportunity for other rhythmic parts to move, thus creating interest and dynamic in the spatial field.

• Having a concept (spatial) or a design approach direct the mix and avoids aimless spatialisation and excessive movement.

Do Different Styles Require a different approach?

This was one of my central questions throughout this process. It seems quite evident to me that different approaches are necessary. If we compare two extremes from my mix examples such as the jazz trio and the techno track, vast differences in approach are evident. The jazz trio required very detailed and considered spatial placement with the source panner. There was also some use of equalisation to help reinforce the positioning, but beyond that not much else.. The techno track was a moment where I could really have fun with all the spatialisation tools. room encoder, directivity, delays and reverbs. It was also a great moment to use different layers of automations. It is also the time when more flashy approaches such as my newly discovered technique of ‘pushing one sound around the space by the energy of another’ can find an appropriate outlet.

Reflections on Mixing Reaper AmbiX vs Dolby Atmos

I approached mixing in a format that I usually leave for experimental/electroacoustic, and I created an electroacoustic score in a format popular for mixing songs spatially - considering these formats using them against my natural rhythms was an interesting interruption and brought into focus the relationships and differences.

The aesthetic and the outcomes of ambiX and Dolby atmos are for me really similar. They both encourage sonic landscapes which honour and create greater sense of the listener being in the space and the nature of the format encouraging dynamics further enhances this. It makes listening to music again an enjoyable experience for me. Much of the workflow and the techniques used in the Dolby atmos soundscape mix were the same or similar ones used in the ambiX mixes. It is for this reason that I didn’t also do separate analysis of the Dolby atmos process (perhaps in future I could also go deeper into documenting this process)

The clear benefits of Dolby Atmos is the availability of more impressive plugins in particular access to the Logic processing, and in particular the spatial plugs. Although the noisemakers and IEM reverbs are quite nice. On deeper though, would it be that the only difference is that the GUI of the DAW’s that work with Dolby atmos as more pretty and aesthetically pleasing than Reaper? Quite possibly. To solve this problem, a next step will be trying to perform similar mixes in each format and then blind testing.

There is a real benefit of ambiX in that it is more easily transportable to a variety of speaker configurations. It is really quite annoying that Dolby atmos limits to the traditional 5.1 style configuration when there are so many different ways to conceive of the set up. The clear benefit of Dolby Atmos is that it can be uploaded to Apple music, it is the clear commercial winner.

Next Steps 

Although I have mixed already quite extensively in spatial formats, until now I really have focussed this on experimental works. By taking on this project it connected for me two previously separated worlds which is quite exciting for me and has created as many new questions as it has answered my initial ones. I will certainly continue to develop my mixing in commercial contexts to include spatial mixes which will likely be in the Dolby format, due to the industry demands.

It is a real shame that these two formats don’t really want to talk to each other. I would love to be able to do a mix in ambiX or converselyand be able to export it to the BWF format required for the Atmos mixes (though perhaps you can and I haven’t explored this enough just yet).

In talking with Steffen Günther (Fraunhof HHI) about the relationship between atmos and ambiX we were discussing the possibility of creating a workflow which can output simultaneously to both formats - this is something interesting to explore. I can immediately say that it would need to be undertaken with a M1 processor.

Bibliography

Holman, Tomlinson. 2008. Surround Sound: Up and Running. 2nd ed. Amsterdam ; Boston: Elsevier/Focal Press.

Roginska, Agnieszka, and Paul Geluso, eds. 2018a. Immersive Sound: The Art and Science of Binaural and Multi-Channel Audio. First published. Audio Engineering Society Presents. New York London: Routledge, Taylor & Francis Group.

Rumsey, Francis. 2001a. Spatial Audio. Music Technology Series. Oxford ; Boston: Focal Press.

Zotter, Franz, and Matthias Frank. 2019. Ambisonics: A Practical 3D Audio Theory for Recording, Studio Production, Sound Reinforcement, and Virtual Reality. Cham, Switzerland: SpringerOpen.