Category: Y2 CONTEMPORARY ISSUES IN SOUND ART ELEMENT 2

Overall Reflection

When I started this line of enquiry I had a pretty clear idea that diegetic underscoring would be a more effective means of scoring than the traditional method, however through research and experimentation I have become more familiar with the intricacies and difficulties of the method.

My first difficulty is that the context within which this method can be used seems to be fairly niche, not every cinematic scene or situation lends itself to diegetic scoring. It’s completely dependant on the emotional narrative of a scene, because a diegetic composer is limited in their approach. If a scene requires an intense composition but is set somewhere with minimal sound, the composer will find their job very difficult. This could result in very creative uses of foley or ambience and doesn’t negate the possibility of great work, I am only acknowledging that this is a difficulty specific to the practice.

The second point id like to make is that using diegetic elements in a musical score seems to be more common than I first thought, there are multiple composers I have researched that engage in this practice as a diegetic anchorage. I think my enquiry was less of an overview of the practice, as it was an exploration of the spectrum of audiovisual correspondence within the practice. Through my experimentation and creative piece I discovered that the approach to diegetic underscoring boils down to abstraction, the degree of abstraction one wants to engage in. The higher the abstraction the higher the emotional intensity (in the majority of cases), however the abstraction detracts from the audiovisual correspondence and therefore the conceivable diegesis of the sound.

The last problem I encountered was the realisation of the piece, the creation of these two audio works was an invaluable process of experimentation and learning, but I wouldn’t say that they reflect the method in the best possible way. I think the first issue was the lack of a film scene to work from, if I were to have a scene I would be able to map out and choreograph the intensity and synchronisation to match the intensity of the scene. Unfortunately I had to conduct the recording as more of an experiment, investigating the transition from one state to another. Chaos and synchronisation. Another problem I had was the lack of participants, the recording would have been more effective had there been more people, and ideally dancers as they would have a better idea of how to use their body. I found the recording difficult to mix into a scene, to me the recording of the eight participants and the atmosphere sound separate. To remedy this I would, as I’ve said, have more participants in a larger space, and perhaps record more footsteps in the same space to mix into the ambience.

I think my piece works well as a middle ground between ambience and foley, and as such is an excellent avenue for subtle manipulation of the audience. The recordings may not be perfect due to the issues listed above but the process worked as proof of my research. If this method were to be used in a production with a decent budget, I as the sound designer would be working from a film scene, be able to increase the level of choreography tailoring it to the narrative, hire more participants who have a dancing background, and record with consideration to the implementation of the sound in the diegetic environment (If I had the time, patience, money and opportunity I would create a convolution reverb at the location of shooting).

Minimal abstraction with dialogue-

Higher abstraction no dialogue-

Tenet

Ludwig Goransson is a Swedish record producer, conductor and film composer. He has worked on productions such as Creed (2015), Black Panther (2018) and The Mandalorian (2019-2022). I will be focusing on his work on the 2020 Christopher Nolan film ‘Tenet’.

One of the most interesting techniques Goransson used for this film was reverse scoring. Ludwig scored a brass band piece, then reversed the score, recorded an orchestra performing that piece, then reversed that recording. This resulted in a very interesting sound, you can hear that the composition is intended that way, not reversed. However all elements within the overall composition are reversed, which is a very pronounced as brass instruments tend to have a distinctive envelope. This score was not only sonically interesting but also thematically relevant as the film is about reversing entropy.

‘The recording is playing backwards but the theme is playing forwards’

This reminds me of the letter from a composer in Schaeffer’s in search of concrete music, drastic manipulation while maintaining the overall form/structure of the music.

For the ‘trucks in place’ score, Gorranson utilises the sound of a truck engine for the low frequency rumble. The recording is pitched down and manipulated, but retains the essence of the truck. This is a version of what my research is about, its a diegetic anchor in the music. I think that the music for a film scene is more impactful and interesting for the listener when the instruments are informed by the screen/context.

‘You’re in this world with these trucks and firetrucks and buses, and the music you’re hearing is also coming from that world. Its not too on the nose … manipulated sounds from the world you’re actually in’

Goransson utilised a similar technique for the film Creed, he recorded a boxer doing their workout and used the recordings as musical elements for the score. These recordings also acted as inspiration informing the composition. The sound of a train was also used in the score as a train plays a prevalent part in the film, the rhythms of the tracks are manipulated and weaved into the score.

‘The movie was so powerful so you didn’t need a traditional score, you didn’t need melodies to tall the audience what he was going through, everything is just glued on the screen. So you just want the music to kind of highlight and heighten the tension and the uncomfortableness that you’re watching on the screen.’

Composition

I decided to create two compositions with varying levels of interference/manipulation.

The first is minimal manipulation, I used the original recording of the footsteps session and added a train station ambience. I also decided to use some audio of intense cinematic dialogue, I did this as a means of distracting the listener, giving them something to follow. This is because the footsteps are meant to be a support for a narrative, rather than the focus of the scene. I wanted the footsteps to have a certain level of subtlety to them, so as not to take away from the story. Unfortunately the recording demands more attention than I had anticipated, I think that this is due to the transition between normal and uniform footsteps was too short. Ideally I would like a more gradual transition to give the audience time to acclimatise to the sound, the subtler the better. I would say that this piece is a success, it conveys what I wanted it to convey, its a completely diegetic underscore that supports the narrative of the scene.

For the second version of the piece I took a more abstracted approach to the composition, allowing for more manipulation and more remote audiovisual correspondence. The first challenge I faced was the variations in tempo, the participants were not perfect in their steps and the synchronisation moves in and out of the 115 BPM I had set with a visual metronome. I had to find an instance of consistent tempo, edit it slightly and duplicate the clip. This is the first step away from audiovisual correspondence as the ambient sound has now been sampled. I then imported a field recording I had made a few years ago, the recording was of some construction work being conducted near my house. The sound I decided to loop was of what I think is a circular saw cutting stone. An uncomfortable sound to support an uncomfortable scene. I manipulated this sound with a tremolo, this gave the sound a rhythmic quality blending nicely with the steps. I was using Ludwig Gorranson’s compositions for the 2020 Nolan film ‘ Tenet’ as a reference, I wanted to create a similar palette. I also applied a tremolo to a field recording of wind. To emphasise the first beat of the bar I used two elements, a reversed clap and a sped up recording of an egg shaker increasing in tempo. Both of these sounds are very high in frequency so they cut through the constant low frequencies from the contact mics. Lastly to add some melody I manipulated a recording of a train engine. This choice was informed by the writings of Pierre Schaeffer on sound retaining meaning (specifically the train and the foundry).

This more musical approach was interesting, I felt I could achieve a more dense level of emotional information but the manipulations detracted from the audiovisual correspondence. The main problem I had with my process was that I had decided to use Protools as a workstation to compose, this resulted in many problems, from instruments not working to audio files not being able to be imported, mainly just a slower and less efficient process of creation.

KMRU

KMRU (Joseph Kamaru) is an ambient music artist who has a focus on field recording. He is drawn to meditative and calming music, possibly as a result of his upbringing in the noisy streets of Nairobi. He describes it as a means to ‘slow things down.’

In high school, Joseph moved from the city to a more rural area. This drastic change of sonic environment caused a sensitivity to his surroundings and a focus on his aims. Kamaru started to conduct field recordings, collecting material to use in his music production. He emphasises minimalism as a means of inspiration, the limitation forces you to make the best work with what you have. There is also an emphasis on being open to whatever happens, improvising on synthesisers while not restricting the outcome.

Kamaru is currently doing a three year master’s program in the Berlin university of arts. The transition from Nairobi to Berlin was difficult as the sonic landscapes are so different, Kamaru started to wonder if his sound would change. Joseph’s intention and understanding of field recording developed as he went deeper into the practice. His relationship to the practice was strictly functional, he valued the recording itself, for its sonic qualities and utility for music. Now, Kamaru has started to consider the location as an important aspect of the practice.

‘I feel that sound itself is the best way to learn and understand our surroundings‘

Kamaru has been making an effort to remove the act of recording occasionally, just to sit and listen as a meditative activity. Sometimes he will bring a notebook and pen to describe his surroundings, this seems to be a method of becoming more perceptually literate. Experience is as important as the act of recording. Occasionally he will draw what he hears as a way to document experience.

Kamaru’s ambient music philosophy is to use elements of recordings in extended ways to create new textures/melodies. The aim of his method is to blend music with field recording to create a sense of insecurity in the listener as to whether music is happening, the field recordings also provide a narrative behind the music. The listener has something to connect to enhancing their experience.

I find KMRU’s work inspiring as its a practice I’m unfamiliar with, I don’t make ambient music and I only rarely conduct field recordings. His idea of giving the listener something more to connect to and giving instruments a narrative is very interesting. I believe this philosophy applies to my interest diegetic underscore, the recordings you compose with carry a narrative that is evident even after manipulation.

Recording session

The recording session went well, there was not as many people as I had hoped but the experiment still worked. Eight people attended the experiment, this resulted in a decent ‘crowd’ sound. The first recording went well, the lack of definite tempo and lack of practice made the synchronisation gradual (which is what I was aiming for). I set a tempo of 115bpm for the next recording, this is because I believed it to be a good tempo for composition if I want to layer more musical elements on it. The synchronisation was less gradual with each recording as the participants became more familiar with the action, which I should’ve anticipated but I believe with more participants the synchronisation would take more time. The one thing I forgot to mention to the participants until the last recording was to increase the volume of the footsteps after the synchronisation had occurred, I assumed that this would happen anyway but it didn’t.

Im glad I decided to use contact microphones as I believe those recordings will come in very useful for creative manipulation. My initial idea is to bring the contact mic recording of the synchronisation section up in the LFE, preceding the binaural synchronisation. A subtle foreshadowing of the incoming intensity.

‘Planning recording session’

I have decided to use footsteps as a means of diegetic underscore as they are present in a large quantity of film and television, they also offer an opportunity to compose with rhythm, chaos and synchronisation.

The purpose of this entire practice/experiment is to find a less intrusive way of influencing an audience, maintaining their suspension of disbelief. Footsteps permeate everyday life, we hear them constantly, this makes them an excellent candidate for manipulation. Through manipulation/choreography, this sound can go from its natural chaotic state, to a synchronised rhythm. provided this synchronisation is gradual enough, the audience won’t notice the transition from close to remote audiovisual correspondence. Technically the audiovisual correspondence or synchronous integrity of the sound cannot be destroyed, as the sound source could be constantly present in a scene (People in a train station), and it is not impossible that people’s footsteps would synchronise in that narrative world, unlikely but not impossible.

I think it is necessary to use 3 types of microphones in the recording session to maximise utility of the sound. Firstly a binaural microphone to capture a realistic spatial recording of the space when this synchronisation happens, this will especially emphasise the immersion and experiencial connection between character and audience. Secondly a condenser microphone to capture an overall perspective of the room, I think this is necessary as I believe it’ll be easier to use in a mix. Thirdly a contact mic (maybe more than one) to capture the impact of the feet on the floor. These lower frequencies could be useful in a 5.1 mix, for the LFE to drive more impact into the mix. I’m considering using an ambisonic microphone for the possibility of implementation into VR.

I will be conducting the piece in two ways with varying levels of interference/choreography. The first will be minimal choreography (Fig. 2), I will ask the participants to walk normally (as if they were walking to catch a train, maybe even ask to avoid synchronisation), then after 40 seconds ask the participants to synchronise. This will result in unpredictable results as it’s impossible to know how long it will take them to synchronise, but with result in a very organic transition.

The second level of choreography (Fig.3) is separating the people into groups (for groups see Fig.1), after 30 seconds I will signal group 3 to synchronise, then at 45 I will signal group 2 to synchronise, then at 60 seconds group 1.

An important aspect of this recording session is the material of the floor, obviously carpet would be too quiet, but the material must also match the materials present in film scenes. For a train station I think the ‘Black Box’ room is the best option, it has a slight rubber texture to it (Fig. 4). It’s a little quieter/softer than I would have liked, but it is usable.

Possible elements for manipulation

 Ability to manipulate vs keeping original quality of sound. 

Ability to manipulate as melody = conceivable range of frequency created by sound object. 

When analysing sounds for use of melody, we must first analyse their conceivable range of frequency. A pencil on paper has a range of conceivable frequency after which it becomes no longer believable to the audience.

It might also be useful to understand the natural envelope of the sounds, if one were to put a slow attack on a slamming door it would defeat the natural sound of the action. I think only through trial and error I will find that breaking point of manipulation and mimetic continuity. I don’t want to restrict myself to the world of mimetic sound but I do think a certain level of continuity must be maintained in order to keep the audience immersed. Perhaps drastic manipulations with immersion might be made possible through careful structuring of gradual increasing manipulation. Howard Ashman’s musical theatre inspired philosophy must be kept in mind, gradual seamless transitions from differing levels of emotional intensity. 

This issue is best defined, I believe, by Martin Stig Anderson in one of his lectures in which he displays a graph explaining the spectrum from soundscape composition to acousmatic music. 

Sounscape composition —————————————-Acousmatic music 

Contextual immersion —-   Abstract middle ground —- Complete abstraction 

Close audiovisual correspondence — Remote audiovisual correspondence 

Martin Stig Anderson Lecture

Martin Stig Anderson begins his lecture by reading a couple of reviews people had written about his work on the 2010 game ‘Limbo’. As you can see the reviews are completely polarised as to whether there is music in the game. Music can be much more than rhythm, harmony and melody. Ambient and environmental noises are the feature of the show. 

Anderson lays the foundation of his practice with the definition of music as ‘organised sound’, then going on to talk about Pierre Schaeffer’s music concrete. There must, however, be a distinction between soundscape composition and music concrete, although they are essentially the same practice there is an important varying factor. In the practice of soundscape composition, ‘the original sounds must stay recognisable and the listeners contextual and symbolic associations should be invoked’. – Barry Truax

Soundscape composition exists on a spectrum with music concrete, between contextual immersion, abstracted middle ground and absolute abstraction. Music concrete is the movement from a abstracted middle ground to absolute abstraction, and soundscape composition a movement from contextual immersion to abstracted middle ground. 

A technique Anderson uses to merge orchestral elements into the diegesis of the film is spectral interpolation, for the game limbo he did this with an orchestral recording and a recording of a foundry. ‘It retains the nuances of the orchestral but its not really there at the same time’. This was for one of many rotating rooms in the game. The second was a room of circular saws, for this he used a recording of a bowed cymbal, yet again he interpolated the orchestral sound with this recording. This results in a diegetic sound with complete contextual immersion, which contains harmonic/melodic qualities. Anderson uses source filtering and convulsion techniques to merge the two audio files. 

Makoto Oshiro

Makoto Oshiro is a Tokyo based audiovisual and experimental music artist, making instruments with technology, installation pieces and performances.

Self made instruments

Makoto begins his lecture with a recording from one of his self made instruments, a high pitched high tempo glitching note, it sounded like an artificial synthesis of the sound of extending a role of cello-tape. The second part of the recording was a slower tempo and slightly lower pitched sound, but the slower tempo of the beats revealed a spatialisation of the sound, the instrument was moving and populating the entire stereo field. The third movement of the recording was more dense in texture, there were multiple organic sounding rhythms colliding creating fleeting/evolving patterns and beats. This was created using a self made instrument that utilised electromagnetic relay. These are usually used to switch higher voltage to lower voltage, for example the headlights in a car would use these relays. The clicking sound is caused by a small metal plate being pulled into the mechanism by electromagnetism. The frequency of clicks is controlled by a 555 timer IC, a very common chip in electronics. A volume knob controls the frequency of the on and off saw wave, this brings a human element to the instruments, the ability to feel the frequency as a means to compose. Makoto creates what he calls ‘acoustic oscillators’. Although Makoto’s focus is creation of music through electronic instruments, he tries to make the sound generators acoustic. The environment is always a component of the composition.

Makoto is updating these instruments by including an Arduino alternative as a way to drive the relays, he also added an 8 step sequencer for more creative expression. In the video he presents, you can see him change the pitch of the oscillators by placing rocks of varying size. I think this is a very interesting means of creating sound works, these instruments are not the only material to compose with, they also turn every other object into a modulator. Involving the real world in something that is usually confined to the world of electronics.

Installations

Oshiro presents a video of his 2017 audiovisual installation ‘mono-poly’. Makoto was interested in audiovisual works but did not want to arbitrarily make audio that matched the visual, he wanted them to be intrinsically linked. He started to investigate how visual signals worked, and how to use a speaker as an object rather than a sound projector. The piece was created by combining two sine waves with a difference of 1Hz, this is called ‘beating’. These waves were then fed through two speakers, one with a 2Hz beat and one with a 1Hz beat. This resulted in a visual beating and slightly audible beating, this piece was called ‘Kinema’ (2009). Makoto then created a piece called ‘Braids & knots’ (2010), in which he tried to attach objects to the speakers to provide a more drastic visual to the work. He found success in attaching a string to the centre of a subwoofer and stretching it across a room, a representation of the wave would move through the string. He attached LEDs to this string but did not manage to achieve the level of visualisation he desired.

He developed this idea further with ‘Strings'(2014), in this piece Makoto achieved a much better visualisation of the beats. Strings were attached to a subwoofer in the corner of a room, and extended to the opposite corner. There was a strobe light placed on the floor that was modulated by an Arduino to illuminate the strings at different frequencies, the result was an etherial representation of audio.

‘mono-poly’ was the final development of this series. Makoto managed to find a way to make the LED strobe produce multiple frequencies at once, this allowed for more dynamic change in the visualisation of the string.

I find Makoto Oshiro’s work fascinating, especially the visualisation of sound. His use of electronics are alien to me but still very interesting as a means of creation. I admire the simplicity of his acoustic oscillators and above all his ability to involve physical space in his works.

‘Diegetic Underscoring’

Underscoring is traditionally a non-diegetic element of film, occupying a space separate from the narrative in which the music almost responds to what is happening on screen. Sometimes scores can act as an emotional roadmap, guiding the audience. This conception of underscores is influenced by the first implementation of music into cinema where a pianist would sit next to the screen playing the score whilst watching the film. The primary use of music is to emotionally influence the audience, but it exists outside of the narrative. Its format is at odds with its use. In order to influence the audience’s emotions whilst maintaining the suspension of disbelief, we can start to bring underscoring into the narrative by changing our methods and instruments. 

Diegetic underscoring can be achieved through the utilisation of Musique Concrete, a stye of composition developed by Pierre Schaeffer. This style of composition is about the appreciation of the audio quality of everyday sounds rather than their contextual value, through layering and manipulation one can make compositions exploring different formations. Although contextual value is not a focus in this practice, the ‘meaning’ of the sound can survive the creative processes. This ‘meaning’ is what can anchor the underscore to the narrative, by either composing a music concrete style underscore from sounds evident in the scene, or interweaving a traditional score with such a composition, the film composer is able to maintain diegesis. 

The diegetic capabilities of underscores are rarely utilised, but I think the practice can result in extremely immersive and compelling scores. Diegetic scoring can not only be used for immersivity, it can also be used to dive narrative and represent an unseen character.