Author: Harry Charlton

Posted on

ELEMENT 2

‘Planning recording session’

‘Black Box’ Room at LCC

I have decided to use footsteps as a means of diegetic underscore as they are present in a large quantity of film and television, they also offer an opportunity to compose with rhythm, chaos and synchronisation.

The purpose of this entire practice/experiment is to find a less intrusive way of influencing an audience, maintaining their suspension of disbelief. Footsteps permeate everyday life, we hear them constantly, this makes them an excellent candidate for manipulation. Through manipulation/choreography, this sound can go from its natural chaotic state, to a synchronised rhythm. provided this synchronisation is gradual enough, the audience won’t notice the transition from close to remote audiovisual correspondence. Technically the audiovisual correspondence or synchronous integrity of the sound cannot be destroyed, as the sound source could be constantly present in a scene (People in a train station), and it is not impossible that people’s footsteps would synchronise in that narrative world, unlikely but not impossible.

Fig.1

I think it is necessary to use 3 types of microphones in the recording session to maximise utility of the sound. Firstly a binaural microphone to capture a realistic spatial recording of the space when this synchronisation happens, this will especially emphasise the immersion and experiencial connection between character and audience. Secondly a condenser microphone to capture an overall perspective of the room, I think this is necessary as I believe it’ll be easier to use in a mix. Thirdly a contact mic (maybe more than one) to capture the impact of the feet on the floor. These lower frequencies could be useful in a 5.1 mix, for the LFE to drive more impact into the mix. I’m considering using an ambisonic microphone for the possibility of implementation into VR.

I will be conducting the piece in two ways with varying levels of interference/choreography. The first will be minimal choreography (Fig. 2), I will ask the participants to walk normally (as if they were walking to catch a train, maybe even ask to avoid synchronisation), then after 40 seconds ask the participants to synchronise. This will result in unpredictable results as it’s impossible to know how long it will take them to synchronise, but with result in a very organic transition.

Fig.2

The second level of choreography (Fig.3) is separating the people into groups (for groups see Fig.1), after 30 seconds I will signal group 3 to synchronise, then at 45 I will signal group 2 to synchronise, then at 60 seconds group 1.

Fig.3

An important aspect of this recording session is the material of the floor, obviously carpet would be too quiet, but the material must also match the materials present in film scenes. For a train station I think the ‘Black Box’ room is the best option, it has a slight rubber texture to it (Fig. 4). It’s a little quieter/softer than I would have liked, but it is usable.

Footsteps in Black Box room recorded by holding iPhone 12 while walking
Footsteps in Black Box room recorded by placing iPhone 12 on chair (Fig. 1)

Posted on

ADAPTIVE AUDIO

Video game sound designers have a unique task in the world of sound design. In a film, the sound designer knows exactly what will happen, but in a game the designer must account for the player’s agency. The world will change depending on the player’s interaction with it. This is ‘adaptive audio’, audio that adapts to player input. An example of adaptive audio in game design is footsteps, they must first be tied to the event of walking (then the player inputs the signal to walk), the audio must then be matched to the walking animation. The necessary work to achieve audiovisual correspondence in games is far more than film. There is another degree of adaptivity beyond this, adapting to environment. The footsteps must change sound depending on where the player moves their character. This is especially important when there is an informative element to it, for instance if ice is slippery there should be an auditory cue for the player.

Another degree of adaptivity is how the sounds we implement interact with the environment in the game. Inaccurate reverb will bring the player out of the game, ruining the immersion. As sound designers our job is to immerse the player, and to avoid distracting their attention from where it ought to be.

To solve these problems we must include variation within the sounds triggered by the event. This can be done though designating certain reverbs to certain zones within the game. We can also implement gradual gradual transitions that is dependant on movement thought environments, this is usually tied to music transitions from the different locations. The most effective utilisation of adaptive audio, in my opinion, is when designers pitch sound effects to key of the background music. This results in a merger between diegetic interactive sound and the non-diegetic sonic/melodic world of the game. I hope to have the opportunity to explore implementing these parameters in FMOD.

Posted on

SOUND IN HORROR GAMES

Five nights at Freddy’s is a horror game in which the player is trapped in the security office of a pizzeria while animatronic creatures attempt to kill them. The horror feeling is built out of two concepts, absence of action and absence of information.

The absence of information refers to the fact that the animatronic creatures move when the player is not looking at them, and so the player must rely on clues as to their whereabouts. The player has access to some obvious visual clues such as the CCTV system, but a lot of the information is subtilely delivered to the player through the audio of the game. The audio is a more complicated source of information as some audio helps you and some deceives you, this adds to unsettling nightmare like sense of not knowing.

Within the game sound there is advantageous audio, this is the audio that helps the player with their goal of survival. These sounds do not reveal themselves straight away, when the player begins the game the sounds present themselves as chaotic. As more time is spent in the game, the player gains an understanding of the auditory patterns and their listening transitions from casual to semantic, deducing clues from the patterns. An example of this is the sound of pots and pans, at first this sound seems like an element of ambience, put in the game to induce panic in the player. However as the player interacts with the game more they will find that this sound becomes louder when they look at the kitchen camera (which has broken visuals, its audio only). When coupled with the player noticing that one one of the animatronics is missing, they can deduce that the animatronic is in the kitchen. These unsettling sounds have a purpose in the gameplay.

The most interesting part of the sound design for this game is the detrimental audio, the audio that is there to be a distraction from the truth. One element of this is the choice to have very poor transmission of audio from the cameras, the audio is filtered and manipulated heavily as a means to muddy the information. There are also sounds that are implemented simply to get in the way, to demand attention at a time of intense audio sensitivity. This is the mechanical sound of the cameras turning or the intermittent static. All of this sound is also lightly obscured by the fan in the office, this is a constant sound that is off a similar frequency to the audio from the cameras. These sound design choices are a very clever means to disorient and frustrate the player in a similar way to how one might experience a nightmare, like the feeling of running and going nowhere, there are simple little things that make success so much harder. When paired with a life or death scenario the feeling is intense frustration.

I think that sound has a unique capability in horror games specifically. In my opinion audio hallucination is much less obvious to the subject than visual hallucination, yet holds a great deal of weight in our sense of orientation in reality. We understand a lot of our environment through listening, and in a game context where this audio can not only be manipulated but also interacted with (giving it a greater weight in reality), sound designers can achieve a uniquely disorientating experience.

Posted on

MA COLLABORATION

Mechanical Bird Sound First Draft

The first thing I did when constructing this sound was identify the different samples I could use. The sound was divided into three elements, natural, mechanical cog and Motor. I started by finding audio of a bird flapping its wings, this is what I would use as a blueprint to lay the mechanical sounds on top of. I tried to match the real sound source’s size to the fictional object’s size. The first mechanical sound I added was the sound of the cog pulling the wing down, for this I used layered camera shutters, because its a high velocity small mechanism. The second element of the mechanical bird was a motor, I imagined that a motor would be necessary for a mechanical system. The only motor that would fit in a mechanical would be a watch motor. These layered created a textured mechanical system to place under the natural sounds. In the interest of time I created a 6 flap loop, when I revisit this sound I will find more diverse samples and create a longer loop to make the sound more believable.

The natural sound provided the hardest problem when designing this sound. I found that recordings of real birds had a Doppler effect as the birds were always flying past the microphone, this was unusable for me as I needed a steady flapping. The only alternative was foley recordings I found in sound libraries, the drawback to this was they all had a very artificial quality to them. Another problem I experienced was determining the amount of low frequency that should be present in the flapping. From the perspective of a person the air displacement wouldn’t be that much, therefore there wouldn’t be a lot of low frequency in the sound. However, from the perspective of the bird there would be much more low frequency. The problem I face is that the amount of low frequency in the wing flap is an auditory signifier of the size of the bird, how do you express the size of the bird accurately?

Posted on

MA COLLABORATION

Martin Stig Anderson Lecture

Martin Stig Anderson begins his lecture by reading a couple of reviews people had written about his work on the 2010 game ‘Limbo’. As you can see the reviews are completely polarised as to whether there is music in the game. Music can be much more than rhythm, harmony and melody. Ambient and environmental noises are the feature of the show. 

Anderson lays the foundation of his practice with the definition of music as ‘organised sound’, then going on to talk about Pierre Schaeffer’s music concrete. There must, however, be a distinction between soundscape composition and music concrete, although they are essentially the same practice there is an important varying factor. In the practice of soundscape composition, ‘the original sounds must stay recognisable and the listeners contextual and symbolic associations should be invoked’. – Barry Truax

Soundscape composition exists on a spectrum with music concrete, between contextual immersion, abstracted middle ground and absolute abstraction. Music concrete is the movement from a abstracted middle ground to absolute abstraction, and soundscape composition a movement from contextual immersion to abstracted middle ground. 

A technique Anderson uses to merge orchestral elements into the diegesis of the film is spectral interpolation, for the game limbo he did this with an orchestral recording and a recording of a foundry. ‘It retains the nuances of the orchestral but its not really there at the same time’. This was for one of many rotating rooms in the game. The second was a room of circular saws, for this he used a recording of a bowed cymbal, yet again he interpolated the orchestral sound with this recording. This results in a diegetic sound with complete contextual immersion, which contains harmonic/melodic qualities. Anderson uses source filtering and convolution techniques to merge the two audio 

Posted on

ELEMENT 2

Possible elements for manipulation

 Ability to manipulate vs keeping original quality of sound. 

Ability to manipulate as melody = conceivable range of frequency created by sound object. 

When analysing sounds for use of melody, we must first analyse their conceivable range of frequency. A pencil on paper has a range of conceivable frequency after which it becomes no longer believable to the audience.

It might also be useful to understand the natural envelope of the sounds, if one were to put a slow attack on a slamming door it would defeat the natural sound of the action. I think only through trial and error I will find that breaking point of manipulation and mimetic continuity. I don’t want to restrict myself to the world of mimetic sound but I do think a certain level of continuity must be maintained in order to keep the audience immersed. Perhaps drastic manipulations with immersion might be made possible through careful structuring of gradual increasing manipulation. Howard Ashman’s musical theatre inspired philosophy must be kept in mind, gradual seamless transitions from differing levels of emotional intensity. 

This issue is best defined, I believe, by Martin Stig Anderson in one of his lectures in which he displays a graph explaining the spectrum from soundscape composition to acousmatic music. 

Sounscape composition —————————————-Acousmatic music 

Contextual immersion —-   Abstract middle ground —- Complete abstraction 

Close audiovisual correspondence — Remote audiovisual correspondence 

Martin Stig Anderson Lecture

Martin Stig Anderson begins his lecture by reading a couple of reviews people had written about his work on the 2010 game ‘Limbo’. As you can see the reviews are completely polarised as to whether there is music in the game. Music can be much more than rhythm, harmony and melody. Ambient and environmental noises are the feature of the show. 

Anderson lays the foundation of his practice with the definition of music as ‘organised sound’, then going on to talk about Pierre Schaeffer’s music concrete. There must, however, be a distinction between soundscape composition and music concrete, although they are essentially the same practice there is an important varying factor. In the practice of soundscape composition, ‘the original sounds must stay recognisable and the listeners contextual and symbolic associations should be invoked’. – Barry Truax

Soundscape composition exists on a spectrum with music concrete, between contextual immersion, abstracted middle ground and absolute abstraction. Music concrete is the movement from a abstracted middle ground to absolute abstraction, and soundscape composition a movement from contextual immersion to abstracted middle ground. 

A technique Anderson uses to merge orchestral elements into the diegesis of the film is spectral interpolation, for the game limbo he did this with an orchestral recording and a recording of a foundry. ‘It retains the nuances of the orchestral but its not really there at the same time’. This was for one of many rotating rooms in the game. The second was a room of circular saws, for this he used a recording of a bowed cymbal, yet again he interpolated the orchestral sound with this recording. This results in a diegetic sound with complete contextual immersion, which contains harmonic/melodic qualities. Anderson uses source filtering and convulsion techniques to merge the two audio files. 

Posted on

VISITING PRACTITIONERS

Makoto Oshiro

Makoto Oshiro is a Tokyo based audiovisual and experimental music artist, making instruments with technology, installation pieces and performances.

Self made instruments

Makoto begins his lecture with a recording from one of his self made instruments, a high pitched high tempo glitching note, it sounded like an artificial synthesis of the sound of extending a role of cello-tape. The second part of the recording was a slower tempo and slightly lower pitched sound, but the slower tempo of the beats revealed a spatialisation of the sound, the instrument was moving and populating the entire stereo field. The third movement of the recording was more dense in texture, there were multiple organic sounding rhythms colliding creating fleeting/evolving patterns and beats. This was created using a self made instrument that utilised electromagnetic relay. These are usually used to switch higher voltage to lower voltage, for example the headlights in a car would use these relays. The clicking sound is caused by a small metal plate being pulled into the mechanism by electromagnetism. The frequency of clicks is controlled by a 555 timer IC, a very common chip in electronics. A volume knob controls the frequency of the on and off saw wave, this brings a human element to the instruments, the ability to feel the frequency as a means to compose. Makoto creates what he calls ‘acoustic oscillators’. Although Makoto’s focus is creation of music through electronic instruments, he tries to make the sound generators acoustic. The environment is always a component of the composition.

Makoto is updating these instruments by including an Arduino alternative as a way to drive the relays, he also added an 8 step sequencer for more creative expression. In the video he presents, you can see him change the pitch of the oscillators by placing rocks of varying size. I think this is a very interesting means of creating sound works, these instruments are not the only material to compose with, they also turn every other object into a modulator. Involving the real world in something that is usually confined to the world of electronics.

Installations

Oshiro presents a video of his 2017 audiovisual installation ‘mono-poly’. Makoto was interested in audiovisual works but did not want to arbitrarily make audio that matched the visual, he wanted them to be intrinsically linked. He started to investigate how visual signals worked, and how to use a speaker as an object rather than a sound projector. The piece was created by combining two sine waves with a difference of 1Hz, this is called ‘beating’. These waves were then fed through two speakers, one with a 2Hz beat and one with a 1Hz beat. This resulted in a visual beating and slightly audible beating, this piece was called ‘Kinema’ (2009). Makoto then created a piece called ‘Braids & knots’ (2010), in which he tried to attach objects to the speakers to provide a more drastic visual to the work. He found success in attaching a string to the centre of a subwoofer and stretching it across a room, a representation of the wave would move through the string. He attached LEDs to this string but did not manage to achieve the level of visualisation he desired.

He developed this idea further with ‘Strings'(2014), in this piece Makoto achieved a much better visualisation of the beats. Strings were attached to a subwoofer in the corner of a room, and extended to the opposite corner. There was a strobe light placed on the floor that was modulated by an Arduino to illuminate the strings at different frequencies, the result was an etherial representation of audio.

‘mono-poly’ was the final development of this series. Makoto managed to find a way to make the LED strobe produce multiple frequencies at once, this allowed for more dynamic change in the visualisation of the string.

I find Makoto Oshiro’s work fascinating, especially the visualisation of sound. His use of electronics are alien to me but still very interesting as a means of creation. I admire the simplicity of his acoustic oscillators and above all his ability to involve physical space in his works.

Posted on

ELEMENT 2

‘Analysing Examples’

‘La Grande Bellezza’

Diegetic anchoring of underscore:

The great beauty- The film opens with the sun shining on multiple beautiful landmarks in Rome, tourists take their pictures, locals smoke their cigarettes, all of which underscored by a church bell ringing. The church bells are shortly accompanied by an all female choir chanting harmonies that sound somewhere between apathetic and melancholic. As the opening continues with shots of tourists admiring the beauty of Rome, a solo singer takes the lead with a melancholic passionate melody. This stark contrast between the seemingly eternal beauty of Rome and the melancholic singing is the crux of the film. A surrounding of decadence and beauty, while feeling unfulfilled, unable to reach ‘the great beauty’ in regards to the human condition. With this musical contrast, Paolo Sorrentino was able to set up the concept of the film, and by making the source of the music a diegetic element, the representation of the melancholic humanity (the choir) is made a part of the city itself, the solo vocalist stares out at the beauty as if to have a dialogue with it. 

Delicatessen (1991)

Underscore as a narrative drive:

Delicatessen- Delicatessen is a post apocalyptic black comedy about a butcher who has gained power and influence through selling human meat in a world where protein is scarce. The film explores the dynamic between the butcher, his assistant who is next lined up for the slaughter, the butchers daughter and the people living in his building. There is a scene in which directors Jean-Pierre Jeunet and Marc Caro use sound as a narrative device to demonstrate the dynamic of the building. 

The scene opens as the butcher is getting intimate with a woman on a very noisy metal spring bed, the subsequent creaking noise echoes through the pipes of the building permeating the residents rooms. First a cellist plays her scales to the tempo of the creaks, then man painting his ceiling matches the tempo, a person inflating a bike tyre, a woman beating a rug and lastly a man working a punching machine. As the creaks rise in tempo, the actions of the residents become comically fast. Sound is acting as a narrative tool to drive the story, both resulting in a comical situation and demonstrating the power dynamics of the household. 

Posted on

ELEMENT 2

‘Diegetic Underscoring’

Underscoring is traditionally a non-diegetic element of film, occupying a space separate from the narrative in which the music almost responds to what is happening on screen. Sometimes scores can act as an emotional roadmap, guiding the audience. This conception of underscores is influenced by the first implementation of music into cinema where a pianist would sit next to the screen playing the score whilst watching the film. The primary use of music is to emotionally influence the audience, but it exists outside of the narrative. Its format is at odds with its use. In order to influence the audience’s emotions whilst maintaining the suspension of disbelief, we can start to bring underscoring into the narrative by changing our methods and instruments. 

Diegetic underscoring can be achieved through the utilisation of Musique Concrete, a stye of composition developed by Pierre Schaeffer. This style of composition is about the appreciation of the audio quality of everyday sounds rather than their contextual value, through layering and manipulation one can make compositions exploring different formations. Although contextual value is not a focus in this practice, the ‘meaning’ of the sound can survive the creative processes. This ‘meaning’ is what can anchor the underscore to the narrative, by either composing a music concrete style underscore from sounds evident in the scene, or interweaving a traditional score with such a composition, the film composer is able to maintain diegesis. 

The diegetic capabilities of underscores are rarely utilised, but I think the practice can result in extremely immersive and compelling scores. Diegetic scoring can not only be used for immersivity, it can also be used to dive narrative and represent an unseen character. 

Posted on

VISITING PRACTITIONERS

‘Lindsay Wright’

Lindsay Wright is a British composer, her practice involves blending traditional orchestral elements with experimental methodologies.

‘The mystery of D.B. Cooper’-

This documentary focuses on the highjacking of a plane in the 1970’s, the documentary forwent investigating the perpetrator of the crime, rather it highlighted public obsession with the mystery. Lindsay’s brief for the documentary included many different elements, mainly a 70’s theme with modern sections.

Firstly, Wright talks about scoring the cue ‘attention all agents’. This is the moment the FBI agents are notified about the highjacking, moving from footage of police cars racing to the scene onwards to talking head interview elements. The cue begins with a classic 70’s cop drama style intro, with guitars violins and drums, the score then reduces in volume and frequency occupation to pave way for dialogue. These talking head sections were underscored with low frequency synthesisers (bringing in the modern influence) with drums maintaining the narrative intensity/pacing. Lindsay also made use of a key change to provide a sense of urgency and build, in her words an easier way to build tension without actually having to write anything new.

Write tried to give each person being interviewed their own musical identity. She talks about one person having a string belief that DB Cooper did not survive the crash. Scoring this Lindsay tried to give it a more ‘Fargo’ inspired sound, off kilter, curious and sometimes fun. This part of the scoring process highlighted the importance of creating more than one version for cues, as sometimes your scores don’t reflect the directors vision.

Lindsay talks about starting a composition with a few days dedicated to finding the right timbre or sound palette, to find the sonic identity of the piece. Sounds have themes in integral in their timbre, finding a good instrument to attach to a certain element of the narrative is an important part of the process.

‘Mudlarks’-

Mudlarks Is a film about two homeless girls living in a tent besides the thames, during the film one of the girls doesn’t come home from a s shift of finding money/food. The aims of the scoring was to highlight the girls relationship, and the anguish caused by one of their absence. Lindsay tried to capture the vastness of the city whilst also commenting on the intimacy of their relationship. They focused on gestural ideas, and built up textures, each they could reuse later in the film. EQ automation is a useful technique in building gestures, filling out the sound as the score grows. A quality the director wanted from the sound is a homemade feel, which was convenient as the budget only really afforded home studio recording. Lindsay talks about the importance of bringing in tones from the rest of the score into each cue, it gives the whole underscore a fluidity and solid sonic identity.

‘Lines’-

This was Wrights self made EP constructed during the first lockdown, the project was influenced by Patrick Jonsson, Angus McRae and Keaton Henson. Exile was a track inspired by moving from a busy street in south east London to a small town in Cambridgeshire, how moving gave her more space to think and work healthily. Other tracks inspired by peoples resilience to carry on working through the pandemic, also a reflection on some peoples negative experience such as her friend having to see his family though a glass window.

Musical Traits to consider when constructing a piece-

  • Timbre
  • Constant motion
  • Cyclical Layers
  • Texture
  • Rhythm
  • Harmony
  • Instrumentation
  • Structure
  • Slides
  • Bass
  • Space and Depth