Mixing 'Virtual' Music With The 'Real' World

Locative and Geo-Located Audio: Editing a Place with Sounds

After my presentation at UoS Creative Digifest2, Guy Stephens of Cap Gemini invited me to speak at their London offices: Monday 22nd October, presenting recent musical and academic research on virtual sound in real environments.

Fantastic opportunity to discuss my musical and academic research with an audience at Cap Gemini yesterday, in their astonishing 8th floor "Accelerated Solutions Environment" - an entirely mobile arrangement of walls and furniture on wheels, capable of almost instantaneous transformation. 

I was more than a little apprehensive about speaking straight after the man from Google - Ed Parsons, Geospatial Technologist at Google Maps, on the future of Annotated Landscapes - who described some mind-boggling concepts for emerging mobile connectivity.

I presented the reasons for wanting to create music you can walk inside - how the studio lets us do extraordinary things that seem really to be happening but which are impossible - but we are then obliged to listen to music on speakers; aware of the sounds' artificiality and having a rather boring sensory experience, when compared to the highly visual psycho-drama of live performance.

So early last year I approached ISVR at UoS to talk about a system I wanted to build, that is now turning slowly into 3DBARE, a virtual spatialisation tool for immersive and interactive audio.

The audience saw for the first time the audio rendering engine drafted by ISVR's Miguel Galindo and Iyad Assaf's excellent user interface / control platform, which is going to be ready for prototype demonstration in the next few weeks, using a table-top version.

They also heard about the geo-located music tool, noTours, which I have been using to place music in a landscape works directly with the Google Maps API and has the potential for really immersive and interactive audio embedded into the main map search platform.

I'd spent the weekend at St Paul's Cathedral, recording interviews and the acoustic environment. Then I placed my choral piece "Take Me By The Hand" into the landscape and mixed it with the traffic, fountains, machines, bells, skateboarders, fighters, Big Issue sellers, tourists, lovers, wedding parties and historical reconstructions that were all around the area.

Met the inspired team of Unreal City Audio - and found myself on their tour "Hawkers, Harlots & Hacks" - thanks for the brilliant audio, guys!

I've been considering how to develop sharing of virtual soundscapes, distribution to listeners in diverse locations, multi-contribution work that can take place in several locations simultaneously (and each player can hear the other players, selectively as s/he chooses).

It seems that at the very moment of its first appearance, the album as software application is already a fleeting obsolescence, being only a fraction removed from the fixed format of the gramophone disc.

Perhaps this is the first time since the advent of recording that music has been a changeable, living entity, where as one walks inside it, the experience is controlled and determined by one's movement and position.

To dissect and spread a piece of music across a landscape anywhere in the world now permits listeners to download the music and map data then go there and enter the composition as though it were a physical object.

We are doing this not just with musical compositions but audio montages of a particular place and time - sonic snapshots of constantly changing acoustic environments.

Surround stereo clip from St Paul's Churchyard Audio Portrait here (with headphones please!)

For geo-located version at location of your choice, send me a postcode and I will attach the music to the place and send you a link to your private Music You Can Walk Inside download: @benjaminmawson

Painting the landscape itself. No, I mean really, actually painting it. Oh, and in sound.

If ever you decide to demonstrate your crazy, arcane research, the ideas you dream about and discuss with yourself, sometimes inadvertently aloud - then find you’ve accidentally instigated the biggest, most exciting and terrifying project of your life, don’t call me to complain. I will only laugh.
I was working on how to motion-track listeners so they can walk inside a piece of music - we’re getting there, with amazing work from composer-programmer Iyad Assaf, it’s called 3D-BARE.
I called music tech guru and composer Julio d’Escrivan for advice.
He put me in touch with Enrique Tomas, whose noTours software uses GPS and does a similar - well, different - thing to what I was working on but with such interesting results and rich possibilities that I was hooked.
noTours lets you edit a place with sounds: overlapped, interlocking, spliced, hovering in the landscape.
When a composition is complete, I now do something additional with it - splitting it into horizontal and vertical fragments, spreading it across a garden or along the Thames, then inviting people to come and listen.
I recorded singers a few months ago, one at a time, then combined them into a ‘virtual’ choir, in a setting of a poem called “Take Me By The Hand” for Southampton’s Musical Alphabet weekend.
There’s now a version spread between the paths and trees, buildings and water of the university campus. Singers and the place, sonically and physically bound together. Blurring and augmenting the heard reality of a place allows us to do strange and interesting things…
So I've been constructing musical compositions embedded in landscape and decided to make more systematic my approach to recording the landscape itself and, more importantly, the people in it.
Six months on, I’m coordinating the Audio Portrait of Southampton - to capture the place, the year, its noises, sounds and music. An immersive sonic montage spread across the green spaces of the city for listeners to walk inside and investigate, like a virtual city built only of sound.
Southampton Music Hub and Art Asia have recently come on board, bringing fantastic, diverse musical talent to the Portrait and I was recently interviewed by Xan Philips on Voice FM.
We’ll be demonstrating on 11th October at the University’s next Creative DigiFest, SXSC2. Come and hear for yourself!

Music You Can Walk Inside

HOW CAN VIRTUAL OR SIMULATED MUSIC APPEAR ORGANIC AND ALIVE - AS THOUGH ACTUALLY HAPPENING AROUND YOU?

After several years working outside of academe and music, I returned at last to composing and then to study for a PhD in 2010.

I visited ISVR and encountered their 3-D speaker systems and realised a problem for studio-based composers could be solved by a little invention. 

I proposed a means for listeners to walk inside a piece of music and investigate its parts at will, hearing it differently on each audition. They’ve taken it on and the construction of our 3-D Binaural Audio Rendering Engine is under way!

Here are two reasons to do this: composing in the digital studio, your music can exist either as a data file or be heard using speakers. 

But why would anyone go to a concert to see no-one actually playing? 

Concerts are about far more than listening to sounds among other people - we witness live ‘interpretation’, a musician’s struggle to create beauty and meaning by moving horse hair across a string, blowing down a pipe or banging things together.

Music has always been in flux, perhaps now more than ever. 

But the two ways we listen, at least to what is still differentiated as ‘classical’ - or worse ‘contemporary’ - music have effectively been the same since the gramophone and wireless became widespread, around ninety years ago. 

We either listen to a recording or go to a concert hall and sit still in awed, reverential hush as though the composition were an inviolable object to be revered and recreated.

So, secondly, the idea that a composer’s score somehow is the work has been a part of this problem: of course it is only an approximate transcription of what the composer imagined, just as is the performance. 

They are both attempts at reaching something magical, beyond. 

So if music is produced in a studio, without possibility of being ‘performed’, does the output we hear suddenly become this strange, fixed object that we imagine a composition to be? 

How terrible, if there were only a single way to hear a piece of music, in all its deep-seated reference and memory, refraction of experience and heard sounds!

The 3-D BARE is some way from completion but promises to shed new light on both listening and the compositional process as we rethink how to present work in this way. 

Meanwhile, I have hooked up with a collective of composers and engineers called Escoitar.org (“Listen” in Galician), who have built the amazing tool noTours (notours.org), for situating sounds in a place by way of an android phone connected to GPS, a map the phone can read of where the sounds have been placed, and the sounds themselves, all stored in the handset.

In the last year I have composed pieces where the audience enter a space and moves freely, investigating multiple threads and layers that emerge at different rates, in different forms, around the art gallery, foyer, hangar. . . 

Each listener encounters a different version of the music, a combination of interwoven lines, intersecting at changing points in time, according to their physical position.

Then, they step outside with noTours and, under the satellites that encircle the planet, are guided through the same music, transformed now into an invisible structure, stitched and piled, locked together or floating free, in the landscape itself.

Composing like this is about using a space, integrating with it, reflecting it and its sounds back into the musical world you are constructing. 

The real ambient noise of the place is blended with (and played with, repositioned by digital smoke and mirrors) and replaced in the space, transformed and transposed. 

Now, what is the composition and what is the space becomes hard to determine, and less relevant. 

The experience, I am told, is immersive - the sonic reality of a place is both distorted and augmented at once, heightening awareness both of the sounds constantly around us and of the music situated within it.

Next project, an Audio Portrait of Southampton, a snapshot of the city in 2012: 
   a geo-located composition based in song, music and oral accounts of life in the city. 

Contributors sought: please get in touch now!

Audio Portrait of a City

SOUNDMAP SOUTHAMPTON 2012

Currently developing...

An Audio Portrait of Southampton:
in partnership with noTours

A vast soundscape across the city, using the voices and sounds of the people and the place. . .
Geo-located musical labyrinth constructed from spoken, sung, played words and music of residents of the city.

Listen with an Android phone with GPS connection, using special software that allows you to connect sounds with physical places.
It is a location-specific composition that you can enter like a physical structure, where listeners move around as though inside an enormous live performance, between the very sources of sound themselves, investigating as they choose.
What the listener hears depends on where they are at a certain time and how they move within the whole invisible structure.

It is two things at once:
A collection of oral histories, songs, interviews, spoken word accounts of life in the city in 2012 by the people that live here.
It is also a musical composition, an intricately interwoven sonic tapestry of human life and our environment: an interpretation in sound of the city at this time.

It is being built in two forms:
  (1)  Music You Can Walk Inside: accessible via Android phone with headset, listeners navigate with the aid of a printed map, they walk inside the soundscape for any duration, on any route.
  (2)  Web sound archive with interactive city map, an online archive of contributors’ stories and songs with additional related texts and images: the sounds and images of lives in Southampton, 2012.

We need:
Contributors - stories, memories, reflections, song, music, whatever you want to tell!
Collaborators – bringing individuals and organisations together to make this happen in a big, exciting way, creating a strong media profile to promote an innovative initiative in a vibrant, evolving city.
Funding - events for listeners to walk inside the Audio Portrait, each person experiencing the whole in a unique way: and building the online archive of sounds and images.

We need your help to make this a spectacular celebration and reflection of our city, a massive portrait of real lives and a snapshot of the sounds, experiences, memories and hopes of Southampton in 2012.

Please get in touch.