Sonic Writing

Live Coding: A User's Manual

Thu, 24 Nov 2022 12:00:00 +0000

Live coding has been an active force in new music since the early 2000s. A few years ago, Alan Blackwell, Geoff Cox, Emma Cocker, Alex McLean and myself got together to work on a book on live coding. A user’s manual we imagined it, a handbook, a reference into the theoretical and technical frameworks that circumvent the practice.

The book is now out on MIT Press, and find further information about open access here.

Performative, improvised, realtime, on the fly: live coding is about how people interact with the world and each other via code. In the last few decades, live coding has emerged as a dynamic creative practice, gaining attention across cultural and technical fields—from music and the visual arts to computer science. In live coding the composition happens in realtime, where performers can communicate via sound, visuals, robotic and human movements, or basically anything that can be controlled. In live coding the instructions, the code, is communicated too, typically projected on a screen for the audience to follow, should they be interested.

Live Coding: A User’s Manual is the first comprehensive introduction to the practice and a broader cultural commentary on the potential for live coding to open up deeper questions about contemporary cultural production and computational culture. The book provides a practice-focused account of the origins, aspirations, and evolution of live coding, including expositions from a wide range of live coding practitioners. In a more conceptual register, the book engages with how liveness, temporality, and knowledge relates to live coding, as well as speculating upon the future of the practice.

Check it out in open access here.

Intelligent Instruments: a funded ERC project

Wed, 09 Dec 2020 12:00:00 +0000

The European Research Council has awarded me an ERC Consolidator grant for the project Intelligent Instruments: Understanding 21st-Century AI Through Creative Music Technologies. The five-year, 2 million Euro research programme will consist of a team of postdocs, doctoral researchers and an instrument designer from the fields of music, computer science and philosophy. Further information and job adverts coming out soon.

Further information on the grant on the Iceland University of the Arts website.

NEW: The advert for the 1st PhD position is out: INTENT - PhD Scholarship.

Here is some context to the project (the project website will be up in due time):

Artificial Intelligence is drastically changing the world we live in. Our machines have become creative, equally extending our mind and our body. Amazing technologies are emerging where machine learning can be used to parse large and small data sets, such as music or any musical behaviour, and generate new materials from that learning. New music, new sounds, new workings of our musical tools and instruments.

We have been busy focusing on the technology of AI, but an emerging problem is that our critical understanding and language are lagging behind. The Intelligent Instruments project shifts the focus and through technical development of new instruments studies how AI affects us. And here the humanities become crucial in our understanding of AI and its cultural impact.

The project will study the impact of creative AI, conducted in the research domain of music, with a broad humanities basis, involving musicians, computer scientists, philosophers and cognitive scientists in key international institutions. Through a streamlined research collaboration protocol, we seek to explore the language and discourse of creative AI, addressing our changed notions of, for example, agency, autonomy, authenticity, authorship, creativity and originality.

In order to achieve this goal, the technical approach is to implement new machine learning in embodied musical instruments. We invent instruments that interact, learn, and evolve in the hands of the performer. The instruments become boundary objects, studied by collaborators from a range of sciences and the general public. In three respective work packages that are grounded in phenomenology, sociology and epistemology, we study how embodied creative AI transforms our 1) relationship with technology, 2) social interaction, and 3) knowledge production.

The project will be hosted at the Icelandic University of the Arts (IUA), where I’ve been appointed Research Professor. I will divide my time between IUA and University of Sussex where I’m a Professor of Future Music, heading the Music Department and co-directing the Experimental Music Technologies Lab. At Sussex I’m a member of the Sussex Humanities Lab which will be one of the key platforms for the project outside Iceland, together with Cambridge Digital Humanities.

The halldorophone instrument which will be used, amongst other instruments, as a test bed for implementation of creative AI

::: ERC Project Abstract :::

Artificial Intelligence is becoming increasingly human-like and it is now proficient in a key human activity: musical creativity. But what does this mean? How does creative AI change our notions of art, culture and society? As new machine learning technologies begin to mirror ourselves, we need to look into that mirror and ask how AI is changing us. This project takes a pioneering leap in research about AI by answering how new creative AI transforms our relationships with technology and other people.

This ambitious vision is achieved by using music as a platform to establish public understanding of AI. Through technology development we will create the conditions to study higher level theoretical questions on the meaning of creative AI in contemporary culture. Three respective work packages will develop: 1) instruments with creative AI; 2) human-AI collaboration in music; and 3) sonic instruments as scientific instruments. The project initiates a public discourse on creative AI and develops a theoretical framework describing the transformed notions of self, others and knowledge when we adopt intelligent instruments in our work.

The Intelligent Instruments project is interdisciplinary in nature. Applying the methodology of our new research collaboration protocol, we summon researchers from diverse disciplines to conduct frontier science on intelligent instruments as boundary objects. Through open science methods the outcomes will address: a) the role of creative AI in embodied technologies, and b) the understanding and reflection of artificial intelligence in future society. Grounded equally in technology development and the humanities, the project will benefit diverse disciplines by developing a theoretical framework of creative AI, initiating a discourse around human-centred creative AI, and defining principles of human-AI relations in services and products.

Creative AI education from the MIMIC project

Fri, 26 Jun 2020 20:20:00 +0000

We are reaping the rewards from the hard work in our MIMIC project this winter. We are introducing two educational projects on creative coding with machine learning, both free and open to all.

Our Emute Lab Sussex team is running a workshop on live coding language design for machine learning. We are about to launch our Sema live coding system for machine learning. The public launch will take place just before our workshop on Sema, and registration is still open: Sema Workshop

A screenshot of the Sema live coding environment for machine learning

There is also a Creative Machine Learning MOOC set up with FutureLearn. People can sign up for the workshop called Apply Creative Machine Learning : Discover the creative side of machine learning with this free course using hands-on examples.

The FutureLearn MOOC is not on live coding but the general application of machine learning in creative coding. We ask how machine learning can help in creative acts, how we might see it as yet another tool in our diverse toolbox of creative applications. Participants will gain understanding of the key principles of machine learning and how to apply these technologies in the production of new art.

Apply Creative Machine Learning

Previews, Interviews and Reviews of Sonic Writing

Mon, 27 Apr 2020 20:20:00 +0000

My book, Sonic Writing, was published a year ago. I’ve been happy with its reception and to learn that people whose work I’m continually inspired by are praising the book, using it in their teaching and recommending it as required reading for lab members. The book would not exist without these people and I’m glad that ideas are feeding back the other direction too. Here below I present a mix of interviews, presentations and reviews of the book I’ve participated in or been made aware of.

The first review of the book was by Gregory Taylor on the Cycling74 page: Book Review: Sonic Writing

The prolific Darwin Grosse interviewed me for the Art+Music+Technology Podcast

I discussed the book with Eamonn Bell from the New Books Network.

A little teaser presentation of the book at the Instrumental Shifts Symposium during the Dutch ReWire festival:

Neural Magazine had a review of the book in their February Issue

Antonio Poscic wrote a review for the February 2019 issue of the Wire Magazine:

A fine review by Justin Pearson on Sonic Writing

Diana Chester wrote a review for the Issue 7 of the Interference Journal

John Dack wrote a review in The Journal of Technology and Culture

Davic C. Jackson presents a review in The Sound Studies Journal

Jack Armitage wrote a review for the International Journal of Performance Arts and Digital Media

Angel Alvarado Cabellow contributes an in-depth review in the Journal for Artistic Research

Pedro Sarmento wrote a review for the Journal of Science and Technology of the Arts

Anna Xambo wrote a review for the Organised Sound

Sonic Writing - A book on new musics and technologies

Thu, 21 Feb 2019 09:00:00 +0000

Sonic Writing is now out with Bloomsbury Academic. This monograph is the final outcome of an AHRC funded research project that ran between 2016 and 2018. The book is divided into four key parts: instruments (material inscriptions), notation (symbolic inscriptions), recording (signal inscriptions), and new music technologies, or how our analogue and electronic traditions are translated and applied in the design of digital instruments.

Sonic Writing explores the unique relationship we have with our musical instruments - our technologies for making sound. I’ve come to realise there are very few things that humans have such an intimate and caring relationship with as musical instruments. These technologies have evolved over millennia and through instruments, systems design, theory, notation, phonography, computational creativity and now machine learning and AI, music has always been at the cusp of our technological development and understanding. Musical instruments are unique technologies: they are at the same time an instrument to look into our inner self and to speculate about new cultural forms. Sonic Writing tries to understand the role of technologies of music making in wider cultural practices.

As part of this research programme I ran symposia and workshops at STEIM, IRCAM, C4DM, CMC and CNMAT; and a writer’s residency at the Jan Michalski Fondation - all events, places and people that greatly inspired the way the book developed. Through discussions with colleagues working on cutting-edge research, a more refined understanding of the integrated relationship between contemporary music and its technology began to emerge, and I hope I was able to represent that vision in my book.

As part of my travels, I had the opportunity to interview people whose practice shapes the changes I describe in my book. I am extremely grateful to these people: Kristina Andersen, Margaret Birley, Ken Butler, Ed Campion, Suzanne Ciani, Nick Collins, Pierre Couprie, Pete Furniss, Brad Garton, Rama Gottfried, Derek Holzer, Hans Jóhannsson, Shinji Kanki, Roger Linn, James McCartney, Alex McLean, Andrew McPherson, Claudia Molitor, Sarah Nicolls, Godfried-Willem Raes, Christopher Redgate, Ryan Ross Smith, Laetitia Sonami, Laurie Spiegel, Dan Stowell, Bob Sturm, Enrique Tomás, Ge Wang, Anna Xambó, and Pamela Z. I have the feeling that more material will emerge as a result of these interviews.

You can read a sample chapter by clicking on this image:

Although I enjoy writing, initially I wasn’t sure if a printed book would be the right format for this type of research. Perhaps an online publication where I could link to sounds, videos, pictures, and artist websites would be more practical? Where people could comment, add notes, and further contribute to the development of the research. However, in the end the printed book format was simply too attractive, or, as it says in the preface: “there is a quality of the printed format that I value: to constrain, focus, write, rewrite, rehearse, rethink, review, edit and re-edit in an iterative process that involves colleagues, editors, proofreaders, and myself operating in a tradition that extends back to antiquity and is endowed with its own rules, protocols, and formats.” And I don’t regret this decision - hard work, but worth it.

The cover artwork is a piece by Ryan Ross Smith, Study 56. It is an animated notation piece where performers play from a computer generated dynamic score. I think this beautiful still suits the cover of the book, as the book represents a definite moment in time, a snapshot of the dynamic everchaning field of music, just like a screengrab of an animated score. The piece is a generative application, and one rendering of it can be seen here below:

There will be a launch party in the Sussex Humanities Lab on March 14th - all welcome! In addition to drinks and snacks, we will hear music by Feedback Cell, Alex Peverett and Andrew Duff, and Evelyn Ficarra. All of these people have inspired the book, so I’m thrilled to have them performing during the launch.

Bloomsbury gave me a discount code that can be used if you purchase the book on the Bloomsbury website : “GLR MP6”

Ergodynamics - on the haecceity of musical instruments

Tue, 12 Feb 2019 11:00:00 +0000

There is something magical about musical instruments: we pick them up, tune into their mode of communication, and we transcend into a different form of life. The instrument is what constitutes this possibility of a different thought and voice: it is a torch into our inner being. Trying a new instrument for the first time is a peculiar encounter: we rotate the instrument, observe it from all angles, pluck it, blow it, stroke it, shake it, hit it and generally explore what it offers us in terms of interesting expression. This process can take hours, weeks, years.

Having become familiar with the instrument, we roam its expressive scope, as if exploring a landscape that is covered by a deep fog, but there within we come upon beautiful local areas that entice and attract. This is about finding the constraints of the instrument, a path to mastery that requires time and effort: a process of discovery, of research. Not only are instrumental types different, and we operate with complex classifications of musical instruments (see the Musical Organics blogpost), but every individual instrument is unique! No two instruments are the same when the come from the luthier or the factory, and they further develop in your hands, gaining a unique form that separates it from other instances of the same type from the same maker. This is the instrument’s haecceity, its unique qualities, place and behaviour in the world.

Christopher Redgate's 21st Century Oboe

When discussing these things with people, I have been seeking a vocabulary that covers how we talk about our instruments. A terminology that is similar to how ludologists might deliberate about the “gameplay” of a computer game, as its general feel, play, navigation, aesthetics, game mechanics, pace, possibilities, etc. What analogous word to “gameplay” can we use in organology? Design, playability, character, tone, affordances, constraints, remediation, ergonomics, workflow and expressiveness are not ideal words as they are specific, limited and often technical. I became interested in developing a terminology based on the function of the instruments, their working nature and character. Ergos is the Greek term for work (and we can see the direct etymological relation to “work”, “werk”, “verk” in other European languages). It also relates to the term “organ” which is Greek for instrument, but one that is equially a bodily organ (like the heart), a tool (like the hammer) or a musical instrument (like the harp). So instruments work, they operate and are operated; they work and are worked. Instruments are functional tools, they perform functions and are pregnant with undiscovered potential. This is the dynamic range of the instrument, or its dynamis.

In the search for terminology I therefore began exploring whether the term ergodynamics would work as a term describing the potential of an instrument, its range, scope, expressivity. Like the ludologist would say “this game has a good gameplay” we musicians might now describe an instrument as having interesting ergodynamics. This is referring to the mystery, depth, hidden features, discoverability embedded in the object. And what amazing objects musical instruments are! Humans rarely engage with other objects this way, although I must admit that I have talked with skateboarders about skateboards, chefs about knifes, dentists about drills, archers about bows and whilst those are also quite obsessed with their objects, the musical instrument constitutes something more: it’s a way to express inner states withouth the crude and clumsy formulations of language. Human thinking without having to put the world into the rigid categories of symbols.

Related terms became ergomimesis, ergography, and ergophor.

Coming up with a group of descriptive terms for musical instruments as a contribution to organological vocabulary is a strange thing to do, so in my self-doubt I discussed with colleagues, with Greeks, with instrumentalists, with linguists and with luthiers. I presented a paper at a conference, wrote an article in a journal, and gave a few talks and workshops about this, all with the aim to see if people could help me realising how ridiculous this excercise was. Well, they didn’t. So I used the concepts in my forthcoming book, Sonic Writing. More on that book very soon!

References:

Ergomimesis: Towards a Language Describing Instrumental Transductions
In the proceedings of the 2018 International Conference on Live Interfaces in Porto.

Ergodynamics and a Semiotics of Instrumental Composition
Published in the Tempo Journal in January 2019.

Sonic Writing: Technologies of Material, Symbolic and Signal Inscriptions published by Bloomsbury Academic in 2019. [Here is a 35% discount code: “GLR MP6”]

Presenting Musical Organics - Ideation & Feedback

Tue, 26 Sep 2017 12:00:00 +0000

As part of the Sonic Writing research project, I’ve been thinking about the classification of digital instruments, since the materiality of these technologies poses considerable problems for traditional organologies. Philosophical ideas mature slowly, and in my experience they develop best in conversation, presentation, or practical development, where technical problems and alternative views emerge.

Ideation is the process in which ideas are born and take form, for example through language, sound, paint, or machines. The intuition I had for the concept of “musical organics” was based on over two decades of working with all kinds of musical instruments, building and playing them, but always sensing the different nature of our new digital instruments; a certain unease always accompanies the first encounter with a digital instrument. It is not easy to express this feeling, and in conversation with people one senses this unease in what is being said often implicitly, typically tacitly. Now, I love what the computer has enabled us to do in music — what a tool! — but there is something unique in the relationship performers have with their acoustic instruments, with the wood, strings, metal, bones, horsehair and other materials that are so full of character, all so different, to the degree that each and every instrument is unique (a violin bow you pick up in a shop might be very different from an “identical” violin bow lying next to it). And that’s when they are new and have only been manipulated by the hands the luthier - imgagine their difference when they have been “played in” by the instrumentalist!

This is no news to anyone who has a background in traditional instruments but shifted over to the computer when it became powerful enough for musical expression. (Enrike Hurtado and myself did a survey on that question in 2006, with the results published at NIME 2007, with a little upgrade for eContact, and now more recently for the NIME Reader). I’ve been interested in this question for a long time, or since I parked my electric guitar for the new languages of the computer. My perceived sense of alienation from the musical object was immediate: the phenomenological complexity in digital instruments becomes so prominent when compared with acoustic instruments, but this is also what makes the digital domain so powerful (a bit like how AI helps us to do work in cognitive science).

The development of the concept of musical organics involved various engagements: a symposium organised with STEIM, a paper presentation at NIME, a presentation at the DPASSH conference, an article for the Journal of New Music Research, and a provocation at the Touching Sound symposium held at City University. Through these events, following discussions with colleagues, the concept of musical organics is emerging and taking a fuller shape. This will be described here below.

From Kirscher's Musurgia Universalis

The Musical Organics Symposium at STEIM in Amsterdam, held in May 2016, involved instrument makers (designers, luthiers, programmers, engineers), composers, and performers. The idea was to summon a diverse group together for presentations, performances and workshops. The first day ran with presentations and performances, followed, on the second day, by discussions and workshops. All of this extended into the early hours in the wonderful restaurants and bars of Amsterdam. The participants in this symposium all came with their own research agenda, and judging from feedback it was a useful experience for them. I myself had a few theoretical irons in the fire and I used the two days to observe, probe, and interpret the proceedings. I presented an early draft of the musical organics idea, and got very useful feedback.

The time then came for actual research. Having been interested in classification since spending a good part of 1997 reading the whole of Nietzsche’s writings, I subscribe to his pessimism in natural language and our conceptual systems. Classification is about understanding, about ordering, about putting things into place, and that is why, in the 17th century, when modern science is born, we find this obsession with measuring, analysing, and classifying. If we are to understand digital instruments, a natural approach is the attempt to analyse their properties, put them into groups, order them, classify (this is of course useful if we are organising a book about instruments, creating an online shop, or arranging the instruments in a museum). However, provided the diverse origins and disparate fluid nature of digital instruments, we are bound to ask ourselves how this might be achieved? Greek theory, medieval classification, 16th and 17th century approaches, all culminating in late 19th century and early 20th century tree-like classifications; it is clear that this is a road to despair. Instead, a fuller idea emerged, from the vague intuition that drove this research from the start, namely a non-hierarchical, interconnected, perspectival system of dynamic ordering. If we can’t put place these phenomena onto trees, let’s reticulate them in a web that can be rearranged according to the perspective the researcher is interested in.

These thoughts have now been published in the Journal of New Music Research. The article, “Musical Organics: A Heterarchical Approach to Digital Organology,” (it is published as Open Access - download here) serves as a theoretical framework for a potential practical project that I, or someone else (a PhD project?) would develop in the future. However, in order to test the potential of the system I wrote a case-study of how a plugin to this system might work, and this was published in the Proceedings of NIME, a conference held in Copenhagen, in May 2017. The feedback from NIME was invaluable: I discussed the project with organologists, online instrument database curators, instrument designers, composers, and performers. The concept resonates with people, but we agree that the technical hurdles are big, challenging, and interesting. That was the feedback I also got from participants in the DPASSH conference, an altogether different gathering of people, in this case information scientists, digital humanities people, librarians, computer scientists and engineers. It was a real privilege to be able to present the work for them and learn from their feedback. Finally, at the Touching Sound symposium, organised by the SPARC people at City University in London, I had the opportunity to present work on the digital organology as a “provocation,” which is a great format when testing new ideas.

I feel I have gathered enough insight to take this project further. However, it is time consuming and my busy schedule does not allow for such core development. I would welcome collaborations, PhD proposals, or suggestions for grant proposals, as I am confident that this project will become reality in the near future. The next step is to discuss it with my colleagues in the Sussex Humanities Lab.

Sonic Writing Residencies in the US

Tue, 23 May 2017 12:00:00 +0000

I spent a few weeks during April and May in the US, visiting my research partners there: CNMAT at UC Berkeley and CMC at Columbia University, New York. The aim with the trip was to explore these key institutions, study their archives, interview people, and present my own research as well as organising a symposium and a workshop. This was an extremely fruitful trip: these two institutions have been key actors in the development of electronic and computer music for decades. CNMAT was founded in the late 1980s, and has been the location of some important research (I might mention the OSC protocol, the o-dot library, but also countless new music systems, pieces, and performances) under the direction of the interdisciplinary inventor and musician David Wessel. The Computer Music Center at Columbia has a longer history, founded in 1958, as the Columbia-Princeton Electronic Music Center, by Vladimir Ussachevsky (1911-1990) and Otto Luening (1900-1996). Milton Babbitt began working at the center shortly after its inception (hence the early Princeton connection). Other composers affiliated with the center include Jon Appleton, Luciano Berio, Wendy Carlos, Charles Dodge, Halim El-Dabh, and Edgard Varèse. The RCA Mark II Synthesizer was a flagship technology of the centre right from its foundation and it is still standing, although not in a functional state.

The RCA Mark II synthesizer at CMC Columbia. This is actually bolted down in Brad Garton's office.

I was fortunate in that shortly after I arrived to the Bay Area, the Recombinant Media Labs (RML), the Associates of Don Buchla, Gray Area, and Obscura Digital organised a two-day festival in celebration and rememberance of the synth pioneer Don Buchla. The festival was jam-packed with interesting performances, panel discussions, and an exhibition of Buchla technologies. See the programme here.

Panel chaired by Geeta Dayal, with Ramon Sender, Morton Subotnick, David Rosenboom, and Alessandro Cortini.

Panel chaired by Marc Kate, with Tom Oberheim, Dave Smith, Roger Lynn, Keith McMillen, and Jessica Rylan.

There were some fantastic musical performances (and I’ll link to a clip of Suzanne Ciani’s performance here below), and I really enjoyed the two panels too. Buchla was an inspiring, life affirming fellow whose curiosity and engagement with new technologies (and perhaps more importantly, the mind) never ceased to take his explorations into new directions. I will probably post the two panel sessions soon on the Sonic Writing YouTube channel.

The Sonic Writing book project is taking shape and it was great to meet up with the Bloomsbury editor in New York. Considerable part of the trip was spent on meetings and interviews with some very interesting practitioners and researchers and I interviewed the following people (in this order): Roger Linn, James McCartney, Laeticia Sonami, Suzanne Ciani, Pamela Z, Ed Campion, Rama Gottfried, Brad Garton, Ken Butler, and Laurie Spiegel. An amazing group of people, all of whom have contributed so much to the way we think, compose and perform music with contemporary technologies.

I am still digesting the interviews. They were all mind blowing in their own way, but if there is anything that appeared as a common thread in these discussions it might be the sensation that we are at the cusp of some drastic changes in music technology, in the way we as humans make, perform, and listen to music. Common observations regarded a certain fatigue with the digital media, caused largely by how strong and few the new media moguls have become and how their products affect social and political life, disillusioning anyone who was optimistic about the internet in its early days. This fatigue could be exemplified by a retro movement of modular synthesizers, acoustic instruments and hybrid versions of those, and DIY in instrument making. But parallel we observe developments in machine learning, where, in the past few years, some incredible developments have taken place. Machine learning will be used at all levels of musical creativity in the future, from computational creativity (composing whole pieces of music), to mapping and personalisation features in our musical instruments, to core sound synthesis, where parameter morphing will be achievable at a level unprecedented by physical modelling or concatenative synthesis.

We might see a parallel in how new music technologies in the past were the domain of a few privileged people who had access to the equipment at Bell Labs (such as Matthews and Spiegel), CMC (Ussayevsky and Babbit), or the San Francisco Tape Music Center (Buchla, Subotnick, Sender) - and later IRCAM, the Milano Studio, or STEIM. Equivalent technologies are all mostly available today for free in people’s laptops. However, what we consider today as the privileged spaces of new music are, for example, centres for spatialisation, like the wavefield synthesis systems at the Technical University in Berlin or IRCAM, or institutions with strong computer networks that enable research in machine learning applied on multicore GPU matrices. Access to such computer networks is limited to downtime at relatively few research institutions - and this is reminiscent to how computer artists in the 60s and 70s would work in the night in order to gain access to the mainfraime computers of that period.

America was kind to me and I had a fantastic time. There was a large and depressing political shadow hovering over people, but living in the UK, I am accustomed to that.

Below I post a link to a recording that just came out when I was in New York. It’s a documentation of Tony Conrad’s collaboration with Laurie Spiegel and Rhys Chatham, performing in The Kitchen in 1972.

Sonic Writing workshop at Lydgalleriet

Tue, 07 Feb 2017 09:00:00 +0000

This three day workshop is offered to students at UiB (The Faculty of Fine Art, Music and Design), BAS, (Bergen School of Architecture), and Creative writing (Skrivekunstakademiet i Hordaland). The workshop will explore our writing practices with a particular focus on sound. Ranging from abstract poetry, concrete poetry, textual musical notation, algorithmic notation, secondary musical notation, and graphic notation are all forms of writing that we will engage with during the workshop. We will make musical instruments, considering how their design is a process of writing. The workshop will be highly interdisciplinary - with participants from institutions of divergent practices - and we will apply knowledge from the diverse fields to the experimental and explorative process in thinking about sound, writing sound, about composing instruments, writing algorithms, speaking notes, drawing movement in time.

In this workshop, we aim to rethink the established notions we have of instrument, notation, score, poetry, sound, word, letter, gesture, movement. With a starting point in the idea of the metaphor as a process of signification, a prescribed mental movement, we will ask probing questions such as: What is the significance of sound? What’s the role of sound in poetry? What is the role of the visual sign? How do we write sound? Notate? Can we make tangible scores? Can instruments be written? How does a poem inherit music? How can tonality, tonal and timbral variations change the semantics of a recited poem?

The workshop will start in the morning on a meeting and a discussion about the day’s topic. We will form small groups of people who collaborate over the day on a project that will be presented and perhaps performed in the afternoon. After the presentations we will discuss the projects and reflect upon our experience of the day. On Sunday the groups will present their final projects in a public event.

Example project descriptions

A rock, a can, a bottle, a string. What kind of instruments can be built out of the junk we find around us? How does the human body relate to the instrument?
Which symbols do we create for describing sound? Notation for new instruments require new thinking, new design
Beckett's Ping. How does this story "sound"? How is it notated? How can it be recited? What sounds would be appropriate?
A sheet of paper and a pen. How to notate on paper? Can the paper become an instrument? What's the sound of paper? What's the sound of the symbols written on the paper?
How do variations in the recital of a poem change the semantics or the impact of the receiver?
What is a tangible score? Can an instrument contain a musical score?

Times

Morning sessions: 10:00-12:00 Wednesday, Thursday, Friday
Afternoon sessions: 15:00-17:00 Wednesday, Thursday, Friday
Final presentation session: 10:30 - Sunday (A Sonic Writing Brunch)

A more precise time-table with topics will be presented at the start of the workshop.

Prerequisites

Participants don’t need to have any particular skills to join this workshop. We will make instruments out of rubbish, we will compose music with paper cuttings, we will find rhythms in poetry, and study sound in space. In order to achieve this, it is best to be in the possession of, in the spirit of Zen, an empty mind.

Sunday Presentation

Having worked on interdisciplinary investigations over the duration of the workshop, we will conclude the workshop with a public demonstration of projects, ideas, pieces, performances. These projects can be at all stages of development; indeed, we often find it more interesting to discuss work-in-progress than finished pieces. The presentation will be about ideas, about the clouds of dust generated by the workshop discussions, and not about concrete solutions. We welcome everyone to join us at this event of explorative investigations into sonic writing.

The Workshop Leaders

Bergsveinn Birgisson is writer based in Bergen. He has published three books of poetry, four novels and a doctoral thesis focused on the cognitive and aesthetic aspects of Skaldic poetry from pre-Christian times. He writes in Icelandic and Norwegian. He has studied different traditional chanting techniques in Scandinavia. In his doctoral thesis he developed a analytic-method on poetry based on cognitive linguistics and psychology. Bergsveinn is currently working on a novel where feelings are in focus.

https://no.wikipedia.org/wiki/Bergsveinn_Birgisson

Thor Magnusson is a lecturer in Music at the University of Sussex. His work focusses on the impact digital technologies have on musical creativity and practice, explored through software development, composition and performance. He is the co-founder of ixi audio, and has developed audio software, systems of generative music composition, written computer music tutorials and created two musical live coding environments. As part of ixi, he has taught workshops in creative music coding and sound installations, and given presentations, performances and visiting lectures at diverse art institutions, conservatories, and universities internationally. Thor is currently working on a book called Sonic Writing, to be published by Bloomsbury in 2018.

www.sonicwriting.org
www.twitter.com/thormagnusson

Special Issue: Live Coding

Fri, 27 Jan 2017 13:00:00 +0000

The third issue of the twentieth volume of the International Journal of Performance Art and Digital Media was about live coding. Kate Sicchio and I edited this issue, and after a process that took about a year and a half, from early stages of planning to publication, the issue is out. We are very happy with the result: a publication of a cutting-edge technology of music writing by some fantastic authors, including Emma Cocker, Sally Jane Norman, Tim Sayer, Karen Burland and Alex McLean, Hester Reeve, Hannah Elizabeth Allan, Sam Aaron, Andrew R. Brown, Mauro Herrera et al., Chris Kiefer, Charlie Roberts, IOhannes m. zmölnig, with book reviews on Andy Clark’s Surfin Uncertainty by Jelle Bruineberg and Tom Froese, and the edited volume Digital Movement by Sarah Whatley.

Kate and I would like to thank Professor Dave Collins for the great collaboration on this special issue. Dave is the founder of the journal and this issue was actually his last as the journal’s editor. We also congratulate Dr Maria Chatzichristodoulou on her new role as the editor in chief. IJPADM is an excellent platform to think about new technologies in the performing arts. The current call is on “Bodily Extensions and Performance (Avatars, Prosthesis, Cyborgs, Posthumans)” and can be found here. (the extended deadline is January 31st).

Sussex colleague, Chris Kiefer, live coding. Chris has an article in the journal issue.

This is our editorial:

Writing with shaky hands

I am no doubt not the only one who writes in order to have no face. Do not ask who I am and do not ask me to remain the same: leave it to our bureaucrats and our police to see that our papers are in order. At least spare us their morality when we write. (Foucault, 1972)

In his introduction to the Archaeology of Knowledge, Foucault objects to be defined as a writer of a solid identity in order to be able to move, change, and explore the unknown – labyrinths and underground passages – admitting that such writing is at times done with a ‘shaky hand’. This journal issue explores live coding, a performance practice that operates in a similar spirit of writing as an adventure and exploration, of deliberately rejecting definitions, of being heterogeneous in nature, and continually challenging its self-understanding through the practice of writing and rewriting – of defining and redefining – as a public performance.

Although live coding attempts to escape definition, there are denoting statements that have traditionally described its activities, such as: writing software in real-time; changing a programme whilst it’s running; projecting the screen for the audience to participate in; writing as an improvisatory practice; composing live using textual notation; changing rules whilst following them; conversing with the computer in its own native language; thinking in public; creating and using bespoke systems tailored for on-the-fly performance.

Live coding is an improvisatory performance method that has been applied in the diverse art forms, but historically it has been very prominent in music, perhaps due to the importance of the musical score, computer music, and the technological foundations of musical practice in general (as demonstrated by the history of musical instruments). Live coding can be used as a compositional technique in the studio without an audience, but in this issue we focus on live coding as a performance method where code is typically projected on the wall for the audience to engage with – observing the thought process and the composition as it happens, as well as partaking in interpretation (understanding how sounds, visuals, or movements derive from the notation).

Live coders programme, they write in public (Greek: pro-graphein) – but they also pre-gramme, that is, their algorithmic writing is conditioned by a system that has already been designed with careful considerations of expressivity, constraints, interface, and other concerns of human-machine interaction and performer-audience communication. Most live coders adopt or develop a system best matched to the ways they wish to communicate the writing of their work – manifest as music, visuals, or other art forms. It is a writing of a writing that will happen at a later stage, on a different stage! The initial multimodal system thus offers creative solutions to the specific problematics of the stage, the real-time actuality of the performance, and the openness of improvisatory practices. Whilst the live coding system is pregrammed, the subsequent performance act of programming processes digital media as fluid entities whose functionalities are subject to development and change.

The aim of this live coding special issue for the International Journal of Performance Arts and Digital Media is to present live coding in its broader historical and aesthetic context; the issue offers a set of theoretical papers supplemented with focussed artistic statements exemplifying diverse live coding practices. Emma Cocker’s article Performing Thinking in Action: The Meletē of Live Coding applies the Greek term meletē to describe live coding as a meditative thought experiment; one that is publicly shared, a form of ‘thinking-in-action’. Live coding is here framed as a site for experimentation and negotiation between the spontaneous and the planned, the human and the machine. In Setting Live Coding Performance in Wider Historical Contexts, Sally Jane Norman traces live coding practices of projecting liveness and poetic modelling back to historical examples involving the Sun King, Louis XIV, amongst others. A key theme in the article explores how live events are crafted as uniquely artistic material, overriding habitual frames of reference. This is a theme picked up by Tim Sayer, whose article Cognitive Load and Live Coding: A Comparison with Improvisation using Traditional Instruments, explores the cognitive mechanisms applied in improvisation, specifically comparing the improvisational practices of live coding to those of instrumental music.

Several articles address audience perception in live coding, defining it as a practice where most commonly artists choose to ‘show their screens’. In Understanding Live Coding Events, Burland and McLean provide a study that aims to ‘explore the motivations, experiences, and responses of live coding audiences and to examine their perceptions of the role and impact of the projected source code during live coding events’. This is an area that has not been investigated previously in any depth within the live coding literature, but live coders’ gesture of projecting code sets up a relationship between performer and audience that is rather unique in the performing arts.

The connections of live coding with live art are also found within this special issue, with Hester Reeve reflecting on her work collaborating with live coders, and placing it into philosophical discourse. Reeve’s Live Code, Live Art and the BwO Dissection draws out similarities in the practices of live coding and the practices of live art, and in (en)Coding Performance: From Analogue to Digital, Hannah Allan looks to Fluxus performance work in order to find text-based pieces which echo work found in the live coding community.

In conceiving this issue, we decided to call for artist statements in order to establish a more concrete grounding to some of the philosophical arguments presented in the longer articles. These statements focus on issues arising directly from performance, including designing performance systems, playful interpretations of code, and pedagogical implications. The creation of a live coding environment that serves equally for virtuoso performance and as an educational tool is the subject of Sam Aaron’s article. In Sonic Pi – Performance in Education, Technology and Art, Aaron discusses the development of Sonic Pi and argues that teaching is by nature a performance – any good educational tool should reflect that fact. Andrew Brown’s artist statement, Performing with the Other: The Relationship of Musician and Machine in Live Coding, delves deeper into the performer-machine relationship with a phenomenological discussion of his own performance works. For Brown, the live coding system is not a mere instrument, but the live coder’s other – an entity that can respond to the live coder and exert its own character during performance.

Other artist statements explore audience engagement through a variety of live coding practices. The Mexican live coding community’s contribution to this issue is a report authored by Mauro Herrera Machuca, Jaime Alonso Lobato Cardoso, José Alberto Torres Cerro and Fernando Javier Lomelí Bravo, reflecting on their performance system that merges natural language and code. For Herrera et al. have created framework that explores the understanding of code as literary commands aimed at audiences who may not have a computer programming background. Chris Kiefer discusses his web-based experiment ApProgXimate Audio as a way to invite participants to live coding practices through a system that applies simple commands to explore complex sound synthesis. Kiefer wants his work to be accessible to lay audience and novice live coders and he therefore applies the Web Audio API, an audio synthesis framework that runs in most modern browsers. Charlie Roberts’s Gibber system is also written in the Web Audio API, and it enables live updating of code, where the text can respond to sound through the events it defines as notation. Roberts also discusses public engagement through the use of visual elements to reinforce audience attention throughout performances. Finally, IOhannes m zmölnig critically explores the issue of how text is understood by the audience on screen, discussing his multi-layered representation of code, simultaneously hiding and revealing, thus lending itself to multiple readings by audience members.

In reflecting on the articles of this live coding special issue of IJPADM, the culture of live coding emerges as one that is not overly concerned with the act of programming itself, but rather with the techno-social aspects of its performative systems, representations of composition, rule-based improvisation, and performance activity. These critical discussions are not on technology per se, but rather on how audiences, artists and learners read, engage, and create with code that is developing in real-time, on the fly, or live.

Live coding is maturing as an artistic performance method. This history has been documented from its beginnings as a focussed practice in the early 2000s (see Collins et al. 2003), to Magnusson’s (2014) discussion of products, practices, experiments, and performances drawing on 10 years of hindsight. The latter article speculates on live coding as a method that will become increasingly ubiquitous in diverse art forms, to the degree that we may not need the term live coding in the future. As this special issue demonstrates, the activities currently denoted as live coding are methods that can be seen to have historical precedents in the arts, but they also relate to problems studied in computer science, often under the term live programming (Tanimoto 2013). Presenting live coding in these broad terms, relating it to past practices and related research fields, makes it increasingly hard to incisively define. It appears that the popularity of live coding can be explained by its appeal to people growing up with interactive media who reject being subjectified as passive consumers of technology. Live coding represents a mindset of sorts that conceives of technology as fluid phenomena, offering an openness to change and exploration, such as live redesign of instruments, live rewriting of scores, or indeed of writing with ‘shaky hands’ in order to have no face.

References

Collins, Nick, Alex McLean, Julian Rohrhuber, and Adrian Ward. 2003. “Live Coding in Laptop Performance.” Organised Sound 8 (3): 321–330.

Foucault, Michel. 1972. The Archaeology of Knowledge and the Discourse of Language. New York: Pantheon Books, 17

Magnusson, Thor. 2014. “Herding Cats: Observing Live Coding in the Wild.” Computer Music Journal 38 (1): 8–16.

Tanimoto, Steve. 2013. “A Perspective on the Evolution of Live Programming.” LIVE 2013 symposium proceedings, ICSE conference, San Francisco.