Automating Creativity
Karl Marx famously said that man is himself and his instruments of production. Musicians, of course, do not live alone in their own creative bubble. Indeed, the shifting frontier of music making is arguably best observed today by dropping in on a gathering of audio and game developers, audio and technology manufacturers, audio production specialists, and musicians. Changes in the way we make things are likely to carry over into music production, and a list of such carry-ons might include more automated processes in the creation of music itself, the use of artificial intelligence in the selection of musical events that make up compositions, and even the outsourcing and more efficient use of creative nuggets to bring out the right musical muse.
The Advanced Audio & Applications Exchange (A3E) conference, recently held in Boston, afforded the MBJ a good opportunity of observing music futurists in action. An instance of this was Google’s current involvement with the development of new standards for Web Audio production. Such standards would enable musicians to collaborate creatively in different physical spaces instantaneously by just using their Internet browser, avoiding the current incompatibility of user plug-ins and mismatches of software translators (the reader should consult our interview, elsewhere in the issue, with Google’s Chris Wilson).
Stems
Here we choose to report on the new use of ‘music stems’ for purposes of musical creation. Irish startup Score Music Interactive (SMI), for example, is beta testing Xhail, a program that is meant to automate music production for film, advertising, television, and video game projects.
SMI uses templates of chord changes that it outsources to chosen musicians and composers around the world. The idea is that these musicians and composers would create unique compositions based on the templates—and supply one or more recordings of their work, known as stems, on a single instrument. Each stem would be sent back to the content library, and then a team at SMI would listen and tag emotions and other characteristics from a predefined list of fields (romantic, slow, moonlit, and so on). When a producer or music supervisor asks for material for a project, SMI would look for the various stems in the cloud and put them together to create a completely new, and original composition.
Since each individual composition was composed against the same template, once they are drawn together, every part will work harmonically. If the user is not satisfied with a specific part or perhaps even the entire composition, SMI will extract different stems to add a new part or even create another entirely new piece.
Mick Kiely, the Irish founder of SMI, built a reputation as one of the leading composers for Irish television programs and video games and received international recognition, signing publishing deals in Ireland, the U.K., and the U.S. In 2010, while figuring out how to integrate enough content to drive music in a game engine for Xbox/PS3, Kiely wondered if, under the right conditions, music stems could be blended together, arguing that “if you look back in history, thousands and thousands of songs have been written to similar chord-maps or identical chord-maps, the Beatles did it all the time.”
This begs some elaboration. Even if Kiely was right, any musician that is a Beatles fan will know that the devil of the music is in the details and that whereas rhythm-and-blues inspired much of their music, the Beatles blended chord, melody, and lyrics in ways that surprised and won the admiration of listeners and musicians worldwide.
The Business of Stems
If Kiely is suggesting that randomly made music from trunks of common harmony have a place hitherto unexplored in the music business—and for starters, this would work well with video games—the licensing of such music is intriguing.
To begin with, Kiely argues that for the first time in history, session musicians will be given a publishing deal: “if you play harmonica really well…we want to get harmonica tracks that work interactively with our templates, and [we will pay you] every time that [your] harmonica is used.” This is because SMI wants to be constantly building up their sound database in order to keep generating completely unique compositions with the Xhail platform. The incentive acts as a supply management tool.
Secondly, Kiely claims that every final composition licensed will never be generated again for any other user: it remains unique to that user and that product. Individual stems, however, will return to the content library to be used multiple times in different unrelated combinations. Nevertheless, if the original user decides to use the same composition in a sequel, the user will pay SMI again to get a new license.
Thirdly, SMI, who becomes the publisher of the work, recognizes the creative input from contributors and rewards each of them by equally splitting the performance royalties: “[we] allow the user to give the piece of music a name that is relevant to the project, we register that piece of music with the PRO’s and we make sure that it’s tracked; any royalties that are collected go to the creators of the parts, including a share to the user.”
Finally, unlike US publishing deals, SMI gives their musicians 100% of the writer’s share, plus a 50/50 split of the sync license fee, a popular European model. Kiely emphasized the importance of keeping musicians and composers in the revenue lane.
Based on research, SMI believes that for every 1,000 stems created in their library, SMI can create up to 50,000 unique licenses. Keil says that if a composer made available a piano piece for one of their so-called fantasy templates, they could expand that piano part to all the other templates in their database adapting the harmonic structure of that part “[while] keeping the emotional performance of the composer intact ”.
New copyrights would also be generated for each reproduction of the original stem, and the same publishing deal would be given to the composer as though every reproduction was his/her original work. SMI would provide too a back portal for creators, allowing them to upload and track all of their content and enabling them to see which stems are being used and more importantly which ones are being licensed.
SMI is planning a soft launch in Europe in January and fuller launch in the US next summer. It is still dependent on a successful investment round. Nevertheless, the company epitomizes a new paradigm for music making, where algorithms dominate, random functions rule, and music is ever more disembodied in its more complex forms from the human touch.
It is odd that a composer like Kiely would surrender the aesthetics of music production to a machine. But video rules supreme these days and the synching of music in this automated way may be expeditious when sound is used as a complement to a larger production. Moreover, some form of artificial intelligence in music is seen already among services that cater to even dedicated listeners: the individualization of playlists by Spotify, a great reason for its success, is mostly machine driven.
By Esteban Roa
