Audio Engineering Society Singapore Section

On Friday, 29 January 1999 at 7.00pm, 13 members and 24 guests of the AES Singapore Section gathered at Kent Ridge Digital Labs (KRDL) to listen to a talk on "Interactivity: Audio in the New Millenium" by Dr Pete Kellock. A senior research staff at KRDL, and also Member of the AES, Dr Kellock is a pioneer in the field of interactive sequencing. He helped introduce multimedia to Singapore by founding and managing the first local series of corporate short courses in the field, and has been co-leading the development of the FlexEffex system at KRDL, a global first in the area of interactive sound effects and music, for the last three years. 

The meeting started with an opening address by Mr Jibby Jacob, the Chairman of the AES Singapore Section. He noted that there was an increasing number in the audience who were first timers at an AES Singapore Section activity. After the introduction, Dr Kellock began his talk by giving an introduction to Interactive Audio. He explained that I-Audio is any Music, Sound Effects(SFX), or speech that, playing from a computer, would respond to a user's action (or multiple users' actions) in real time. Although many musicians and audio producers are already working with audio tools which are interactive, the main promise of I-audio is that it will increasingly allow the "audio consumer" to become the participant. I-audio has many applications including computer games, virtual environments, infotainment/edutainment (CD-ROMs, multimedia-kiosks, presentations, etc.), web sites, interactive movies, interactive music systems, music education, and many more.

Dr Kellock questioned the view that I-Audio is, and will remain, "a niche". He presented some interesting facts about the existing state of the art (for example that the computer games industry is already bigger than the television industry on some measures) and floated some questions that may help us imagine the future for this emerging field: Does music always fit your mood from moment to moment? Would you play a role in a movie if you could? How many people fancy themselves as musicians? Do humans ever resist the temptation to get more control of their environments? The realisation of these would be dreams come true, and represent a revolution in audio. I-Audio will certainly be the hot technology in time to come. 
 

Dr Kellock next explained the technology of I-Audio, comparing it to the traditional Linear Audio systems of today. He explained that Linear Audio systems (for example, recorded audio on a CD) are very rigid and allow little interaction - little more than the select_Track/Play/Stop controls on a CD player. I-Audio gives the user much more intimate control, for example by allowing much smaller building blocks to be combined in real time allowing numerous "paths" through the audio experience. Of course, such interactivity requires lots of intelligence in I-Audio systems compared to the relatively dumb nature of traditional Linear Audio systems: much of the composition, performance and production, traditionally done by people in studios needs to be done "on the fly" in the computer system, (which, by the way drives a need to move the entire studio into the PC.) 

Speaking on the challenges that I-Audio faces, Dr Kellock cited three universal challenges. Firstly, there is presently a trade-off between flexibility and realism. Highly flexible systems, such as, for example a physical model of a flute (which computes the resultant sounds according to given parameters) generally fall short of delivering the realism of the actual flute sound. Conversely, a CD track or even technologies such as wavetable synthesis, provide relatively realistic sounds, but lack flexibility. The challenge is to make the sound both real and flexible.

The second universal challenge is the human interface. A person from the professional audio industry will be familiar with operating sophisticated audio equipment, but these complicated technicalities will baffle the everyday "audio consumer". Thus I-Audio tools must provide "handles" which the average person can understand - a model of footsteps might provide "walking speed" and "shoe type", not "filter resonance". 

The last challenge, but not at all the least, is the problem of morphing a sound from Sound A to Sound B. This process must generate a smooth and natural transition in which one sound becomes the other, not a cross-fade from one into the other. This can be useful in applications such as games where the sound morphs pleasingly when moving from one location to another, or in web applications when the user navigates
between web pages. Because there are numerous paths possible in a morph, producing a good morphing algorithm really requires not only technical but also audio and/or musical expertise. 

Of the I-Audio technologies available today, 3D Sound, or Spatial Audio as Dr Kellock prefers to term it, has been enjoying much attention within the community. He noted that 3D Sound is not necessarily interactive, but is often associated with interactivity because it is such an important addition to the visual elements of 3D virtual environments. 

Dr Kellock then gave an overview of the companies, organisations and musicians that are presently involved in the advance of I-Audio technology. He said that sound card companies like Creative Technology, who dominates the sound card market, are constantly bringing pro-audio and music features into the PC. This is essential in his opinion, as I-Audio requires some or all of the sound production to be done in real-time on the computer. Other thrust areas of I-Audio include Microsoft's and Intel's efforts in recent years to enable software synthesis on the PC CPU. Microsoft's DirectSound programming interface, which provides a common sound service for applications such as games, and also DirectMusic, incorporating technologies like DLS and IMA, nudges I-Audio a step closer. Dr Kellock went on to present two demos of Microsoft's DirectMusic, simple programs in which the music changes to suit the game play interactively.

Other industry players Dr Kellock cited are Headspace, Staccato, and Mixman. In the demo of a Mixman product, he showed how a piece of dance music could be created almost instantaneously. According to Dr Kellock, it is becoming possible, to an increasing degree, for people without any formal musical training to compose music. He also mentioned several professional musicians known for their work incorporating I-Audio, of the more popular are Peter Gabriel (Xplora), Todd Rundgren (No World Order), The Residents (Gingerbread Man), and his personal favourite, Laurie Anderson (Puppet  Motel). Professional organisations involved include MMA-IASIG (I-Audio Special Interest Group of the MIDI Manufacturer's Association), whose members include Creative, Roland, VLSI and many others. 
 

The mood grew to be more exciting as Dr Kellock talked about his group's work in I-Audio at KRDL. Using demos, he illustrated their main work, the FlexEffex sound effects and music system. Based on early work by Dr Lonce Wyse, joint leader of the KRDL team, the FlexEffex system generates sound effects using algorithmic techniques, producing convincingly realistic sounds. Classes of sound effects are modeled and a high level of interactivity is provided for each model through parameters that can be controlled - for example by virtual sliders. In a footsteps model, the user can control the speed of the walk/run, the surface type walked on, the weight of the person walking, the styles of the walk (long strides, marching, etc.). No previous program has been able to create "on the fly" footsteps on wood, then on concrete, then a combination of concrete and mud, and so on under real-time control. 

Furthermore, their algorithms automatically introduce random variations into the footsteps, very effectively increasing the realism of the generated sound. Dr Kellock went on to demo, to the fascination of the audience, a guns model, a wind model, and a bell model. In the guns model, the user had a wide variety of controls such as the gun type, variable speed of fire, controllable pitch of the shot, and the amount of reverberation following the shot. Each of the models provided the ability to create  extremely flexible sounds, simply by changing parameters which would be easily understood by anyone, not just audio experts. Dr Kellock said that the team at KRDL is presently continuing to work on this system, increasing the number of models in their library. 

In his prediction of future trends, Dr Kellock said that while traditional Linear Audio had a strong future, the growth rate of I-Audio would be faster, with more and more titles produced in both versions. Over the next few years, he thinks that I-Audio experiences will gradually approach the realism of movie experiences today. Furthermore, with the rapidly decreasing cost of PCs, Dr Kellock reiterated that the studio would gradually disappear inside the PC, a crucial enabling factor for I-Audio. 

Another trend will be the blurring of the roles of producer and the consumer -the consumer will to some degree create his own produtions drawing on material produced by others, in much the same way some professional musicians uses drum loop libraries. In conclusion, Dr Kellock assured the audience that most pro-audio skills will be in demand for a long time to come, but the project goals may be different with I-Audio requiring audio that includes behaviour. To achieve this, there must be good communication between the pro-audio industry and the audio software programmers. 

In the question and answer session, Dr Kellock told the audience that the team at KRDL had started this project from scratch, and up to today had written computer code totalling more than 10MB. Answering a question on the commercialisation of the FlexEffex system, he mentioned that marketing work has already begun, with the team at KRDL considering commercialisation through either a startup or by licensing the technology. He said that because of the wide variety of applications this technology has, FlexEffex has generated much interest when it was shown to software houses, film companies, and others worldwide. 

Following a few more very interesting demos of morphing drum rhythms, the linking of FlexEffex from Cakewalk, and a virtual environment game, the meeting ended at about 9:45pm. 

The AES Singapore Section would like to thank Dr Pete Kellock and his colleagues at KRDL for an eye-opening talk, the use of the KRDL auditorium and their kind assistance during the meeting.

Copyright 1999 AES Singapore Section