Reactive Audio in the Fighting Genre
“A good fight should be like a small play, but played seriously.” – Bruce Lee
Similar to the state of martial arts before Bruce Lee’s Jeet Kune Do style, the state of the fighting game genre continues to trudge forth with a flawed mentality. Developers are doing the player a disservice by creating fighting games that lack the martial artist understanding. A wise martial artist will tell you an effective technique is an emotional technique. Lee described it as “emotional content” in Enter The Dragon. It cannot be explained in writing, but only as an observed action with an appropriate reaction. The problem with fighting games, however, is just that — they are games. How does the visceral emotion of a punch translate from avatar to player? The answer is at once simple and complex — reactivity.
Dynamic Range
“Empty your mind, be formless, shapeless – like water. Now you put water into a cup, it becomes the cup, you put water into a bottle, it becomes the bottle, you put it in a teapot, it becomes the teapot. Now water can flow or it can crash. Be water, my friend.” – Bruce Lee
I contacted Rev. Dr. Brad Meyer, Audio Director of Sucker Punch Productions, to discuss what could be done to improve fighting game audio, to which he said “the most important [aspect] is dynamic mixing.” An effective dynamic range is like water filling the volume of a glass. In this case, our capacity for sound represents the glass and the dynamic range represents the water. When you look at a full glass of water you still see the water beneath the surface, and when you listen to dynamic audio you still hear the sound below the threshold. “Be it fighting games or any other genre, they all revolve around crafting a more dynamic experience,” said Meyer. “There’s a lot more we can do here, not necessarily to make games more cinematic, but to make them more emotionally effective. Composers from Beethoven to Bobby McFerrin have demonstrated the near-universal character of music as an emotion engine and arguably every composer of the past several hundred years have helped to strengthen this common language.”
Other genres have the luxury of relying on a written narrative for emotional weight, but the narrative (the only narrative that matters anyway) of fighting games is ludological. The punches and kicks tell the story and the music provides emotional reinforcement. “I would put dynamic music right after this, in part because music is the heart of the emotion of sound,” continued Meyer. “We can do a lot with other audio assets from ambience to character sounds to help convey mood and tone, but music is a known entity where emotion is concerned.” This isn’t to say the music of fighting games is bad, it just hasn’t been fully-realized. In fact, and excuse the pun, music plays second fiddle at best in the design of fighters.
Speaking with composer and music producer Tom Salta (Halo Anniversary, From Dust) I was told “When it comes to choosing music styles, most audio directors go with the obvious choice: Heavy fight equals Heavy fight music.” It’s quite ordinary and the results more than often reflect that. Tom suggests manipulating the emotions of the player by “juxtaposing music that you might not expect.”
“It can be risky and it certainly takes more experience and experimentation, but the results can be extraordinary,” said Salta. “Dynamic range can certainly play a role in this. It’s important not to stay too long on one dynamic level, otherwise people will just tune it out. If you want to keep people excited and engaged, then keep changing things.”
Composition and Implementation
“The highest technique is to have no technique. My technique is a result of your technique; my movement is a result of your movement.” – Bruce Lee
A reactive sound design in the fighting genre should not be assertive, but adjustive. That is, adjustive to the action and reaction. According to Tom, the music should still seek to define its characters. “My first instinct would be to come up with specific instruments and sound design that pertains to the different characters,” he said. “Then compose multilayered music that contains these various elements and fade in and out those different parts based on what’s happening in the game.”
“The trickier, albeit more fun part is to compose music that matches the scheme and can seamlessly blend from one state to another,” said Meyer. “Due to the frenetic nature of fighting games, the transitions would need to be quick, yet still feel polished and musically sound.” That is, if approaching composition on the granular level, i.e. individual notes.
The problem with this, according to Ben Abraham’s 2008 interview with Marty O’Donnell, is that granular samples on such a small scale lose their live musical feel, as well as their fidelity. While granular samples can be employed to enhance a specific event, the affair cannot be completely granular. The main function of the composition should instead seek to react to the entire balance of the fight, more so than the individual actions of the fight. The more rapid moments will find their own rhythm and musicality in the sound of fighting itself.
In his thesis, An Investigation of New Musical Potential in Videogames, Ben Abraham discussed how ambient and diegetic sounds create an undertone of constant musical current. “Potentially, these sounds can be purposely more musical than they would ordinarily be,” he said. In particular, the musicality of fighting can serve as instrumentation for synchronization and immersion. Current fighters do employ decent sound effects such as the tapping of footsteps, the thud of pounding flesh, the clang of swords colliding and the crunch of bones breaking. But as with any great piece of orchestration the composition should highlight the right “instruments” at the right time.
“Besides the obvious ‘cross-fade’ method where you can switch between multiple cues, another alternative is multilayered music,” said Salta. “There can be a common bed and depending on what is happening, you can bring in and out elements that are linked to what’s happening.” In order to enhance the brutal, clinical feel of the act of fighting, the music should know when to bow out and give the sound effects their time to shine. Ideally, this type of reactive audio system enhances the narrative of a fight by emotionally augmenting the player-controlled flow of combat.
“On the implementation side, the task is not terribly difficult on a high level,” said Meyer. “Passing the player distance from his opponent can drive an intensity parameter to dictate switches in music or adding/subtracting parts of the mix. Additional layers of complexity can be added by factoring in other parameters as well such as each players’ health, player’s current combo, or even the number of enemies targeting the player if it’s more than 1 vs. 1. The reactivity and switching can easily be prototyped and implemented using Wwise, FMOD, or scripting.” Of course, there would be some technical limitations when dealing with different platforms. “You probably couldn’t stream an 8 channel audio cue on iOS which selectively plays various tracks at different volumes to match your design,” said Meyer. “At the same time, midi with custom instruments a la the DS can create powerful and sonically rich experiences for low overhead.”
Possibilities
“I find it surprising that, with the wealth of varied music games available now from shooters to puzzlers that no one has done a music fighting game yet,” said Meyer. There are numerous possibilities for the fighting genre using a reactive audio system, such as music influencing the developer-intended story of single player mode. To this end, the developer-controlled narrative could employ a musical structure that matches the intended consequences of the battle. The player must live up to to implied triumphs and failures established by universal musical tropes in order to progress through the story. A reactive audio system built into the game engine can also allow for much needed innovation in the fighting genre, such as a “Sound Versus” mode. This can allow two (or more) players to fight toward building their respective songs up to full orchestration by maintaining the battle’s momentum for long enough. A player’s momentum, and thus song strength, is broken when the other player gains the upper hand.
“The challenges in a versus music mode are a bit more daunting,” commented Meyer. “How does the music work? Does each player have his own cue that fights for audio space in a volume war tied to health (perhaps with a third piece to crossfade in during non-action times)? Or do you go for a Peter and the Wolf type scenario where each player has his/her own representative instrument or instrument section which plays the melody and whoever is winning has more of their instrument play? You could even have two melodies (as long as they work musically with each other) so the winning player’s melody gains dominance and during struggles both play, ideally harmonizing each other and augmenting the tension.”
Filed Under: Blog
