Idea:
In order to make the world of WHALIEN more lively and immersive, we decided to make our music diegetic, meaning that the audio sources are actually located in the world. We accomplished this by placing loudspeakers all over our levels. So, the players can determine what direction the music is coming from and how far the speaker is away.
We first tested this idea in one of our elevator levels, where we wanted the elevator to play its smooth elevator tunes by itself. The player should hear the music fade in as they approached the lift. We liked this concept of localized music so much that we decided to implement it throughout the entire game.
(The current model of the LevelMusicPlayer is a gramophone)
Implementation:
Conveniently, UE4 already provides a lot of audio functionality so that we were able to implement the entire system without the need for any middleware or the likes. At the core of our LevelMusicSystem, we have the LevelMusicPlayer that plays music in a conical shape in front of it.
The more intricate task was synchronizing all LevelMusicPlayers, since it was very off-putting if the tracks were running even a little bit asynchronously. We had to make sure that the LevelMusicPlayers started their playback at the same time when they were loaded in. This was simple when LevelMusicPlayers were placed in the same level, because we could simply start playback in the BeginPlay() function. If, however, the LevelMusicPlayers were located in different levels, we couldn’t guarantee that the BeginPlay() functions were called simultaneously, so we had to come up with another solution.
In order to save processing power, we also decided to stop playback whenever the player left the LevelMusicPlayer’s range and restart it once they re-entered the range. Therefore, we needed to know at which timestamp to restart the LevelMusicPlayer on re-entering.
We solved these problems by creating the LevelMusicPlayerManager, a static class that keeps track of all music tracks and at what time they were started. Whenever a LevelMusicPlayer wants to play its track, it first needs to ask the LevelMusicPlayerManager when this track was started first to calculate the timestamp at which it should start playback.
The music tracks should loop indefinitely, and they need to be restarted at the correct playback time when the track has already looped before. In order to determine whether a track has looped, we need to know its duration. We did a bit of research and eventually found a way to retrieve the duration of the .wav files in a USoundCue, which we could use to determine if our track had already looped. Below you can see a code sample of how we retrieve a SoundCue’s duration.
(Code sample for retrieving a USoundCue's duration)
In order to ensure that our LevelMusicPlayers could not be overshadowed by an abundance of other sound effects in the area, we gave them the highest priority. We then changed the SoundConcurrencySettings in UE4 to cull music and SFX solely based on priority, whenever too many audio sources are playing simultaneously.
(Left: Project setting to define the SoundConcurrency-class; Top-Right: SoundConcurrency setting to cull music based on priority; Bottom-Right: Set priority in our LevelMusicPlayer's AudioComponent)
Usage:
Besides providing a nice feeling of presence, we can use our LevelAudioSystem to set auditory highlights in our levels. If we want the player to check out a particular area, we can make the music come from that direction only. The music then slowly fades in as the player approaches the designated area. We can also start and stop individual LevelMusicPlayers at will in order to dynamically change the highlighted area.
The feature also provides an automatic way of fading between music tracks when the player is walking from one LevelMusicPlayer playing track A to another playing track B. Simply by moving away from A and towards B, the built-in attenuation system in UE4 creates a smooth transition between the two tracks.
As another small detail, we made the LevelMusicPlayer react to the players’ abilities. When they influence it with Mr Push - the pearl that repulses everything in its vicinity - we apply a low-pass filter to remove all the high frequencies. When the player throws Mrs Pull - the pearl that attracts everything in its vicinity - we apply a high-pass filter to remove all the low frequencies.
Future:
We still want to create a prettier model as well as an animation for our speakers. Ideally, the speakers would bounce up and down with the rhythm and emit neat particle effects.
We created a small demonstration video for you to hear the results of our new LevelMusicPlayers:
As always, if you have any questions about our LevelMusicSystem, we are happy to answer them!
- Andi, Ralf