You can now fade, swell, and sustain notes with the new “bow” and “prox” playing tools. The bow tool behaves a bit like a violin bow, making sounds as it passes (at various speeds) through a ringer. The prox tool uses its distance from a ringer to control the sound’s volume. Both new tools support an alternative mode by squeezing the trigger. These new tools immediately expand the available EXA playing styles beyond the percussive approach of the “strike” tool.
Behind the scenes, EXA has a new audio system to support these new tools and improve audio performance. This system bypasses Unity audio, talking directly to the OS instead, which fixes the “popping/cracking” audio issue. The structure of this new system should also make future features (like MIDI support and audio effects) easier to introduce.
In the loop display, the old “triangle” shapes are now replaced with a “waveform” style display. Each displayed note now represents the volume and duration of the performed note, including support for the new sustained notes (with variable volumes and fades) performed with the bow and prox tools.
Thank you to the EXA community for the continuous feedback and ideas! Please watch/share the new DevUp video and read through the full details below.
DevUp VIDEO
https://www.youtube.com/watch?v=Q6GdqZcZVAY
(https://www.youtube.com/watch?v=Q6GdqZcZVAY)
TOOLS
- The “bow” tool behaves similar to a violin bow, where the ringer gets louder as the bow passes through it faster. To generate sound, the bow tool must pass through the plane of the flat ringers (rectangle, triangle, ellipse), or move across the axis of the line ringers. By default, the bow tool will use motion in any direction to increase a ringer’s volume. Squeeze the trigger to make the bow tool only use motion that is parallel to the bow’s direction/axis.
- The “prox” tool uses the distance to a ringer, where the ringer gets louder as the tool moves closer. Beyond the outer distance threshold, the tool has no effect on the ringer. By default, the prox tool affects all ringers within its distance threshold. Squeeze the trigger to make the prox tool only affect the single ringer that is nearest to the tool.
- The bow and prox tools have several common traits. For these tools, the ringer sounds keep their original “attack” audio and fade-in times, but the envelope’s “sustain” volume is forced to 100%. The ringer audio is automatically normalized so that the looping/sustaining part of the audio is closer to 100% volume. With this approach, the player’s tool usage/speed can control a broader range of possible volumes. Both tools generate haptic feedback, with intensity that varies with the playing strength/speed.
- The bow and prox tools do not force non-looping audio samples to loop/sustain. The ability to loop/sustain for any particular note is built into the individual Soundfonts. For example, using a prox tool on the default piano or guitar sounds will allow the note to sustain forever, since those audio samples have a loopable sections. However, using a prox tool on the default cymbal sounds will not sustain; the sound’s volume still reacts to the tool, but the non-looping audio sample will play to its stopping point even if the tool stays near to it.
- Upon playing a ringer, the trails that follow the tool now glow using more information, including the volume and duration of the ringer’s sound. The trail colors now blend together as each new ringer is played (rather than only using the color from the latest ringer).
- The mute tools now show a small gray sphere in their center. This sphere functions as the position of the menu cursor, and also allows the mute tools to hit the interface-based ringers (for metronome, loops, and sections).
- The “length extender” lines on the tools (on the longer play tools, and others) are now a darker gray.
AUDIO SYSTEM
- A new approach to EXA’s audio system was required in order to support the new “bow” and “prox” playing tools, and to enable more advanced Soundfont effects (like oscillator-based vibrato and tremolo). The system is more flexible for future EXA features, including audio effects, MIDI support, etc.
- The new audio system improves audio performance, reduces latency, and fixes the occasional audio “popping” that would occur in previous versions.
- Regarding audio popping: this seems to be a common issue for Unity when using its “Best Latency” mode (256 samples per buffer). Low latency is vital for EXA, however, so EXA now bypasses Unity audio. Audio-processing performance itself was not the issue (since popping could occur immediately upon startup, playing a single ringer, etc.). The root cause was most likely the attempt to perform realtime audio processing in a “managed” environment (Unity and .NET), where things like automatic garbage collection can interfere. The new audio system runs in a “native” environment, which avoids those things.
- The audio system has become more centralized and is driven by various well-defined ringer events. This has been implemented with an eye toward MIDI output. Due to the potential complexity of EXA performances, MIDI output will likely map ringer events to their own MIDI channel (continuously cycling between the 16 available channels).
- Added a “Max Voices” slider to the “Application > Audio” menu. This controls the maximum number of samples that EXA can play simultaneously. Internally, EXA has a priority-based system to stop the quietest audio when the “Max Voices” threshold is reached. This setting defaults to 32, and allows values between 16 and 48. If audio performance or quality seems to degrade, especially during complex and highly-layered loop playback, consider decreasing this setting. The exact performance and audio capabilities will vary based on each player’s machine.
- Removed the “Buffer Size” slider from the “Application > Audio” menu. The native audio system uses the equivalent of the previous “256 samples” setting (lowest latency). Future versions may permit changes to sample sizes of 128 or even 64.
LOOPS
- Updated the loop recordings to capture, playback, save, and load all the new event types. This new data structure is different than previous versions, but EXA can still load/play loops from older saved layout files.
- Updated the loop panel to display notes in a “waveform” style. Note displays gradually fade out with the “strike” tool, can swell larger and smaller with the “bow” and “prox” tools, and can get smaller (or immediately cut-off) with the “mute” tool.
- The note displays consider the duration of a note’s audio and/or its “release” time, which is useful for distinguishing shorter sounds (like drums) via the loop display. The loop display updates whenever the player modifies a loop’s sound, pitch, etc.
- Added support for recording a sustained note (created via bow/prox tool) that was started before the loop recording started (and is still playing at that time). The recorded note behaves as if it started at the loop’s first beat, fading into the target volume.
- Added support for releasing all sustained notes (created via bow/prox tool) upon stopping a loop’s playback, or when it cycles back to the first beat, or when the loop is deleted.
- Improved the quantization of “per-frame” events, which can be generated by the mute, bow, and prox tools. Rather than quantizing/clustering these events to the nearest beat, they shift in relation to their corresponding start event. If quantization moves a start event 0.1 beats to the left, all its corresponding “per-frame” events also shift 0.1 beats to the left. Note that the “mute hit” events (which are different from the “mute proximity” events) are still quantized the normal way.
RINGERS
- The tool’s angle of approach now makes a slight difference in the sound. More severe angles can create a bit of a “twang” sound characteristic (technically, it controls the “Q” value of the sound’s low-pass filter). Initially, the distance from the ringer’s center created a “duller” sound (reduced the higher frequencies), but I have disabled this for now. Eventually, the mapping of hit positions/angles to audio effects may be configurable.
- Ringers that use invalid sound+note combinations (or use a missing/unavailable sound) are now unplayable and have a partially transparent appearance. An invalid sound+note combination can occur when a Soundfont only supports certain notes – this is common with “drum kit” Soundfonts.
- The ringer glow and flex effects now utilize both the ringer’s audio output and the hit/play strength. The flex effect now behaves more consistently at different ringer sizes, notably with the amount of “sideways wobble” that occurs when the ringer is hit/played at steep angles.
FIXES
- Included the loop and section “volume” settings when saving/loading layouts.
- Fixed issue causing Steam to keep showing “running” status after closing EXA, caused by a failure to exit all internal threads upon app exit.
- Updated the Vive control to allow the thumb to move away from "up" position (while touchpad remains pressed) without triggering the teleport action. Note that the Vive touchpad leaves its “pressed” state if the touch position is lost, so sliding the thumb position too far to the top of the touchpad (which cause touch position to be lost) will still cause the teleport action to occur.
- Fixed issue that allowed the teleport position to change (based on controller motion) for a couple frames after releasing the teleport button, which could lead to out-of-bounds teleports. Also, the teleport boundary line is now always visualized (even when teleport action is inactive) using a dark gray color.
- Fixed issue causing the VR rendering resolution to not respect the “VR Pixels” setting upon startup, which only occurred when launching the app via Steam.
- Fixed issue causing certain sounds, upon being played with the “strike” tool, to loop forever without fading out.
- Fixed issue causing the old/invisible ringers to still be playable for a moment after loading a new layout.
- Fixed issue causing the Hover UI Kit cursor positions (indicated by the gray ring around the tool) to lag one frame behind the tool position.
- Fixed issue causing the “Ringers > Notes > Intervals” menu’s Pentatonic button to generate the Chromatic “preview” sounds upon hover action.
- Removed some unnecessary memory consumption caused by the rebuilding of a few meshes and label strings.