Hello everyone! I got a huge update for you! :D
Awwdio is now our in-house audio system, replacing the previous hybrid system with one that's fully our own! Not only does this finish the last major piece needed to be moved fully into FrooxEngine for The Splittening to happen, but it comes with another cool feature - Spatial Variables! A double combo of coolness!
I'll be on a lookout for any issues that pop up and there's few small features I plan to add post-release, but generally... Awwdio is done! Meaning that focus will soon be shifting towards the splittening, so stay tuned!
The system will automatically convert any existing content to the new system, with some caveats - notably for Reverb Zones - check the notes below for details. If you run into any issues, DO NOT SAVE the content, because that will bake the issues in. We'll be on lookout for problems post release too and we can fixup conversions as long as you have original versions of worlds and objects.
Spatial variables themselves are a huge feature on their own - while relatively simple in principle, they have tons of usecases, from spatially interactive systems, games, culling systems, effect zones and many more! You can watch video on them here if you're interested: https://www.youtube.com/watch?v=83cVUv8on5g
Huge thanks to everyone who helped test Awwdio and polish it up for the public release: @anquietas_sys, @bluecyro, @zyro1331, @modernballoonie, @vrbat, @shininghero, @stiefeljackal, @.fulgens, @tobs2007, @foxobread, @dustysprinkles, @ohzee, @bredo, @martysh12, @nytra, @qualium, @gawdl3y, @freyar, @marsmaantje, @989onan, @enziv, @catboy.slim, @epiceaston197, @zenuru, brecert, SylviaAnim8or, @goldenrecord, @grand__, @euphieeuphoria, @badhaloninja, @shdw_x, @lucasro7, @orange3134, @jackthefoxotter, @unskilledwolf, @elektrospy, @art0007i, @yoshiyoshiyoshiyoshiyoshyoshyosh, @karasutengu, @mpmxyz, @pjb, @rabbuttz, @zozokasu and anyone else if I forgot to include you! (poke me and I'll add you to the list).
New Features:
Awwdio - new audio engine
Introducing a fully custom in-house audio engine - Awwdio! Replacing the previous hybrid system (which was about 80 % our custom audio handling and 20 % Unity's system), this is the last major piece needed for the "splittening" - the final phase of the big performance update with switch to .NET 9. It also gives us full control over how audio works in Resonite, fixing a number of issues and opening door for many new faetures in the future.
- Implemented & integrated Awwdio - our in-house audio system (requsted by me, issue #2430)
-- The system is heavily multi-threaded for high performance, with lock-free concurrent mixing
-- Spatialization is provided by Steam Audio with direct integration
-- Default number of audio voices is 64 and can be increased in settings (be careful with very large values as they impact performance and might cause popping)
-- The default latency is configured to be same as Unity - 1024 buffer for simulation, with 4096 playback buffer
-- You can lower the latency in Settings. However note that lowering the latency can cause more crackles and RunAfterFinalPoseUpdate
- Added AudioListener component which provides an explicit way to determine where audio is rendered from
-- It can also be used as a source for audio data for AudioOutput, working as a virtual microphone (This only works on listeners that aren't bound to specific user - ActiveUser must be null)
-- The listener has a list of effects that are applied to the audio it hears (currently just reverb zones, but there will be more in the future)
-- By default this list is managed by CloningReferenceSpatialVariableCollector, but this is not necessary - the list can be updated statically too (right now we don't provide mechanism to replace the default template yet)
- Implemented ZitaReverb to replace legacy ReverbZones
-- ZitaReverb has been integrated by @bluecyro
-- This is a different implementation so it does sound different, which is expected
-- Legacy reverb zones are mapped to a preset that's closest to the legacy one (mapping has been implemented by @bluecyro to match the vibe of old presets as much as possible)
-- This is a stereo reverb effect
-- Reverb zones now work with spatialized audio sources - including user voices in normal mode!
- Implemented AudioFilterBlendWrapper, which wraps other filters around and provides blending functionality
-- This is useful for audio reverb zones, allowing each listener with its own blend weight to share the same instance of AudioZitaReverb (and potentially other filters in the future)
- Added "Preserve Legacy Reverb Zone Handling" setting (under Music -> Legacy Feature Settings") (based on request by @bluecyro, issue #4231)
-- By default this is off - meaning by default all spatialized audio outputs will now be affects by reverb zones (unless they explicitly checked "IgnoreReverbZones" setting)
-- While this is a breaking change, it generally should be a positive one, making existing content and avatars more immersive
-- By turning this setting on, the legacy behavior will be preserved exactly for any subsequently loaded content - you can re-save the content loaded this way to bake the behavior in
- Added Pitch to AudioOutput, which allows altering the pitch of the output without affecting its length
-- Note: Altering pitch by altering Speed of the clip player is still preferred method - it's more performant with less artifacts (and better handling for extreme values)
-- Unlike Speed, this works on any source - even realtime sources like user voices
-- This uses realtime pitch shift effect, which will have some artifacts - especially the larger the pitch shift is
-- Pitch shifting algorithm integrated by @bluecyro
- AudioOutput now exposes SpatializationStartDistance and SpatializationTransitionRange
-- This controls where the spatialization starts and blends from stereo audio to spatialized
-- It's useful if you want to make the audio go "broadcast" when you get near the source
- Implemented auto-audio ducking, that automatically lowers the gain when the signal exceeds 1.0 and then slowly(ish) recovers
-- This is applied at three levels - individual audio output renders for each listener, listener mixes and final output mix (e.g. combining multiple listeners)
-- This fixes clipping caused by the binaural effect, which adds gain to the signal (reported by @.fulgens, issue #4127)
-- This was also requested by @.fulgens, @Zyro1331 in issue #4117 to happen globally
- Added support for 7.1 channel audio buffers
-- This fixes audio not working with 8 channel headsets (based on report by @dustysprinkles, issue #4079)
Spatial Variables
Coming alongside Awwdio is the spatial variable system, which was added to handle the reverb zones with the new system! This provides a new and easy way to sample values at various points in space and build variety of different systems.
- Implemented Spatial Variables (issue #574, requested by @frooxius (me :3))
-- This system allows to define values in space and then sample points in space to determine value at that point
-- Sampling spatial variables is very efficient, using BepuV2 acceleration structure for spatial queries
-- There's a number of spatial variable shapes and samplers with different behaviors for various use-cases and more will be added in the future!
-- Spatial variables are identified by a combination of their type and name
-- Name can contain characters, digits, spaces and following symbols: `- _ .`. The rest are reserved for future use. Spaces cannot be at the beginning & end.
-- Each spatial variable shape supports blend distance - blend weight is calculated based on the distance from the "outer shell" of the shape
-- Blend weight is used for proportional blending and priority computation
- Added BoxConstant(Value/Reference)SpatialVariable
-- The value/reference is the same throughout the whole volume
- Added BoxGradientValueSpatialVariable
-- The value changes from Start to End value along a Direction
-- This works best when the direction is aligned with one of the axes
- Added BoxVertexValueSpatialVariable
-- Each corner of the box can have a unique value
-- The value is interpolated based on the relative position within the box
- Added Texture3D_SpatialVariable
-- This is specifically for colorX spatial variables - you're reponsible for mapping these values to other things
-- This is a box spatial variable that provides colorX spatial variables sampled from a Texture3D (this must be readable)
-- You can adjust the scale/offset of the texture in local space
-- You can either use absolute coordinates or normalized ones (normalized ones will squish/stretch with the box)
-- You can control how the texture wraps in each of the directions
- Added SphereConstant(Value/Reference)SpatialVariable
-- The value/reference is the same throughout the whole volume
- Added SphereStartEndValueSpatialVariable
-- The value interpolates from the center to the outer edge of the SphereConstant
- Added (Value/Reference/Type)SpatialVariableDriver
-- This will drive a field with value sampled at the current global position of the slot it's on
-- The value used is one with highest priority (highest explicit priority, then highest blend and then highest ReferenceID)
-- You can define a default value/target
- Added NumericValueSpatialVariableDriver
-- This works specifically for numeric values and allows for multiple sampling modes:
--- HighestPriority - this simply gives a single value that has highest priority (same as the simpler variants)
--- WeightedAverage - all found values at particular point are averaged, with the blend weight used to weigh the values. If total blend weight is less than 1, it's blended with the default value
--- PrioritySortedBlend - the final value is series of Lerps from the default value and then lowest priority to the highest priority values, with the blend weight of each being the lerp factor
--- Additive - the final value is sum of all the found values multiplied by their blend weight
- Added BooleanSpatialVariableDriver
-- This is specifically for boolean values, which combines all the found bool spatial values at given point
-- Supported combine modes are:
--- Any - result will be true if there's at least one true value
--- All - result will be true if all found boolean values are true
--- XOR - each found boolean value (sorted by priority) flips the value. The initial value is the default value.
- Added MinMaxValueSpatialVariableDriver
-- This is for numeric types specifically
-- It drives two fields (it's not necessary to have both), one with the Minimum found value at given point and one with Maximum
-- If there's no spatial variables, Min/Max will be the Max/Min possible values for that datatype
- Added ReferenceSpatialVariableCollector
-- This is specifically for reference types
-- This drives a list of references, filling it with all the found spatial variables
- Added CloningReferenceSpatialVariableCollector
-- This works similarly to ReferenceSpatialVariableCollectoro, but specifically for components
-- For each component it finds at the point, it will create & maintain a duplicate
-- It can duplicate either just the component itself or the whole Slot it's on
-- The duplicates can be made Local only (it's your responsibility to make sure that the target list can reference local elements)
-- It will never clone spatial variables present on the slot
- Added SpatialVariableBlendDriver
-- This combines with CloningReferenceSpatialVariableCollector
-- If a field on a spatial variable component is driven by this component, the cloned version will be driven with the current blend weight for that clone
-- This is used for the BlendWeight for the new reverb zones
- Added ProtoFlux nodes to sample spatial variables (requested by @jackthefoxotter, issue #4215 and @unskilledwolf, issue #4216)
-- Sample(Value/Object)SpatialVariable - works with any time, samples the highest priority one at given point
-- SampleNumericSpatialVariable - works with numeric types, supports various way to combine them (same as component)
-- SampleMinMaxSpatialVariable - works with numeric types - finds the Min & Max value at certain point and whether any was found at all
-- SampleBooleanSpatialVariable - samples boolean variables with multiple modes to combine them (same as the component)
Other stuff
- Added SetActive actions to the Skybox and AmbientLightSH2 components (requested by @aegis_wolf and @ryuvi, issue #4218, implemented by @ryuvi)
-- This allows you to set an instance of either component as the "active" one via ProtoFlux, button action triggers, etc.
- Added IsActive bools to the Skybox and AmbientLightSH2 components (requested by @aegis_wolf and @ryuvi, issue #4218, implemented by @ryuvi)
-- This is a read-only field on each component which is true if that instance of the component is the currently active one.
- Added "RunAfterFinalPoseUpdate" to PositionAtUser to run a second update after user's final pose for the frame has been computed
-- This only works when the target user is local user (either through TargetUser or PositionAtLocalUser)
-- It runs a second update outside of normal update cycle specifically after the user's final pose has been computed to ensure the slot is positioned to the latest value
-- This fixes listener root falling behind when moving fast (reported by @ohzee, @bluecyro, @dustysprinkles, @epiceaston197, brecert and others, issue #4251)
- Added ReferenceList component
Tweak:
- IgnoreReverbZones is now IgnoreAudioEffects on AudioOutput
-- It functions largerly the same, but it will skip all effects on a listener
-- In the future we'll expand the system
- ViewPosition & ViewRotation nodes now actually position based on the actual target user (e.g. on PositionAtUser), rather than the local user
Locale:
- Merged Japanese locale update by @.aesc
- Merged German locale update by @muppeq
- Merged Korean locale update by @mirpasec
- Merged Mongolian locale update by modimobeikete
Fixes:
- Reworked resampling mechanism for CircularAudioBuffer to fix incorrect math
-- This fixes audio sizzling artifacts when using non-48 kHz audio devices (reported by @martysh12, issue #4072)
- Fixed exceptions in LegacyAudioPlayer when some pieces are missing
- Fixed references between multiple components not transferring properly when a group of components is cloned together to a different slot
- Fixed clipping audio samples not limiting the negative amplitude (reported by @Nytra, issue #4208)
- Fixed Enum shifting skipping enum members which have variants that have been marked obsolete (based on report by @Zyro1331, @shdw_x, issue #4206)
- Fixed MultiValueTextFormatDriver showing obsolete names of enums used as arguments (based on report by @Zyro1331, @shdw_x, issue #4206)
- Improved robustness of disposing of systems when disposing worlds (e.g. when they are closed) which prevented subsystems from being disposed when there are errors disposing other systems
-- This fixes audio system not getting disposed in case of errors with other systems, causing audio to continue playing (reported by @yoshiyoshiyoshiyoshiyoshyoshyosh, issue #4249)
- Fixed inconsistent generic type validation, resulting in some systems not validating some generic types
-- This fixes being able to instantiate or load certain generic types (based on report by @Tobs2007, @Zozokasu and @UnskilledWolf, issue #4282)