There seems to have been some confusion in the initial days of Soundscape about why the program does not “sync to music.” At first thought it seems like a no brainer for a program like this, why wouldn’t you have it? There are several reasons why I developed the program the without it and I want to make this post to clarify my design choices and outline the future of this project.
1) In the old "Soundscape – demo edition" nothing synced to music, it was all programmed manually per song to match up to the blueprint of the song based on BPM and instrument changes throughout the track. In the new version I let the player control the BPM of the visuals as well as all the changes in color, pattern, rotation, on/off, etc. My belief is that you yourself can create a much more accurate and immersive experience than any beat matching algorithm can.
2) If you have ever played a "music reactive" VR program (like Audioshield for example), you will know that that it is not really all it is cracked up to be and 75% of the time it doesn't even work right. It's just changing so much that it is bound to land on the beat some of the time. These programs may advertise to “sync to music” but the computer algorithm is an inferior substitute for the human mind which can not only keep rhythm exceptionally well, but also knows what is coming next in the song and when different instruments are coming in and dropping out.
3) By putting the power of the synchronization in your hands, I allow you to use your music from any source or music service. Take a look at other “music reactive” games, in every single one you need to play music their way which limits you on the songs you can play and how easily you can access them. This program is about freedom to play your music any way, even via voice control or externally.
4) Lastly, I use the UE4 engine to develop, and UE4 simply does not allow for the possibility of "sync to music" in packaged (finished) products. UE4 is currently working on rebuilding their audio engine, so when that feature is available to make use of, I will definitely be exploring how it can be integrated in a way that allows that does not disrupt the previous listed design goals.
My vision for this program is to make it a multiplayer experience for friends where they can listen to music together in a VR space across the globe and easily play the role of "lighting guy" creating perfectly synced lighting patterns for one another, playing around in this sandbox space with particle effects. I created the program this way because I believe it is much more fun to be the one creating and fine-tuning the experience than just watching it happen. The touch controllers and vive wands allow a unique opportunity to interact with your music in way you never could before and I'm trying to make the most of all that power at your fingertips.
