In May we announced the new SteamVR Input system to allow developers to more easily support existing and future controllers, enable gamers to more easily customize their controls, and empower hardware makers to continue to innovate on VR input. But there’s another aspect to controller input unique to VR that we did not address at the time: extra information from the controller about what the player’s body is doing. Current hardware can tell us a lot about where the user’s hand is and the pose of their fingers, and that information is only going to get more accurate and detailed as the industry continues to innovate. To facilitate this innovation, we’re proud to announce the release of the first beta of SteamVR Skeletal Input.
The Skeletal Input system will allow controller makers to capture the pose of the user’s hand as accurately as their sensors are able, and provide it as a stream of animation data to the running app. App developers can then use this stream to animate the hands of the user’s avatar on its own or in combination with their own animations.
What this means for users is that they will be able to get the full benefit of their investment in new VR controllers in their favorite games without the need for the developers to release a patch to support the new controller. App developers will not need to create unique animations for each controller they intend to support when their game is released, or issue patches to support new controllers. And hardware makers can continue to innovate in this space, and write their drivers so that their controller will work with any game that supports the new Skeletal Input system.
Controller drivers will be able to provide two animation streams to apps for each controller: one stream intended to track the estimated pose of the user’s hand as accurately as possible, usually meaning with the controller in-hand. The second stream is intended to provide a range of motion for the hand as if it was no controller in the way, ideal for cases where the user’s avatar is not currently holding anything in-game. The app developer can then choose which of these streams to sample based on the current context of the game.
SteamVR Skeletal Input will also provide the option to convert either data stream to additive animation. Additive animation can be useful to app developers because it can be layered on top of other animation to provide additional, more dynamic motion. For example, you could use the additive finger animation from SteamVR Skeletal Input as a layer on top of an artist-created pose of the hand holding an object, to allow users to appear to adjust their grip or gesture while holding an object.
We also wanted to ensure that multiuser experiences would be able to take advantage of this new system, but networking an entire animated hand skeleton can require too much bandwidth for many users. So we have included as part of the API support for compressing the animation down to a size that is more practical for networking.
This initial beta release is intended as an opportunity for developers to try out the API in their games and provide us with feedback. Vive Wands, Oculus Touch, and the new Knuckles EV2 are already supported, and we invite other input makers to work with us to add support for their controllers during the beta as well.
A full SDK for all developers will be released soon, but for now Unity developers can access the Skeletal Input API with this early release of the SteamVR Unity plugin. The documentation is available on the OpenVR GitHub Wiki. We look forward to hearing your feedback!
HTC Vive Wands
Oculus Touch Controllers
Knuckles EV2
