HereSphere update v0.10.0 for PC is now available on the HereSphere Steam default branch. This update adds support for passthrough by using Virtual Desktop's passthrough feature for Quest and Pico headsets. Native PCVR passthrough still isn't supported yet, but will be added with the UE5/OpenXR update.
Passthrough with Virtual Desktop:
- In the Virtual Desktop settings, go to "Streaming", make sure the VR frame rate is set to 90fps or less. Under the "VR Passthrough" settings, enable the checkbox and then click on "Configure". Set the settings to Red: 0, Green: 255, Blue: 0, Similarity: 45, Smoothness: 0.
- You can switch to passthrough mode on a global basis (for all videos) by going to the user settings, clicking on the background dropdown box under the environment group, and then select passthrough. If you want to only enable passthrough for the current video, go to the advanced video setting when the video is playing, and then change the background dropdown box under the environment group to passthrough.
- By default, the player will color the background green in order for Virtual Desktop to remove the solid green color and replace it with the passthrough feed from the Quest or Pico headset. You can change the default color or disable this option by going to the user settings, and under the environment group, switch the background to "Passthrough", and change the color settings or disable the "Replace Passthrough With Color" toggle.
Chroma key masking:
- Added chroma key masking, which removes parts of an image based on the color (useful for removing green screens for passthrough). To enable chroma key masking, go to the advanced video settings, and under the environment group, change the mask dropdown box to chroma key. Press the cogwheel icon next to the dropdown box to edit the chroma key mask settings. You can define up to three different chroma key colors, a light despill color, and a light injection color. Click on the eyedropper icon to select a color by choosing a location in the image. Pixels that are similar in color to the chroma keys will be made transparent. You can define a cutoff value, soften value, exponent, and spill mask for each chroma key to specify the range of affected pixels. Pixels that are close to the light despill color will have their color adjusted to remove the tint of the despill color. This helps with removing green screen halos and skin tinted by the green screen. After the light despill operation, the color may look unnatural, so the light injection can be used to add color back into the image to make it more natural. For more details, view the tooltips by holding the grip button when hovered over the settings.
Alpha packed masking:
- Added alpha packed masking, which defines the alpha (opacity) of the image by packing it into another region of the image. To enable alpha packed masking, go to the advanced video settings, and under the environment group, change the mask dropdown box to alpha packed. Press the cogwheel icon next to the dropdown box to edit the alpha packed mask settings. The alpha coordinates shift x and shift y settings define the position of the alpha image relative to the normal colored image. The alpha coordinates scale x and scale y settings define the size of the alpha image relative to the normal colored image. For more details, view the tooltips by holding the grip button when hovered over the settings.
HereSphere Web API:
- Added "alphaPackedSettings" and "chromaKeySettings" field, which can be used to load settings for alpha/chroma key masks. Check the HereSphere discord for more details on the HereSphere web API.
- Added "hspArray" field, which contains the url of hsp file and the version of the hsp file. Newer hsp files aren't backwards compatible with older HereSphere installations, so the version number is used to determine which hsp file can be read.
Quality of life improvements:
- Added support for local cmscripts used by Connect & Control.
- AB Loop times are now saved per video
- Added "Toggle AB Loop" keybinding
- Searching for tag ratings without a tag name will now match with all tags rather than just tags that have a blank name. For example, searching for ">3" will match with all tags that have a rating greater than 3.
Bug fixes:
- Fixed crash that occurs when pressing the seek keybinding without a video loaded.