Fix some extremely dumb issues affecting video depth prediction, caused by python being garbage, I swear I didn't even touch the code.
Fix some extremely dumb issues affecting video depth prediction, caused by python being garbage, I swear I didn't even touch the code.
-(Hopefully) improved perspective correction, especially for the new high depth model. -Fix high video model inverted depth -Fix 3D export not working -Most things should use case sensitive paths now which i guess matters if you're loading images from a NAS or w/e -Probably some
-(Hopefully) improved perspective correction, especially for the new high depth model.
-Fix high video model inverted depth
-Fix 3D export not working
-Most things should use case sensitive paths now which i guess matters if you're loading images from a NAS or w/e
-Probably some other boring stuff I forgot about
Well, this took way too long. I fixed the perspective stuff mostly a few months ago but didn't want to push the update until i could confirm that it looks better in VR but the stupid quest was busted so I gave up and then got really busy with other things and just kept putting it off I guess. Woops.
Anyway whatever. I thinkkk it should be better now and mostly fix the whole 'tunnel' effect thing.
I think before I was using some hacked together exponential algorithm that worked well for shallow depth ranges of older models but if the new one outputs a high range depth the old perspective method just kind of couldn't cope at that scale and even if you tried the background would expand way too much and break everything.
Now I've added a more physically correct model and some automatic adjustments so that it should stay relatively stable when moving the image away or scaling it because there was definitely an issue with that before as well.
You may need to adjust your saved perspective values but I don't think they should change again.
Might want to back up your img data before running if you care about them, just in case.
-Updated the built-in 'High' model to DepthAnything3 Mono Large ~I finally had some time to mess with this again and somehow managed to figure out how to get the DA3 models baked into onnx while still supporting both variable resolution input and fp16 ~At least for now only the l
-Updated the built-in 'High' model to DepthAnything3 Mono Large
~I finally had some time to mess with this again and somehow managed to figure out how to get the DA3 models baked into onnx while still supporting both variable resolution input and fp16
~At least for now only the large model is used as for some reason on small and base models for DA3 the quality is really bad? In my testing anyway, maybe because they're not specifically mono versions idk.
~This should be a straight upgrade over DA2 though as I've observed much better overall relative scaling and details. Faces on DA2 looked pretty weird off angle, but now they look pretty decent still I think.
~I made this the default model when using 'auto' and you have a cuda capable GPU, for anyone else it can be forced on by selecting high in the advanced settings. The performance is probably still fine for images.
-Image foreground/background clip now saves with per-image settings
-Added a hand size slider in the visualization setting of the toolbox
-Added a new model setting "Manual" which disables the depth generation for images
~If you really like tinkering, this can perhaps be used to set up an external tool to handle generating depth for images.
~The depth gen window will still be opened to handle generation for videos as that is a much more tightly integrated process.
~Whatever custom solution just needs to read the files paced in steamapps/common/AutoDepth Image Viewer/Midas3/input and put whatever output into the corresponding output folder
~pfm and 16bit grayscale png are supported
I wrapped the transform stage into the onnx model to avoid cpu stuff I guess, along with some other minor improvements this should slightly increase performance on non nvidia GPU's
I wrapped the transform stage into the onnx model to avoid cpu stuff I guess, along with some other minor improvements this should slightly increase performance on non nvidia GPU's
-added an option to disable changing the image while screen capture is active -added ability to load custom backgrounds -added ability to crop the screen capture boundary -fixed vr controller bindings for video seek left/right and play/pause not working (oops)
-added an option to disable changing the image while screen capture is active
-added ability to load custom backgrounds
-added ability to crop the screen capture boundary
-fixed vr controller bindings for video seek left/right and play/pause not working (oops)
-Updated depth models and depth generation backend Hopefully this doesn't break things even more for most people. But this should give proper cuda support for 50 series GPUs, as well as less buggy support for AMD and others. If it doesn't just explode anyway. Also the way depth r
-Updated depth models and depth generation backend
Hopefully this doesn't break things even more for most people.
But this should give proper cuda support for 50 series GPUs, as well as less buggy support for AMD and others. If it doesn't just explode anyway.
Also the way depth resolution works has changed now so that the long edge of the image matches the set depth resolution. Before, the short edge would be used and the long edge would be scaled up proportional to the aspect ratio which could lead to really high resolution and vram usage for images with a really wide aspect ratio.
Anyway all that to say you will have to increase the depth resolution to achieve the same quality as before.
Also, since models have changed (multiple times by now) and steam doesn't really delete the old files, you may delete all of the non .onnx models in the app folder under midas3/weights, if you want.
Assuming all of this goes well, I should be able to do another round of QoL soon.
-Updated some backend stuff on the depth generator -Better GPU capability detection, should fix issues with certain cards/drivers -Hopefully grabbing the image with a VR controller should be more generous now and harder to get stuck etc. -Added keyboard shortcuts for image manipu
-Updated some backend stuff on the depth generator
-Better GPU capability detection, should fix issues with certain cards/drivers
-Hopefully grabbing the image with a VR controller should be more generous now and harder to get stuck etc.
-Added keyboard shortcuts for image manipulation
--W,E,R controls position, rotation, scale when held
--Holding shift modifies the axis the same way right clicking the buttons does
-Added depth resolution and show window settings to video depth preprocessor -Video thumbnails in the image viewer should now work regardless of if ffmpeg is installed
-Added depth resolution and show window settings to video depth preprocessor
-Video thumbnails in the image viewer should now work regardless of if ffmpeg is installed
-Fix issue causing images to be shown at a lower resolution -Also improved loading times for anything that isn't a rotated jpeg i guess
-Fix issue causing images to be shown at a lower resolution
-Also improved loading times for anything that isn't a rotated jpeg i guess
Added hue/saturation/brightness controls to the visualization tab in the toolbox Added support for image alpha cutout in png/webp Added a new tab to the toolbox with options to duplicate image displays so you can have more than one at a time Added new hotkeys and optional control
-Fixed VR pointer line not rendering in front of the image -Fixed VR cursor dot disappearing -Probably fixed an issue with image orientation when opening/closing the menu -Fixed issues with certain resolutions when screen capturing -Changed the screen capture resolution limit to
-Fixed VR pointer line not rendering in front of the image
-Fixed VR cursor dot disappearing
-Probably fixed an issue with image orientation when opening/closing the menu
-Fixed issues with certain resolutions when screen capturing
-Changed the screen capture resolution limit to be a percentage instead of a fixed limit
-Fixed bug that would break screen capture when a program entered exclusive fullscreen
-Added option for screen capture to disable depth while the mouse is moving
-Added option to change the screen capture cursor size
-Added a new feature to the toolbox to cast a screen/monitor and depthify it. Woooo More difficult than I thought to implement this. Individual parts aren't that hard but getting it all to come together seamlessly is tricky. I think I lied when I said it would run the same as the
-Added a new feature to the toolbox to cast a screen/monitor and depthify it.
Woooo
More difficult than I thought to implement this. Individual parts aren't that hard but getting it all to come together seamlessly is tricky.
I think I lied when I said it would run the same as the realtime video stuff.
It's just inherently more taxing to capture the screen, but more than that, frames can't really be buffered and batched together because you cannot simply look into the future to see what the next frame will be like you can with a video.
I might revisit batching as a setting at the cost of extra latency but I just wanna get this feature out for the sale going on right now.
I think there may be various performance pitfalls but eh, I get about 30 depth frames a second on my kinda older mid range system, so I think it's well enough for now.
Part of the problem is just that without batching, the effect of turning down the depth model/resolution isn't really noticeable past a certain point because the performance bottleneck becomes one of just moving the data onto/off the GPU and scheduling less work, more often.
Anyway, enough techno-rambling.
Enjoy the screen casting, and let me know if you have any ideas to make it nicer to use.
I think the holy grail would be being able to capture specific windows as well and be able to virtually send clicks so you dont have to tab between things, but that's wayyy too much work just to avoid some extra clicks.
Though it may be possible to implement the virtual clicking part at least which may be useful for VR/second monitor situations.
Not sure if that would increase the chances of antivirus programs flagging it falsely as malware.
I'm already hoping that implementing screen capture libraries doesn't do that 😅
Also I will soon look into some other requested features now that this is done.
Just trying to figure some things out. In other news, I'm working on a feature to stream the desktop. This should let you view whatever content on your computer without needing to specifically open it within AutoDepth. Pretty neat huh. It should run about the same as the current
Just trying to figure some things out.
In other news, I'm working on a feature to stream the desktop. This should let you view whatever content on your computer without needing to specifically open it within AutoDepth.
Pretty neat huh. It should run about the same as the current realtime video depth feature, so If you can run that it should be fine.
Feel free to let me know if you think this is cool 😎.
-Added a new tab to the toolbox with a bunch of neat visualization options. -Show raw depth map -Show surface normals -Disable edge clipping to extend image edges infinitely, can create a pretty cool effect -Controls for foreground/background clipping to isolate a section of the
-Added a new tab to the toolbox with a bunch of neat visualization options.
-Show raw depth map
-Show surface normals
-Disable edge clipping to extend image edges infinitely, can create a pretty cool effect
-Controls for foreground/background clipping to isolate a section of the image
-I reworked how data is handled for realtime video -Should be more stable now and transfer bottlenecks have been virtually eliminated, so you should be able to render video depth as fast as your system can handle -Resolution and model has a much more drastic impact now, and lower
-I reworked how data is handled for realtime video
-Should be more stable now and transfer bottlenecks have been virtually eliminated, so you should be able to render video depth as fast as your system can handle
-Resolution and model has a much more drastic impact now, and lowering them can allow to reach high speeeeeds
-Fixed a bug that caused everything to break when switching realtime video model/resolution while it was playing. (why did nobody report this??)
-Added support for new "ML Depth Pro" model in the advanced settings. -Currently this model is locked to a 1536x1536 resolution, and is fairly detailed, though perhaps a bit slow. Hopefully someone will figure out how to unlock the resolution on it later so it can run faster. -Th
-Added support for new "ML Depth Pro" model in the advanced settings.
-Currently this model is locked to a 1536x1536 resolution, and is fairly detailed, though perhaps a bit slow. Hopefully someone will figure out how to unlock the resolution on it later so it can run faster.
-The pro model also performs worse for animated styles from my testing.
-Added a slider in advanced settings to control number of preloaded images.
-Added an optionally bindable button to the VR controller config for favoriting an image. (can be assigned from the steamvr controller settings menu.)
-Improvements to image processing speed which should have general improvements and especially noticeable in realtime video. Though there's still a bottleneck there I think that is holding it back a lot and I will continue to try and fix that, but I wanted to push out this update with support for the pro model so, here we are.
-Increased the maximum thumbnail limit in the browser to 2000.
-Added a slider to control the resolution for depth estimation under advanced settings. Should allow beefier PCs to crank out a bit more detail at the cost of estimation speed. Most people shouldn't really bother with this but if you feel inclined to tinker with it and test estim
-Added a slider to control the resolution for depth estimation under advanced settings.
Should allow beefier PCs to crank out a bit more detail at the cost of estimation speed.
Most people shouldn't really bother with this but if you feel inclined to tinker with it and test estimation speeds at certain resolutions and on different models, it's there for you.
-Changed the way the model selector works slightly so that resolution is no longer factored into the model selection but rather taken from the aforementioned slider.
This also comes with being able to use the base Depth Anything V2 model instead of just the small and large. Does require a download if you don't already have it though.
-Added a setting for the realtime video stuff that lets you set a default frame skip value to hopefully mostly avoid waiting for each video to just skip at least once.
-Made sure the 'void' background is actually properly completely black oops.
Got some QoL here boys changelog: -Pointer in VR now freezes when pulling the trigger to make clicking easier -Added a separate toggle for allowing GIFs, off by default -Browser now prioritizes loading thumbnails for visible content first -Thumbnails for videos are now taken from
Got some QoL here boys
changelog:
-Pointer in VR now freezes when pulling the trigger to make clicking easier
-Added a separate toggle for allowing GIFs, off by default
-Browser now prioritizes loading thumbnails for visible content first
-Thumbnails for videos are now taken from partway through the video to avoid black frames and such from intros
-Wobble settings are now more detailed
-Using the analog stick on a VR controller to change images should be less likely to adjust tool scale
-Improved seek behavior for realtime video depth, should stay in sync now when seeking
-Added an option to use the headset tilt when recentering to enable reclined viewing
-Recentering is now delayed by a few seconds when clicking the recenter button
-Added an option to show show a prompt on startup asking if you want the session to be sandboxed
--This loads a clean browser with no browsing history, playlists, favorites, etc
--Activity while sandboxed will not be saved
-Fixed scroll list bounds sometimes not working
-Fixed some flicker when changing images
Still working on some stuff that should be good hopefully.
Thinking about adding a control for foreground/background clipping 🤔
Not sure if it's worth it as while neat it didn't seem to add much in my tests and I don't wanna add even more controls to mess with and make it more fiddly than it already can be.
-Changed default model to the new Depth Anything V2 small model and added support for downloading other V2 models These models definitely have more detail in them and run about the same as the previous ones. I think the consistency is better as well for realtime video and such.
-Changed default model to the new Depth Anything V2 small model and added support for downloading other V2 models
These models definitely have more detail in them and run about the same as the previous ones.
I think the consistency is better as well for realtime video and such.
-fix UI bugs causing duplicate folder appearances and persistent drive letter folders -fix playlist and favorites going outside of UI if list is too big -improved sorting by name when filenames have numbers in them (a bit slower to sort but eh) -add reset all button for image con
-fix UI bugs causing duplicate folder appearances and persistent drive letter folders
-fix playlist and favorites going outside of UI if list is too big
-improved sorting by name when filenames have numbers in them (a bit slower to sort but eh)
-add reset all button for image controls
-add setting for alternative way to handle default image settings when using individual image settings
-add height offset setting for VR
-something fun