During the Steam Autumn Sale, icosi-do is available at a discount of 40%! Also, don't forget to nominate icosi-do in the Labor-of-Love category of the Steam Awards 2023.
During the Steam Autumn Sale, icosi-do is available at a discount of 40%! Also, don't forget to nominate icosi-do in the Labor-of-Love category of the Steam Awards 2023.
As I've mentioned , I'm working on hand gesture controls for icosi-do with Tilt Five glasses, which includes quite a bit of testing. To some degree, I was surprised how good these controls feel (as long as they work as intended). That's not to say that my implementation works suf
As I've mentioned, I'm working on hand gesture controls for icosi-do with Tilt Five glasses, which includes quite a bit of testing. To some degree, I was surprised how good these controls feel (as long as they work as intended). That's not to say that my implementation works sufficiently well already, but I hope that it will once I can access the wide-angle head tracking camera of the Tilt Five glasses.
Now I'm wondering: what makes hand gesture controls feel better than hand-held controllers? Before answering that question, it's useful to consider what is required to make hand gesture controls feel at least as good as hand-held controllers.
Of course, the detection of specific hand gestures has to work sufficiently well. On top of that, many hand gestures will include some form of motion tracking, which has to be sufficiently accurate and fast to facilitate comfortable, interactive controls. While these requirements are not trivial, they are only the most basic ones in a longer list.
Long-term use of input devices such as keyboards, computer mice, touch pads, etc. is only comfortable if hands may rest on their palms while using these devices. Similarly, lower arms should be allowed to rest while using hand gesture controls. For Tilt Five, it might sometimes be possible to rest one's elbows on the same surface that the game board is placed on. Armrest pillows or adjustable armrests might be even more comfortable as demonstrated by the armrests of the surgeon's console of da Vinci Surgical Systems.
Another challenge for hand gesture controls is the lack of touch. Part of the problem is that touching is inherently pleasurable to most humans - even very young humans: some start thumb sucking even before birth. But there is more to it: holding a button down and keeping it in exactly that position is a lot easier than keeping a trigger or joystick in a specific position midway between its extreme positions. That's because we can hold a button down by applying a varying force as long as that force is always great enough to keep the button down; thus, there is no need for continuously correcting the applied force based on sensory feedback to keep its position, which means that the required cognitive load is smaller. One problem with mid-air hand gestures is that you are usually not able to push against other objects to reduce the required cognitive load. Thus, if a hand gesture requires users to keep their fingers in specific positions mid-air, holding that gesture can feel quite demanding and uncomfortable.
A possible design solution to this challenge might be two-fold: one part is to include touch by requiring two fingers to touch each other, e.g., as part of a OK or ring hand gesture. The other part is:
With "clutch", I mean a way for users to temporarily stop controlling a user interface. Lifting a computer mouse up from a surface or lifting a finger up from a touch surface or muting a microphone are clutches in this sense. Often these clutches only exist in software: a microphone "muted in software" might technically still record sounds, but that doesn't matter if the software discards all recordings.
There are multiple benefits and uses of clutches. One of them is that users may relax and don't have to worry that their actions may have unintended consequences. Thus, if hovering with a mouse (without pressing a mouse button) doesn't have any consequences (other than changing the position of a mouse pointer), then mouse buttons become a clutch, too.
As mentioned above, holding your fingers or hand in a specific, controlled position may feel quite demanding and uncomfortable, thus, it is useful to offer users a clutch that allows them to temporarily exit this requirement. A good candidate for such a clutch is the touching of two fingers, i.e., as long as a user's fingers don't touch each other, no actions are triggered - similarly to hovering with a mouse.
Assuming that all above issues have been solved: why would hand gesture controls feel better than hand-held controllers? A useful comparison might be to the relation between finger touch and stylus touch. Nowadays, finger touch is considerably more popular than stylus touch for a couple of reasons. For example:
In my opinion, both points are relevant for the comparison between hand gesture controls and the Tilt Five wand: finding the wand, putting batteries into it, and turning it on does not take long, but it adds to the friction of using the system.
While pointing with the wand is fine, the long stick at its front means that the user's hands are usually more than 15 cm away from the point of interaction. Compare this with finger touch or with hand-held VR controllers, which are sometimes required to be moved directly next to or even into virtual objects to interact with them.
While the comparison to finger touch and stylus touch is useful, in my opinion it doesn't fully explain how good hand gestures may feel. The missing element might be the magical feeling of using hand gestures to interact with objects at a distance without touching them. In fact, magicians often distract the attention of their audience with dramatic hand gestures that pretend to affect objects at a distance. Similarly, telekinetic abilities in films are often visually communicated with hand gestures, for example, the use of the "Force" in the Star Wars universe. The VR gaming industry has long recognised grabbing at a distance as a more convenient alternative to grabbing only those virtual objects that are very close to the virtual representations of hand-held controllers. One example of distance grabbing is provided by the "gravity gloves" in "Half-Life: Alyx" (which was predated by the "gravity gun" in "Half-Life 2").
Remote controls, laser pointers, and VR controllers have familiarised many people with the idea of interacting with objects at a distance. However, the possibility of interacting with distant objects by means of hand gestures without holding any technical device might cause a renewed feeling of performing magic when using hand gesture controls.
The feeling of magic might be an important benefit of hand gesture controls. But make no mistake: all the other issues mentioned above have to be solved first before one can hope to reap this prize. The good news is that reproducing the experience of grabbing or touching virtual objects is not necessary - and probably not even desirable.
This small update includes another small test of hand gesture controls for users of Tilt Five™ glasses. If you have access to Tilt Five™ glasses, please try it out with the full game or the demo. Some instructions (only for Tilt Five™ users): In the Tilt Five™ Control Panel under
This small update includes another small test of hand gesture controls for users of Tilt Five™ glasses. If you have access to Tilt Five™ glasses, please try it out with the full game or the demo.
Some instructions (only for Tilt Five™ users):
The limited field of view of the Tangible Camera is certainly a problem; in particular because the app only detects fingertips near the center of the camera image. Nonetheless, I'm quite enjoying this new way of solving icosi-do puzzles. Since the system has grown quite a bit, I would again appreciate feedback on this test from as many users of Tilt Five™ glasses as possible. Please leave a comment below this announcement, or in the discussion forum, or on the Tilt Five™ discord. Thank you!
PS: I just realised that the raised XE gameboard may confuse the hand gesture detection. For best results, please avoid the raised XE gameboard configuration and use the flat XE gameboard configuration instead.
UPDATE July 15, 2023: I'm making good progress on the next version; however, the two main pain points are still the lack of access to the wide-angle head tracking camera and the frequent crashes of the glasses' firmware. Both points appear to be outside my control. Since I don't want to test everyone's patience too much, I'll wait with the next version until there is some progress on either of these two issues.
UPDATE August 15, 2023: Sorry, no news: version 1.4 of the Tilt Five drivers and SDK are still in beta, and I've heard nothing new about access to the wide-angle head tracking camera. I'll just keep waiting.
icosi-do is participating in the Steam Summer Sale 2023 with a whopping 30% discount! And the demo is still free ! In other news, I continue working on hand gestures for selecting rods when using Tilt Five glasses. Currently, the pointing is quite jittery, but it's getting there.
icosi-do is participating in the Steam Summer Sale 2023 with a whopping 30% discount!
And the demo is still free!
In other news, I continue working on hand gestures for selecting rods when using Tilt Five glasses. Currently, the pointing is quite jittery, but it's getting there.
Thanks to everyone who has provided feedback about the first test of hand control for Tilt Five users in icosi-do ! (If you wonder: these discussions happened on the Tilt Five discord.) Here is a summary of my conclusions: 1) Performance matters: the approach of downsampling the
Thanks to everyone who has provided feedback about the first test of hand control for Tilt Five users in icosi-do! (If you wonder: these discussions happened on the Tilt Five discord.)
1) Performance matters: the approach of downsampling the camera image on the GPU and continuing the CPU computations with a 32 x 32 pixels image appears to provide a sufficient refresh rate for somewhat smooth interactivity.
2) The connected component analysis works quite well to detect whether thumb and another finger touch each other and thereby complete the OK/ring sign/gesture. The passive haptic feedback of this moment is a nice touch (pun intended). Of course there are limitations due to the single, infrared camera, but within these technical limitations, it appears to work very well.
3) The "head cursor", i.e. controlling a 3D reticle with head movements, doesn't feel very good. This becomes particularly obvious when comparing the feel of rotating the puzzle with hand control and the feel of selecting rods - the former feels a lot better. One part of the problem might be the required control loop between head movements and movements of the 3D reticle, which feels quite unnatural. Furthermore, the required head movements make the next point worse:
4) The limited field of view of the tangible camera is a real problem. The visualization of the camera image helps, but ideally the interaction design should encourage users to keep their hand in the center of the field of view such that they don't lose tracking. The field of view of the tracking camera would improve the situation, but I don't think that it would completely solve it. Also, I'm not sure whether or when images of the tracking camera are going to be available.
1) I'll focus on rod selection. Since the selection is confirmed by closing the ring formed by thumb and another finger, my current idea is to aim at (and highlight while "hovering over") a rod with an "open clamp" hand pose where the tips of thumb and another finger are somewhat close to each other and the center point between them is in line of sight of the selected rod. (The lack of depth information limits the feasible designs.)
2) To implement this design, I'm trying to find the positions of all fingertips in the image by computing the shortest-path distance of dark pixels from the image boundary and picking the pixels with locally maximal distance. I hope that I can easily identify the position of thumbs based on their small y-coordinate. The second fingertip might then just be the closest fingertip in the image. I suspect that positions on a 32 x 32 grid will not be accurate enough for a pleasant interaction, but that's a problem for another day (and I have an idea how to address it).
This small update only includes a small test for users of Tilt Five™ glasses to see how icosi-do might be controlled with hand gestures (as suggested in my previous post ). If you have access to Tilt Five™ glasses, please try it out with the full game or the demo. Some instructio
This small update only includes a small test for users of Tilt Five™ glasses to see how icosi-do might be controlled with hand gestures (as suggested in my previous post). If you have access to Tilt Five™ glasses, please try it out with the full game or the demo.
Some instructions (only for Tilt Five™ users):
All that might sound a bit complicated because it is! In fact, the small field of view of the camera can be quite frustrating. Moreover, I'm not sure about some of the parameters used in the detection of the OK/ring sign, specifically the minimum and maximum allowed size of the inside of the ring. Thus, I would really appreciate feedback on this test from as many users of Tilt Five™ glasses as possible. Please leave a comment below this announcement, or in the discussion forum, or on the Tilt Five™ discord. Thank you!
PS: Apparently, one can crash the app by activating gesture control with SETTINGS > AR GLASSES > OK SIGN > TEST and then performing a gesture right away. So, please, exit the menu before you start gesturing!
On June 5, 2023, Apple has presented the Apple Vision Pro and with it a new "spatial computing" interface based on (among other things) eye tracking and hand gestures. While the keynote presentation had its controversial moments, the reports by journalists who were able to experi
On June 5, 2023, Apple has presented the Apple Vision Pro and with it a new "spatial computing" interface based on (among other things) eye tracking and hand gestures. While the keynote presentation had its controversial moments, the reports by journalists who were able to experience early devices were quite enthusiastic.
So, this made me think: how could I integrate some of the features of this spatial computing interface into the Tilt Five version of icosi-do? To answer this question, let's step back a bit: At its core, eye tracking provides a direction in 3D space, which is similar to the tracked direction of Tilt Five glasses. The hand gestures potentially provide much more information, but the most basic information is just a single Bit: users are or aren't pinching with their hands, which is similar to pressing or not pressing a single button.
How to build a user interface based on a tracked direction and one button? It turns out that this is not only similar to using Google Cardboard but also to using icosi-do with the Tilt Five glasses but without the Tilt Five wand. And even better: I've already dealt with this situation and implemented a kind of reticle that can be controlled by moving the Tilt Five glasses! This covers the tracked direction.
What about the button? Instead of the trigger and buttons of the Tilt Five wand, the buttons of a gamepad controller, mouse, or keyboard may be used. To create an interface without such controllers, icosi-do could use an infrared camera of the Tilt Five glasses and detect a specific hand gesture. If the hand gesture is detected, the game could behave as if a button was pressed. Easy, right?
But which hand gesture is suited best? "Pinching" is not ideal because some users might "pinch" with their thumb and index finger forming a small angle (almost parallel fingers) as if they try to hold a large "pinch" of salt between thumb and index finger. This would be difficult to detect because of fingers overlapping in the camera image. A better hand gesture is the "OK sign", i.e., forming an "O" or "ring" with the thumb and index finger, and spreading the other fingers to form a "K". An alternative is the "ring gesture" (or "ring sign"), which doesn't require spreading the other fingers.
How to detect an OK/ring sign/gesture? I did a few experiments with the available infrared camera of the Tilt Five glasses and concluded that the gesture has to be performed in front of the retroreflective part of the gameboard, otherwise there probably isn't enough contrast between fingers and background to reliably detect hand gestures. Also, this isn't a big limitation because the gesture has to be visible to the camera anyway. The easiest way to detect an OK/ring sign/gesture in front of the retroreflective gameboard is probably to classify the pixels into "foreground" (high intensity of retroreflected infrared light) and "background" (low intensity of infrared light diffusely reflected by skin), run a connected-component analysis and search for a connected component ("blob") that represents the inside of the closed ring formed by thumb and index finger. My assumption is that this blob should be larger than the blobs corresponding to the dots on the boundary of the gameboard and smaller than the blobs corresponding to the rest of the visible retroreflective parts of the gameboard.
The interesting questions is then: how to run an efficient connected-component analysis on a relatively large image at more than 10 frames per second? My first attempt was to create a GPU-based pyramid algorithm for this. It worked in some cases, but wasn't very reliable. Also: I concluded that I had to analyse a low-resolution version of the camera image on the CPU in any case, which inspired another idea: a GPU-based downsampling algorithm that preserves the connected components of the image, followed by a connected-component analysis on the CPU.
So, that's what I'm working on right now. To be clear: this won't include any eye tracking nor will it allow to detect more than a single hand gesture. But at least this might be a step towards a controller-free interface when using Tilt Five glasses, which might be a lot of fun.
On YouTube, when you upload videos of icosi-do that include the built-in music, you might get hit by Content ID claims . That's because Bach's music is very popular and a couple of companies claimed copyrights for specific performances. Here is what you need to know: It is safe t
On YouTube, when you upload videos of icosi-do that include the built-in music, you might get hit by Content ID claims. That's because Bach's music is very popular and a couple of companies claimed copyrights for specific performances.
Here is what you need to know: It is safe to dispute any Content ID claim related to Bach's music in icosi-do, because there are no copyrights on the 300 years-old music by Bach and the performance in icosi-do is not copyrighted by third parties because I programmed that performance myself based on sheet music.
Personally, I got 4 Content ID claims on the music by 4 different companies, and successfully disputed all of them without problems. If you want to avoid the issue of Content ID claims altogether, you can turn off the background music under SETTINGS > AUDIO > MUSIC.
Please let me know if you have any questions or comments.
Another small patch polishing the interface for users of a single-button mouse: A "hint"/"undo" button has been added in the lower-left corner, which has the same function as the "X" key/button. In the tutorial, the "right arrow" button is now clickable in the hands-on parts of t
Another small patch polishing the interface for users of a single-button mouse:
Please let me know any further ideas to improve the interface for users of a single-button mouse, or other comments or feedback. Thanks!
Only a tiny patch polishing the user interface: The left/right/esc icons have been moved slightly towards the centre to improve the layout for some aspect ratios (and the main text is a bit smaller to make space for them). The areas to click the left/right/esc icons now move with
Only a tiny patch polishing the user interface:
As always, please let me know any comments!
A new regular update featuring better mouse support! I guess it was about time to improve the mouse interface such that icosi-do may be controlled with a single-button mouse. (My excuse is that it took me a while to realise that a carousel-like user interface fits the game menu v
A new regular update featuring better mouse support! I guess it was about time to improve the mouse interface such that icosi-do may be controlled with a single-button mouse. (My excuse is that it took me a while to realise that a carousel-like user interface fits the game menu very well.)
Other improvements in version V-10 (and in the demo):
As always, please let me know any bugs, comments, and feedback!
icosi-do is participating in the Steam Puzzle Fest with its biggest discount ever: 30% ! If that's not good enough: try the free demo! In other news: icosi-do is now officially part of the Tilt Five franchise ; check it out!
icosi-do is participating in the Steam Puzzle Fest with its biggest discount ever: 30%!
If that's not good enough: try the free demo!
In other news: icosi-do is now officially part of the Tilt Five franchise; check it out!
I found a YouTube video by @CertifiablyIngame covering the fictional game "kal-toh" (starting at 4:44 ). Very interesting, indeed! For more information about kal-toh, see its page on Memory Alpha .
I found a YouTube video by @CertifiablyIngame covering the fictional game "kal-toh" (starting at 4:44). Very interesting, indeed! For more information about kal-toh, see its page on Memory Alpha.
YouTuber Red Event Games plays up to level 5+ in the first game. Special thanks for going through the tutorial!
YouTuber Red Event Games plays up to level 5+ in the first game. Special thanks for going through the tutorial!
YouTuber Mariatxi plays through the tutorial and then up to level 5+ - with comments in Spanish. Thanks for playing on your channel!
YouTuber Mariatxi plays through the tutorial and then up to level 5+ - with comments in Spanish. Thanks for playing on your channel!
As announced yesterday, icosi-do has been officially included in Tilt Five's games catalog at . To celebrate the occasion, I've started a weeklong discount of 25%! Oh, and don't forget that you can play the first 7 levels with the free demo! With or without Tilt Five™ Glasses! Th
As announced yesterday, icosi-do has been officially included in Tilt Five's games catalog at https://www.tiltfive.com/games. To celebrate the occasion, I've started a weeklong discount of 25%!
Oh, and don't forget that you can play the first 7 levels with the free demo! With or without Tilt Five™ Glasses!
This might be a good opportunity to clarify the importance of Tilt Five for icosi-do: icosi-do would simply not exist without Tilt Five! While the actual game design was inspired by a fictional game from a popular TV series, my original motivation to develop and publish a game on Steam was to create something for the Tilt Five™ Glasses that I got as Kickstarter reward. You see, it's all connected.
And today is not only Valentine's Day but also the day that icosi-do became officially part of the evergrowing Tilt Five™ games library at ! Tilt Five 💞 icosi-do: a match made in Unity ❣️ Joking aside, there is even more good news to come soon! Stay tuned!
And today is not only Valentine's Day but also the day that icosi-do became officially part of the evergrowing Tilt Five™ games library at https://www.tiltfive.com/games!
Tilt Five 💞 icosi-do: a match made in Unity❣️
Joking aside, there is even more good news to come soon! Stay tuned!
YouTuber Maris from GamerQuest plays icosi-do up to level 5+, which is pretty good for a first game! Thanks for playing on your channel!
YouTuber Maris from GamerQuest plays icosi-do up to level 5+, which is pretty good for a first game! Thanks for playing on your channel!
Another day, another patch: I had a closer look at the volume levels of sounds and music, and decided to increase the internal volume levels quite substantially. You probably want to adjust your volume settings in SETTINGS > AUDIO > SOUNDS and MUSIC. Also, I cleaned up the
Another day, another patch:
As always, please let me know any comments!
This patch fixes three issues that came up when I was playing with COLORS set to off, i.e. in star mode and in x-bar mode. Hovering over a rod changes the color of the rod to highlight it. For gray rods, that highlighted color is now brighter, which shows more clearly which rod y
This patch fixes three issues that came up when I was playing with COLORS set to off, i.e. in star mode and in x-bar mode.
Unfortunately, I tried to update the app while Steam had world-wide connection problems. The demo updated fine, but it took about an hour to update the full game. Sorry for any inconvenience.