Hand tracking
by Timo Haavisto, project lead developer at TUAS
MarISOT project user interface controls are operated solely with Varjo Technologies VR headset’s state of the art hand- and finger tracking features. The user interface elements are shown inside the virtual environment, and all panels and menus are triggered by virtually touching the elements with fingers. The menus and panels have sense of depth, meaning the operator’s finger proximity is taken into consideration: the user experience can be described as a touchscreen.
Inside the scenarios, the hand-tracking controls are integrated into the ship command bridge operations and panels. The controlling buttons have proximity events, which trigger when user’s fingers reach out to the buttons. A highlighting of buttons will ensure that user is aware of the operations he has triggered using the control panel. Extra care has been taken to arrange the buttons in such a way, that hand- and finger-tracking blind spots do not adversely affect controlling the panels.
Radar- and ECDIS screen panning has gesture operation: the radar and ECDIS screens can be pinched with index-finger and thumb, to grab the screen view. After grabbing the screen, the radar and ECDIS can be panned by moving hand to appropriate direction – the radar and ECDIS will span accordingly.
The MarISOT ship’s thrust control is a lever, which again is operated solely by hand- and finger tracking. To operate the thruster, user grabs the virtual lever by clenching their hand to a fist – as they would also in real lever operation. After grabbing the lever, it can be tilted forwards and backwards as wished. The lever locks in place after user releases their grip.
Tiller control is a cog, with a virtual projection for operation. Using the projected lever, the tiller control can be set accurately and augmented with clear information of the tiller angle. Elevating the control into projection allows user to keep outright posture and stance in the virtual command bridge.
MarISOT command bridge has two independent sets of Radar and ECDIS, which can be operated individually and with different settings. All displays are updated in real time, following the realistic physics engine of the ship.
In upcoming Posidonia 2022 exhibition next week, we will finally reveal these achievements. The first experiences of hand tracking in command bridge scenario was introduced in: https://allaboutshipping.co.uk/2020/07/31/hand-recognition-and-finger-tracking-technology-integrated-in-the-tuas-marsevr-maritime-safety-education-with-vr-technology-command-bridge-safety-vr-technology/
In the Posidonia 2022 exhibition, we will introduce our MarISOT application with all new features such as collecting eye and hand tracking data.
For further information on MarISOT please contact Prof. Dr. Evangelos Markopoulos (evangelos.markopoulos@turkuamk.fi) or Prof. Dr. Mika Luimula (Mika.Luimula@turkuamk.fi)
