Betting on Smartphone cameras to make hand tracking accessible to all, Manomotion launched their SDK for developers. It works on most camera equipped devices, including PC/MAC webcams and Gear VR, but also works with depth cameras. It can be used for VR, AR and regular apps. Many interesting features is offered.
The SDK allows developers not only to substitute hands with a virtual representation like a gun, a plane or another hand, it is also possible to easily let the app show the actual hands over the content. It can be pretty useful in apps or games requiring the users to input a lot of different gestures.
So far, the SDK support one hand to be tracked, but supports both left or right hand. It’s unlike Leap Motion that tracks both, but requires their proprietary external camera. Manomotion camera use also allows for AR applications to be made. They claim to have 27 Degrees of Freedom of motion in a hand., and offers 15 predefined gestures ready to use. Functions such as Grab/Release, Swipes, Point and Pinch are a few of them. It can detect and position and orientation, estimate depth, detect dynamic gestures allowing for clicks and drag actions.
The SDK support Native Android and iOS. Unity for Windows and Android, including the GearVR, is also supported. I’m mostly excited about the depth Estimation feature. If it works well, it could really make a difference in VR mobile inputs. From what I’ve seen, the Paint Demo is the one I find the most impressive. It’s worth watching until the end.
I haven’t had the chance to try any of these demos by myself, or see the SDK, but the technology seems promising. I definitely want to add touchless input on any smart phone. Using the front facing camera, you could easily leave your phone lying and face up and launch apps and browse your favorite apps.
Their website is hosting 6 Videos, showing different demos and tracking examples. ManoMunch is a game where the player control a fish that moves by the tip of the finger. You must eat other fish to grow bigger. The ManoPet demo shows the ability to play a 3D Platformer by pointing directions and make gestures. I’m not convinced yet of it replacing a controller for those kind of games yet.
All I can say is Good Luck to the guys and girls behind Manomotion. The promising technique will definitely easy up development and open up minds on hand tracking use in the different format it offers. Still the limitation of hand tracking is the inability to track it outside of the camera’s vision.