If you have looked around the web for information about the upcoming Daydream controller for Google’s newest VR platform, you probably stumble on the dozens of sites comparing it to Nintendo’s Wiimote. They are not wrong, but there’s a lot more to it as well. Here is all there is to know about the Daydream controller.
So there it is, the Daydream controller. It has many sensors and inputs: The orientation, gyroscope and accelerometer are used to set the position of the remote in the virtual space. The touchpad and buttons are the two physical inputs that are available
When looking at the Daydream controller the first thing that is apparent is the multiple input options. The touchpad is the biggest round section on top of the controller. It is the main input method apps and games will most likely use.
How it works:
The touchpad is a 2D input, it can gather touch coordinates with 2 values, an X coordinate and a Y coordinate both ranging from 0 to 1. This means that the touchpad can be used to detect precise touch location as well as movement like Swipe or Double TAP for example. Don’t mistake TAP with Click as there is a second input possible on it, when pushing harder, it will deliver a CLICK input to the app or program (and could also detect a double click if programmed right).
The touchpad concept is gaining in popularity in the last few years. Sony added one on its PS4 controller and the Steam Controller is also using 2 similar rounded Touch Pads.
Creativity is left with tons of possibilities. Mixing coordinates with CLICK actions can lead to multiple inputs coming from the Touchpad itself. The other usable button is called the APP button and is represented by a simple minus sign. This is a simple clickable input and nothing more. It can still be used anyway a developer want it to be used. The last button visible on the front of the remote is the Home button that is used by the system to bring up the menu. It is not yet clear if developers can use it to bring in their own app menu or home screen or if it exclusively brings the Daydream Home.
The remaining two buttons are located on the right of the controller and they are the Volume button. A nice addition to control volume of the apps without having to remove the device from the headset like you currently have to do with Cardboard and many of its clones.
The sensors are used to detect movement and position of the remote. 3 different sensors to monitor position and force applied to the different axes of our 3D world. Google called it 3 Degree of Freedom (DoF). It includes Orientation, Gyroscope and Accelerometer
The orientation sensor is most likely a magnetometer used as a compass. It will monitor the surrounding electromagnetic field (such as earth own magnetic field) to precisely return the orientation of the controller in 3D space. It is used to calculate the pitch, yaw and roll of the controller.
Pitch: When holding the remote in front of you, the pitch is when you tilt the controller up and down. It’s used to understand at what height you are pointing in the 3d world.
Yaw: This is the rotation from left to right; similar to what is done when you spin a bottle. It is required to understand in what direction you are pointing, north, south, east, west and any points in between.
Roll: This may be the rotation we are the most aware of. We do it every time we turn a key or pour coffee in our favorite mug. It is the angle that tells the controller what is UP and what is DOWN.
How it Works:
It is calculated with 4 values, called a quaternion, to understand the controller in a 3D space with X,Y and Z values, as well as a 4th one measuring the actual rotation. Quaternions are real complex values and are rarely used or modified manually. Instead, we let computers and sensors give us values to use them in our app to rotate objects or any other functions. Here’s an example on how it’s done.
Let’s say I’m pointing at the distance, I need a Vector of 3 values from my point 0 to determine that I’m pointing UP, and Slightly LEFT from point 0 and the 3rd value for the distance. If I twist my arm to put the controller upside down, like turning a key, I’m still pointing at the same coordinates, but the 4th value is used to measure that rotation. Here is a visual example of what this quaternion representation. The intersection of i,j and k is the controller position. The vector v is it’s Orientation, where it’s pointing. And that 120 degree mark is the rotation.
This point can be re-calibrated by holding the HOME button on the controller. When doing so, you probably want to be holding the controller right in front of you, pointing straight ahead. It is one of the most important sensors for a VR controller to work properly in a 360 degree 3D environment.
Unlike the orientation sensor, the gyroscope measures the speed of the rotations in term of rad/s. It only records the speed of the rotation on the pitch, yah and rotation angle. When not moving, it records 0 for all values maning it does not detect if the remote is upside down, only if it’s currently moving and at what speed. It is to say it records strength, not a position which is the orientation sensors job.
How it works:
If we place the remote on the floor and spin it like a bottle, it would record the spin. It will do the same if I spin it like a coin or kick flip it like a skateboard. The higher value for each axis represents a higher speed of rotation the developer can input in the app or game.
This sensor does have limitation and cannot measure over a certain speed of rotation so it’s often useless to go all force on movements and rather do quick and fluid movements instead.
The accelerometer is a dynamic value used to measure a force in a direction, another Vector. This time it has only 3 values. It calculates the speed in m/s for the 3 axis that represent our world. The 3 values are able to calculate the speed at which the controller is swiped left or right (x), up and down (y) and forward or backward (z). The data can then be used for direct feedback in the VR world or simply as an input method. A swipe up of at least 1m/s for example could be required to initiate an app action like a Page Down on a website. Speed below would not trigger anything in return, letting use the pointer to navigate.
How it works:
In conjunction with the orientation sensors and the gyroscope, this can identify the most complex movement. Without the orientation sensor, the controller could not detect the difference of movement with the controller rotation. Meaning that if I turned the remote upside down in my hand and make a slash up, the accelerometer actually detects the remote is going down. The sensor still does though and developers will have to make sure to use both sensors to get movements right.
Fun fact: the accelerometer calculates a force. Since gravity is a force, the controller would record a value on the y axis (up and down) of 9,8(earth gravity force). That could lead to funny behaviors if, let’s say, astronauts were using the controller in space where gravity would record at 0.
While with those three sensors, it would be technically possible to track the remote in space, it has been rarely used in the past due to sensors limitations as well as the requirement to manually recalibrate the controller when the position would be off as there are no internal components that can calibrate the real distance between the controller and the headset/player. HTC Vive on the other hand uses those external sensors, the light house tracking system. That’s why they can make it work.
Connection to the phone:
It is noted that the Daydream remote (or any Daydream certified remote made by 3rd parties) are required to launch Daydream on the supported phones. It is most likely that hardware partners will either rely on Daydream remote, or their own certified version to bundle with the headsets.
Using Bluetooth Low Energy provides efficient connections to the device. The Google VR Services is the part converting all the controller signals into the SDK where to offer 1 standard API for all Daydream controllers. It’s not know if only Partners will have the exclusive access to Specifications to ensure it’s compatible with Google VR Services, or if they will open up some certification process for 3rd party devices.
For example, the current Dev Kit for the Daydream platform lets you emulate a controller on any android phone (it works pretty well except for the click). The emulated remote currently use a totally different communication system, Wi-Fi direct. Because Google VR Services are running in the background there is no need to worry about Wi-Fi, the codes remain the same for Bluetooth so developers can test their apps and games without any problems.
Fun Fact: Because a touchscreen is flat and hitting the right button while in VR is pure madness, they provided developers some kind of plastic screen overlay to put on the emulated remote.
Finally, the phone has one final connector, the USB C that is most likely for charging up the internal battery. The device also has those small holes just below it that is most likely the wrist strap loop. In VR, it makes sens to tie in the remote so you don’t have to blindly search for it if you drop it by accident.
By forcing the use of the controller to users, Google made sure of one thing: The experience will be seamless and we should see variety of controllers pop up over the years to offer a good VR experience, affecting neither the users nor developers. So is the Daydream remote a WiiMote? It could be if we, as users, adopt the input method as well as VR in overall. It’s a logical and intuitive remote for user and retains a lot of hardware for developers. The lack of standard input in today’s ecosystem might be Google’s ace in the hole.
While Daydream should become available within weeks, we will learn and test the whole thing when it launches.
Skip to 0.43 for a demo of the Daydream controller CNET