WebXR has made building and deploying VR interfaces a lot easier, even when compared to developer friendly tools like Unity3D. I used NodeJS and ThreeJS to create an immersive controller for the Tello drone. 

Control

The 6dof motion controllers allow you to control the real-time position of the drone in an intuitive way with clear indicators above the controller to explain how the drone will react. The sphere above each controller will morph into an arrow indicating how the drone will move while pressing the ‘squeeze’ trigger will enable the commands to be sent. The left-hand control controls the X and Y axis of movement. Rotating the controller in place will move the drone forward, back, left, and right. Twisting it left or right controls the yaw, and will rotate the drone in place. The right controller, when rotated forward or backward, will move the drone up or down. while rotating it left or right will perform a left flip or right flip.

Telemetry

The Tello drone transmits an mpeg4 video feed and the back-end server passes chunks of .h264 data through a WebSocket where it’s rendered onto a curved, 3D plane in front of the user. The panel to the left of the screen shows a log of communication with the server and the four buttons below send the commands to connect, take off, land, and emergency stop. The battery to the right of the screen will display the current battery level percentage using the height of a cylinder with a color gradation from green to red. Below the screen, the virtual drone is a digital twin of the Tello and will show its real-time telemetry, including roll, pitch, and yaw, as well as the current state of its four props. (On or off)

Improvements

There’s currently a 2D only interface stub in there at the root index.html file. It’s got the ml5 machine learning image recognition library set up to read the video coming from the Tello drone. A better solution might be to decode the video server side in tandem with the remote client, do image recognition, then pass JSON data back to any clients though the comms websocket.

In addition to the current movement mechanism of tilting the controllers to indicate the direction to fly, the Tello drone provides x,y,z coordinate movement options. One neat way of controlling the drone might be to keep track of it’s current 3D coordinate space and allow the user to pick a point in 3D space for the drone to fly to.

Check out the code for yourself!

π