HoloLens is one of the most exciting technologies that we have in our days to build ‘mixed reality’ in robotics. The technology is designed to change the virtual exploration as we know it. And because we need it to change the way we build robots, the Microsoft’s engineers give us a helping hand to use it in the robotics community.
Back in time, Microsoft announced the Windows Holographic platform in January 2015. Since then, the platform has found its place in NASA’s applications to explore Mars, or in the games industry to play Minecraft. All of these industries are completed by an application for Windows 10. This universal application merges the HoloLens platform with Windows 10 devices.
A short demo shows us how the HoloLens can help the real world with a little virtual reality to create applications part of a mixed reality.
If you plan to use the HoloLens platform in your robotic project, here are the six ingredients that you can use.
01. Build your custom virtual hologram for your robot
You can use the Microsoft’s Windows Holographic platform to bring to life your robot with an artificial body. The 3D modeling techniques applied to a robot can make it react like in the real world.
In the first presentation of the HoloLens technology applied in robotics, the Microsoft’s engineers applied a robot avatar on a small robot. The robot avatar floats above it and creates a more familiar and anthropomorphic form.
02. Brings 3D mapping capabilities
The HoloLens platform is designed with 3D-mapping capabilities. This means you can engineer robot to navigate and move around and guided by 3D maps.
The Microsoft platform for virtual reality provides terrain maps and is capable to recognize obstacle from the perspective of the robot’s operator. In this way, HoloLens create waypoints to guide the robot and avoid obstacles.
This navigation technology combines the spatial computing and sensor processing of Windows Holographic. More than that, it has a huge potential combined with Kinect sensors or other sensor hardware. In future, this system that combines hardware components with virtual reality will be able to control more complex autonomous robots.
The ability to recognize objects and environments is one of the most useful HoloLens features for robot applications. Applied to a robot navigation system, the platform scans points in 3D space and builds a route to move on.
Of course, without sensors this technology is pointless. A single board computer such as Raspberry Pi 2 does not have built-in light or movement sensors. But it has GPIO pins for adding different shields, sensors, etc.
03. Generate control panels as you wish
To control a robot, you need a control panel. If besides buttons there are some data to visualize, everything becomes more clear.
The Windows virtual platform can generate a control panel and display the data you need from sensors. This is only half of the truth. The second part has all the fun. Even you have a virtual control panel that cannot be touched, you can control the robot by touching virtually the buttons, switches, etc.
In the Microsoft’s demonstration with a robot built using a Raspberry Pi 2, the control panel of the robot display data about battery life, temperature, wireless connection, and several other variables.
04. Control the robot
Generating a virtual control system is one of the mind-blowing features of the HoloLens platform. The second feature is the interaction.
Wherever the robot travels, you can have next to it a control panel to control the lights and visualize all the data from the robot’s sensors. The virtual reality platform features a system able to display all these data in real time while the user controls the robot through the holographic interface.
05. Build robots with Raspberry Pi 2 and HoloLens
The link between Raspberry Pi 2 and HoloLens is Windows 10 for embedded devices. The Microsoft’s goal is to install the Windows 10 operating system on 1 billion devices in the next few years. The HoloLens should be one of the key drives to reach that goal.
The HoloLens is designed to help makers build gadgets, robots, or appliances using simple and cheap prototyping boards such as Raspberry Pi 2. In fact, the virtual reality helps the maker to envision how a robot or gadget would look and work.
The Raspberry Pi 2 is only one platform able to run Windows 10. There are also platforms such as Intel Galileo and Microsoft Sharks Cove with support for Windows 10 and compatible with HoloLens.
06. Test and debug the robot
The HoloLens could also change how the makers build and test products with prototyping boards such as Pi 2. The platform opens for these all the aspects of a product to examine before the physical touch.
With it, any developer could gather sensory information and control the robot as part of testing and prototyping before to build physical the product.
The hologram is the key to testing and debugging your robot. Using holograms, the developer can easily separate, examine and focus on individual systems.
Conclusion
In this article, I find six main features to use the HoloLens technology in robotics. Given these reasons, I think that this mixed reality-computing platform is the most ambitious project of Microsoft for the robotics community.
Let me know in the comment section what are your plans to use this virtual technology in robot applications.
