A revolutionary 3D sensor from gaming industry designed to capture motion of players is effectively used in the robotic fields for a wide range of applications including objects recognition and tracking, 3D environment mapping, detect distance, or voice recognition and control. All these features make from Kinect the subject of this article where a set of setup and application tutorials are included.

In the following are available a wide range of setup tutorials for different versions of operating systems as well as different operating systems including Windows, Linux and Mac. All features of Kinect sensor listed above are used in different robotic applications and a series of tutorials to learn how to use these features are available in the following.

With endless possibilities for robotic applications, Kinect is a human-robot interaction tool with an RGB camera and an infrared depth camera. If an RGB camera is not a surprise, the depth camera enables the robot to build a 3D view of the environment.

The biggest disadvantage of Kinect is the behavior in outdoor applications where the performances are very poor. Kinect is designed for indoor applications and at least for the moment, the sensor can be used with high accuracy in indoor robotic applications.

Kinect SDK is a powerful tool released by Microsoft, a tool used to build applications like people detection without writing any programming line.

Why Kinect? Because it is an affordable sensor with 3D image capture in real-time while a laser array has a high price and capture 2D images, or stereo camera with high computing power requirements.

Setup Kinect

Kinect sensor can be used with Windows, Linux or Mac devices.
We are very close to start using Kinect for robotic applications, but first we have to set up the device on different operating systems including Windows, Linux, or Mac. Kinect sensor could be used for a wide range of applications including mapping, object recognition and tracking, voice control, or detect the distance between a robot and objects. All of these applications require a lot of processing power and without any choice, we have to use a mobile and powerful device like a laptop.The powerful device connected to Kinect must have a 32-bit or 64-bit processor with at least 2.66-GHz and dual-core, at least 2GB RAM memory, and dedicated USB 2.0 bus. All of these hardware requirements are available for devices with Windows operating system. Hardware requirements are completed with software requirements including here Microsoft Visual Studio 2010 Express or other Visual Studio 2010 edition, .NET Framework 4.0, and Microsoft Speech Platform SDK if the sensor is used for speech applications.

Below are available a set of tutorials with steps and information about how to setup Kinect device on different operating systems including Windows, Linux or Mac.

• How-to: Successfully Install Kinect on Windows (OpenNI and NITE) – in this tutorial you can learn how to use Kinect sensor on Windows platform. Programming the sensor and used in machine vision applications once with OpenNI framework and NITE middleware;
• Install Kinect on your PC and Start developing your Programs – this step by step guide show you how to install the sensor on Windows XP operating system as well as installing multiple drivers;
• Installing and Using the Kinect Sensor (Beta 2 SDK) – tutorial how to install the sensor and start using the speech recognition feature;
• Installing Kinect Windows 7 64-bit (using OpenNI and NITE) – step by step tutorial for sensor installation on Windows 7 64-bit version. For robotic applications like skeleton tracking or gesture recognition, tools like OpenNI and NITE are required and included in this tutorial;
• Setting up Kinect for programming in Linux Part 1, Part 2 – divided in two parts, the tutorial show you how to setup the sensor on the Linux Ubuntu operating system and start using the device for tracking and detecting objects;
• Getting started with Kinect on Ubuntu 12.04 – OpenNI, Nite , SimpleopenNI and Processing – installing sensor on Linux device is a process followed by a lot of commands. This tutorial explains all steps for a complete setup as well as installing tools like OpenNI, Nite, or SimpleopenNI, and start testing the sensor;
• Installing OpenNI, NITE and SensorKinect for Mac OS X – for processing images a set of tools is useful to be added once to Kinect sensor. Installing the OpenNI/NITE middleware packages along with Kinect sensor is a process described in detail in this tutorial;
• Setting up the Kinect on OSX 10.8 (Mountain Lion) – complete guide to setup the sensor on OSX 10.8 along with middleware packages like Processing, openFrameworks, or AS3;

Kinect Applications in Robotics

Navigation, recognition, tracking, detects distance, or voice commands are most common applications for Kinect sensor in robotics.
From DIY robotic applications and to advanced humanoid robots, Kinect is a precise and cheap tool for indoor navigation, recognition, tracking, detect distance, or voice commands. A long list of innovative robots came up with this advanced sensor including here a Kinect Quadrotor Bolting robot with ability to navigate and avoid obstacles autonomously as well as creating a 3D map, or an iRobot AVA telepresence robot with two Kinect sensors located into the body aiming to help the robot for self-navigating and detect motion or gestures. In the same category we can include other technologically advanced projects like DaVinci robotic surgical system that uses Kinect sensor for hand gesture control or industrial application where the sensor is used for 3D object scanning.

The same sensor with the same features is used in educational and research projects as well as for hobbyists from where started first applications in the field. A two-armed robot that replicates the movements of the user is the subject of Rensselaer Polytechnic Institute (RPI) engineers, or the researchers from Southwest Research Institute with an autonomous project for a robot designed to grasp objects using Kinect sensor and an Adaptive Robot Gripper.

Mapping, detecting, tracking is just few robotic applications where the sensor can be used. The sensor has to be attached to a robot like in this example, a process that requires technical skills while interface the sensor require software skills. A series of tools were developed and are very useful to start working with Kinect. All these tools include especially middleware packages to perform input/output communication for the device.

A high accuracy in machine vision implies a good calibration of tools used. The calibration of visual sensor is an advanced technical task, and at the same time is a crucial step to ensure a high accuracy and a good performance in 3D vision. Kinect camera calibration is a process to determine the relation between physical dimension of an object and the digital image of this object. Using OpenCV calibration routines, a comprehensive guide for device calibration is available here.

Tracking and Finding Objects

• How to create a basic Kinect application to control arrow keys using gestures – basic tutorial for hand gesture control using left and right arrow keys;
• Kinect – Getting Started – Become The Incredible Hulk – tutorial start working with Kinect SDK as well as start programming the sensor for the Skeleton Tracking application;
• Kinect Interaction with WPF Part II: Getting Started Easier – guide how to use NearMode property of the KinectChooserEx control;
• Kinect on a Pioneer Mobile Robot with Onboard SBC – guide how to use the sensor with Pioneer mobile robot on the Ubuntu operating system and a set of tools like OpneCV and OpenNI Drivers;
• Blob Tracking: Kinect + OpenCV + WPF – tutorial from set up to how OpenCV and Kinect can work together in order to create blob detection applications;
• Kinect and OpenCV 2.1 – another tutorial with programming lines in order to use OpenCV API’s together with the Microsoft vision sensor;
• Tracking Users with Kinect Skeletal Tracking – enabling the skeletal tracking feature of vision device using a lot of programming code in C# or C++ (depending on your preference);
• Implementing Skeletal Tracking -tutorial to learn how to use Kinect SDK and Windows API for Skeletal Tracking and the cursor moving by hand applications;
• Kinect SDK C++ – 1. Kinect Basics – guide with programming lines for sensor initialization and RGB data read from it;
• Tutorial – Kinect Television – tutorial with steps for connecting Kinect sensor, enable color and show camera feed, tilt the camera and setup for supporting different Kinect states, and selecting camera output at runtime;
• Cluster Recognition and 6DOF Pose Estimation using VFH descriptors – advanced guide from where you can learn how to use the sensor as a recognition tool and return potential candidates;
• KISS : Use Kinect for Windows SDK to protect your privacy – tutorial that explains how to use a small program called KISS (Kinect Intelligent Security System) to detect and track users. Used for security reasons, the program is a powerful tool that can be used also in robotics;
• How To Use Kinect Face Tracking SDK – code sample for Face Tracking application;
• Gestures With Microsoft Kinect for Windows SDK v1.5 – tutorial to initialize gesture recognition, capture and execute gesture in front of Kinect sensor;
• Tutorial 4: Depth Stream – first step to enter into the depth stream of Microsoft vision sensor;
• Kinect Physics Tutorial for Processing – tutorial that focuses on using libraries for interaction in real-time;
• Human Detection Using Depth Information by Kinect – tutorial for human detection in images using depth information;
• Getting Joint Position in 3D Space from the Kinect – comprehensive guide how to use the device to trace individual joints in 3D space;
• Integrated Complex Advanced Robotic Unmanned System – guide how to using ROS together with Microsoft sensor for tracking applications. Also, the tutorial offers comprehensive information how to have output commands via MAVLink to an APM board;
• Writing a gesture service with the Kinect for Windows SDK – C# programming lines for the sensor in order to analyze the gestures for control;
• Kibloc – Real time, distance based object tracking and counting using Kinect – tutorial for detection and objects counting using libraries like AForge.net and OpenCV;

Detect Distance to An Object

• Measuring using Kinect – guide how to use the sensor to measure the distance to an object;
• Getting distance-data from the Depth Sensor – comprehensive tutorial how to convert the distance into an image to to express the distance into a map;
• Working with Depth Data (Beta 2 SDK) – using C# or Visual Basic programming lines, this tutorial show you how to calculate distance between sensor and an object;
• Kinect SDK C++ – 2. Kinect Depth Data -code to get depth data from the sensor;
• Kinect Part 4 – Kinect Depth Camera – simple method how to calculate the distance using an infrared projector and infrared camera from the sensor;

Mapping With Kinect 3D

An autonomous robot requires a mapping system. A cheap sensor like Kinect is the perfect tool for a robot to build maps for localization. Using different software tools, we can use the Microsoft vision sensor for our autonomous robots and in the following we have a series of tools Kinect compatible to start mapping the rooms.

• Kinect 3D Mapping – tutorial to learn how to use the open-source software Kinect RGB and start building maps in 3D space;
• Skanect – powerful tool compatible with Microsoft sensor and designed to create 3D meshes of the environment;
• KinFu – a tool for real-time 3D scanning with the ability to save files in different formats;

Kinect Voice Commands for a Robot

• Kinect voice command tutorial – simple program to use the voice feature for Kinect sensor;
• SpeechRecognitionEngine.SpeechRecognized Event – simple example how to use Recognize or RecognizeAsync methods for voice recognition;
• Kinect SDK 1.0 – 5 – Speech Recognition – guide how to use the speech recognition capabilities of the Kinect;
• Speech Recognition in Kinect – comprehensive tutorial how to use Kinect voice feature to control your application;
• Intro to the Kinect SDK–Adding Speech Recognition – guide how to use Kinect NUI Microphone and the Microsoft Speech Recognition SDK for voice control;
• Speech recognition with the Kinect SDK – another guide how to use Kinect NUI Microphone and Kinect SDK for speech recognition and voice navigation;

Robotics community only grows by sharing