Using the XBox Kinect with LabVIEW Robotics Starter Kit

https://decibel.ni.com/content/docs/DOC-15920

Description

I put an XBox Kinect on the LabVIEW Robotics Starter Kit to perform better obstacle avoidance.

Hardware Overview

Part Description	Vendor	Model Name/Number
NI Robotics Starter Kit (includes: Mobile Platform, Sensors and FPGA_Real-Time Processing Targets)	National Instruments	781222-01
Fit-PC	FitPC	fit-PC2-D2G-C2000-W
XBox Kinect	Microsoft

I also recommend using some connectors when hookup power to the Kinect and Fit-PC. I prefer Molex connectors: 70107-0002 and 50-57-9403

See http://decibel.ni.com/content/docs/DOC-12952 for info on how to hack in to the power on Starter Kit.

For the other hardware modifications: I used some standoffs to hold a piece of foam poster board over the Starter Kit. Then I used some velcro to attach the FitPC and Kinect. I wrapped up and secured the wires using zip-ties. That's about it!

Software setup and requirements

NI Robotics Starter kit software, which includes: LabVIEW Robotics, LabVIEW Real-Time, LabVIEW FPGA software modules

The hacked LabVIEW driver for the XBox Kinect

Windows Embedded 7 Standard on the FitPC (takes up less than 2GB!) plus at least the LV Runtime. I installed the full development system.

I set up the FitPC to connect to my wireless network and then used remote desktop to control it.

The code

Okay, so I kind of made this example using some unreleased software from the LabVIEW Robotics Module (whoops). So I can't post all of the code--but I can post the important pieces and tell you where they plug in.

From a high level, for the code on the Starter Kit, I simply modified the existing roaming example. Here's the main modification to the high level roaming example

I also modified the "Calculate Driving Direction.vi" to fuse the sonar and Kinect data. Unfortuately, that VI is missing so I can't post it. However, essentially it would use the Kinect data when available, but then default to the sonar data when there was no Kinect data. Also, if the sonar detected an obstacle that was really close (panic range), the robot would rely on the sonar data to steer clear of the nearby obstacle. Within Calculate Driving Direction.vi, there is a VI to read an element from the network stream to get the Kinect data. I set the timeout to 1ms so that we weren't waiting around forever for Kinect data while we run in to an obstacle.

The main VI is Kinect Obstacle Avoidance. A network stream is created to the sbRIO to transfer kinect heading data. First, the depth image is aquired and converted to a depth in inches. Then the data is converted to 3D coordinates (from pixel space). Afterwards, a small slice of the data is used for obstacle avoidance (after all, we don't need to avoid the ceiling). Then we find the largest gap and drive towards it. Otherwise, we turn around.

Converting to 3D coordinates from pixel space. On the first run of the VI, we calculate the matrix used to multiple the depth image by to get the x, y, and z values. The matrix is stored in a shift register so we don't have to do it every time.

Processing the data can be tough when you're dealing with ~300,000 data points for every data acquisition. Therefore, in order to make computation faster and more efficient, we take a 3D slice of the data. Notice how I preallocate memory for the sliced points and then replace the elements rather than generating an array. Why slice? Because in the next subVI, we're going to do a sort--personally I'd rather sort 10,000 points than 300,000.

Finally we find a heading. It's a little tricky with 3D data. I essentially just looked at the "Y data", which runs from the robot's left to right looking for gaps. If there is a gap big enough for the robot to fit through, we head there. In order to find the gaps, I sort the data and then take difference between neighbors. The largest values will be the gaps.

Adjuntos:

• Slice 3D Cloud.vi (19.0 K) LabVIEW 2010
  37 Downloads
• Kinect Obstacle Avoidance.vi (25.9 K) LabVIEW 2010
  36 Downloads
• Kinect Heading Data.ctl (10.4 K) LabVIEW 2010
  27 Downloads
• Convert to 3D.vi (28.0 K) LabVIEW 2010
  28 Downloads
• Convert Kinect Depth Image.vi (8.8 K) LabVIEW 2010
  33 Downloads
• Calculate heading.vi (14.8 K) LabVIEW 2010
  26 Downloads

Gusto (0)

808 Visualizaciones Categorías: Example CodeEtiquetas: labview, fpga, example, starter, 3d, robotics, robot, real-time, labview_fpga, robots, module, kit, real_time, autonomous, rt, realtime, sbrio, depth,xbox, starter_kit, navigation, obstacle_avoidance, labview_robotics, dani, obstacle, avoidance, labview-2010, kinect, 6d, windows_embedded, we7, wes7,fitpc, fit_pc, fit-pc