RealSense

From TouchDesigner 088 Wiki

TouchDesigner acquires multiple data streams from the Intel RealSense and outputs data from the RealSense CHOP and RealSense TOP.

RealSense.1.jpg RealSense.2.jpg

The RealSense CHOP connects to RealSense devices and outputs data from Intel's RealSense camera.

  1. Marker Tracking, where you provide it a TOP image and it will do object recognition to find that image in the camera view and output its position and rotation relative to the camera.
  2. Finger Tracking, which detects 10 gesture types.
  3. Face Tracking, which outputs a mesh patching key points of your the face it detects.

The RealSense TOP node outputs color, depth and IR data in various forms. It has a number of setups:

  1. Color, which is the video from the RealSense camera color sensor.
  2. Depth, a calculation of the depth of each pixel. 0 means the pixel is 0 meters from the camera and 1 means the pixel is the maximum distance or more from the camera.
  3. Raw Depth, values taken directly from the RealSense camera SDK. Once again 0 means 1 meter from the camera and 1 is the maximum range or more away from the camera.
  4. Visualized Depth, a gray scale image from the RealSense SDK which can help you visualize the depth but it cannot be used to actually determine a pixel’s exact distance from the camera.
  5. Depth To Color UV Map, the UV values from a 32 bit floating RG texture (note, no blue) that are needed to remap the depth image to line up with the color image. You can use the Remap TOP node to align the images to match.
  6. Color To Depth UV Map, the UV values from a 32 bit floating RG texture (note, no blue) that are needed to remap the color image to line up with the depth image. You can use the Remap TOP node to align the two.
  7. Infrared, the raw video from the infrared sensor of the RealSense camera.
  8. Point cloud, literally a cloud of points in 3d space (X, Y, Z coordinates) or data points created by the scanner of the RealSense Camera.
  9. Point Cloud Color UVs which can be used to get each point’s color from the Color image stream.


See the point cloud sample at: http://www.derivative.ca/Forum/viewtopic.php?f=22&t=7977

Installing RealSense Software

To use the RealSense cameras with TouchDesigner you will need to install two pieces of software from Intel. The first is the camera driver software (DCM) for the particule model of RealSense camera you have (F200 or R200). The second is the SDK Runtime Distributable. If you already have the full RealSense SDK installed you don't need to install the Runtime separately.

These downloads are available here.

Direct links:
SDK Runtime Distributable R4
F200 Camera Driver
R200 Camera Driver

See RealSense CHOP, RealSense TOP