Kinect TOP

From TouchDesigner 088 Wiki

Summary

The Kinect TOP captures video from the Kinect depth camera or RGB color camera.

NOTE: This TOP works with the Kinect for Windows hardware and supports the Kinect2.

To use a Kinect 2 device you need to install the SDK or runtime from here.

To use the original Kinect 1 device you will need to install the Kinect Runtime 1.8 for builds greater than 12000, or Kinect Runtime 1.7 for builds below 12000.

It supports color point clouds - getting the camera space positions of the color pixels, outputted as a 32-bit float RGB texture with XYZ in RGB. For use in making point clouds renders.

See also Kinect CHOP, Kinect and Kinect1.

PythonIcon.png kinectTOP_Class

Parameters - Kinect Page

Hardware Version hwversion - Choose between Kinect v1 or Kinect v2 sensors.

Sensor sensor - Selects which Kinect sensor to use. Only available when using Kinect v1.

Image image - Selects between the Color, Depth, Infrared, Player Index, or Color Point Cloud modes.

  • Color - Uses the raw image from the RGB camera.
  • Depth - Textures the range is from 0-1 where a pixel with a value of 1 means the pixel is 8.191 meters from the camera.
  • Infrared - Uses the raw image from the infrared camera.
  • Player Index - The pixel value will be in increments of 0.1 for each player number. I.e a pixel with value ~0.1 will be player 1, ~0.2 will be player 2, etc. The reason is it approximate 0.1 (~) is because the buffer is 8-bit which can't exactly express 0.1, 0.3 etc. So 0.1 * 255 = 25.5 which could round off to a pixel value of 25 or 26 (out of 255).
  • Color Point Cloud - The texture will be a 32-bit floating point texture where RGB pixel values are XYZ pixel values relative to the color camera, in meters.

Camera resolution camerares - Only used for Kinect 1 devices. Selects the resolution of the camera capture.

Skeleton skeleton - Only used for Kinect 1 devices. Specify whether to track full skeleton or seated skeleton.

Near Depth Mode neardepthmode - Only used for Kinect 1 devices. Enables near mode for the depth camera, which allows the depth camera to see objects as close as 40cm to the camera (instead of the default 80cm).

Mirror Mode mirrormode - Flips the image in the y-axis.

Camera Remap remap - Only used for Kinect 2 devices. Enabling this will remap images that are natively from the depth camera (Depth, Infrared, Player Index) to be in the space and resolution of the Color camera instead.

Too Close Value tooclosevalue - Only used for Kinect 1 devices. For depth pixels that are too close to resolve, this pixel value will be output instead.

Too Far Value toofarvalue - Only used for Kinect 1 devices. For depth pixels that are too far to resolve, this pixel value will be output instead.

Unknown Value unknownvalue - For depth pixels whose depth can not be determined, output this value instead.

Unknown Point Value unknownpointvalue - When using the 'Color Point Cloud' some pixel's position can not be determined. This parameter controls what value to assign those pixels instead.

Parameters - Common Page

Resolution - quickly change the resolution of the TOP's data.

  • Input - uses the input's resolution.
  • Eighth, Quarter, Half, 2X, 4X, 8X - multiply the input's resolution by that amount.
  • Custom Resolution - enables the Custom Res parameter below, giving direct control over res in the X and Y axis.

Custom Res - enabled only when the Resolution parameter is set to Custom Resolution. Some Generators like Constant and Ramp do not use inputs and only use this field to determine their size. The drop down menu on the right provides some commonly used resolutions.

Use Global Resolution Multiplier - Uses the Global Resolution Multiplier found in Edit>Preferences>TOPs. This multiplies all the TOPs resolutions by the set amount. This is handy when working on computers with different hardware specifications. If a project is designed on a desktop workstation with lots of graphics memory, a user on a laptop with only 64MB VRAM can set the Global Resolution Multiplier to a value of half or quarter so it runs at an acceptable speed. By checking this checkbox on, this TOP is affected by the global multiplier.

Aspect Ratio - sets the image aspect ratio, which is the visible width vs height, independent of the pixel resolution. If the pixels are not square, the aspect ratio is not the resolution's width/height. Watch for unexpected results when compositing TOPs with different aspect ratios.

  • Input - uses the input's aspect ratio.
  • Resolution - uses the aspect of the image's defined resolution (ie 512x256 would be 2:1), whereby each pixel is square.
  • Custom Aspect Ratio - lets you explicitly define a custom aspect ratio.

Fill Viewer - determine how the TOP image is displayed in the viewer.

  • Input - uses the same Fill Viewer settings as it's input.
  • Fill - stretches the image to fit the edges of the viewer.
  • Fit Horizontal - stretches image to fit viewer horizontally.
  • Fit Vertical - stretches image to fit viewer vertically.
  • Fit Best - stretches or squashes image so no part of image is cropped.
  • Fit Worst - stretches or squashes image so image fills viewer while constraining it's proportions. This often leads to part of image getting cropped by viewer.
  • Native Resolution - displays the native resolution of the image in the viewer.

NOTE: To get an understanding of how TOPs works with images, you will want to set this to Native Resolution as you lay down TOPs when starting out. This will let you see what is actually happening without any automatic viewer resizing.

Viewer Smoothness - This controls pixel filtering in the viewers.

  • Nearest Pixel - uses nearest pixel or accurate image representation. Images will look jaggy when viewing at any zoom level other than Native Resolution.
  • Interpolate Pixels - uses linear filtering between pixels. This is how you get TOP images in viewers to look good at various zoom levels, especially useful when using any Fill Viewer setting other than Native Resolution.
  • Mipmap Pixels - uses mipmapfiltering when scaling images. This can be used to reduce artifacts and sparkling in moving/scaling images that have lots of detail. When the input is 32-bit float format nearest filtering will always be used, regardless of what is selected in the menu.

Pixel Format - format used to store data for each channel in the image (ie. R, G, B, and A). Refer to Pixel Formats for more information.

  • Input - uses the input's pixel format.
  • 8-bit fixed (RGBA) - uses 8-bit integer values for each channel.
  • 16-bit float (RGBA) - uses 16-bits per color channel, 64-bits per pixel.
  • 32-bit float (RGBA) - uses 32-bits per color channel, 128-bits per pixels.


  • 10-bit RGB, 2-bit Alpha, fixed (RGBA) - uses 10-bits per color channel and 2-bits for alpha, 32-bits total per pixel.
  • 16-bit fixed (RGBA) - uses 16-bits per color channel, 64-bits total per pixel.
  • 11-bit float (RGB), Positive Values Only - A RGB floating point format that has 11 bits for the Red and Green channels, and 10-bits for the Blue Channel, 32-bits total per pixel (therefore the same memory usage as 8-bit RGBA). The Alpha channel in this format will always be 1. Values can go above one, but can't be negative. ie. the range is [0, infinite).
  • 8-bit fixed (R) - has 8-bits for the red channel, 8-bits total per pixel.
  • 16-bit fixed (R) - has 16-bits for the red channel, 16-bits total per pixel.
  • 16-bit float (R) - has 16-bits for the red channel, 16-bits per pixel.
  • 32-bit float (R) - has 32-bits for the red channel, 32-bits per pixel.
  • 8-bit fixed (RG) - has 8-bits for the red and green channels, 16-bits total per pixel.
  • 16-bit fixed (RG) - has 16-bits for the red and green channels, 32-bits total per pixel.
  • 16-bit float (RG) - has 16-bits for the red and green channels, 32-bits per pixel.
  • 32-bit float (RG) - has 32-bits for the red and green channels, 64-bits per pixel.
  • 8-bit fixed (A) - An Alpha only format that has 8-bits per channel, 8-bits per pixel.
  • 16-bit float (A) - An Alpha only format that has 16-bits per channel, 16-bits per pixel.
  • 32-bit float (A) - An Alpha only format that has 32-bits per channel, 32-bits per pixel.


Device Support

This TOP only works with the Kinect for Windows hardware device. Kinect for Xbox360 devices will not work.

See Also