Video Device In TOP

From TouchDesigner 099 Wiki

Summary

The Video Device In TOP can be used to capture video from an external camera, capture card, capture dongle, or dideo decoder connected to the system. Multiple devices can simultaneously stream video into TouchDesigner by using multiple Device In TOPs. HD-SDI video can be streamed into TouchDesigner through capture cards such and those from Blackmagic Design.

If the device does not seem to provide a video stream but it is visible in the Cameras parameter menu, make sure no other applications are currently using the device.

For Blackmagic Design, see Blackmagic Design.

Also supported with native drivers are some Point Grey and Datapath SDI.

For Magewell HDMI-to-USB3 capture (highly recommended, no drivers, plug-and-play), see Magewell

For support of Bluefish devices, see Bluefish wiki and Bluefish website (Contact support@derivative.ca for info.)

For NVIDIA SDI hardware, see SDI In TOP. See also Video Device Out TOP.

PythonIcon.png videodeviceinTOP_Class

Parameters - Video Device In Page

On on - When set to one the TOP captures the image stream from the camera or decoder.

Library library - Selects the library to use to interface with the cameras.

  • DirectShow (WDM) directshow - Use DirectShow to interface with the camera on Windows (the camera should say it supports DirectShow or WDM.
  • Media Foundation mediafoundation - A replacement option for DirectShow with better support for custom webcam resolutions.
  • Imaging Source imagingsource - For use with cameras sold by Imaging Source.
  • DataPath (RGBEASY) datapath - For use with DataPath cameras.
  • Blackmagic blackmagic - For use with Blackmagic Design cameras.
  • Allied Vision (GigE) alliedvisiongige - For use with Allied Vision cameras.
  • Image Development Systems (IDS) ids - For use with IDS Image Development Systems cameras. Not supported on macOS.
  • Point Grey (FlyCapture2) pointgreyflycapture - For use with Point Grey cameras using the FlyCapture2 SDK. Not supported on macOS.
  • Point Grey (Spinnaker) pointgreyspinnaker - For use with Point Grey cameras using the Spinnaker SDK. Not supported on macOS.
  • AVFoundation (macOS) avfoundation - Use AVFoundation to interface with the camera on macOS.
  • BlueFish444 bluefish444 - For use with BlueFish444 cameras.
  • AJA aja - For use with AJA cameras and video systems.

NOTE: The cameras supported by the custom libraries above will also work with DirectShow on Windows, but using the custom library allows access to more camera controls and the control settings are saved in the node.

Device device - Select which camera or decoder you want from this menu.

Specify IP specifyip - When using Allied Vision library allows you to specify the camera address by IP.

IP ip - The IP address used when Specify IP above is turned on.

Options options - Opens the options or control panel for the camera.

Deinterlace deinterlace - Sets which fields to capture.

  • Off - Captures all fields.
  • Odd - Captures Odd fields only.
  • Even - Capture Even fields only.
  • Bob (Split) - Alternatively shows the even then odd fields, resulting in twice the framerate being shown, and removing the interlacing artifacts.

Field Precedence precedence - When using Bob (Split) deinterlacing, this selects which field is shown first for each frame.

TV Channel channel - Selects the TV channel if a TV tuner is used as the video input.

Signal Format signalformat - Some Blackmagic cards do not automatically set signal format, this parameter allows it to be set correctly.

Input Pixel Format inputpixelformat - Some capture devices support pixel formats other than 8-bit. For supported devices this will make the node attempt to use that capability.

Async Upload to GPU asyncupload - This will upload video images to the GPU asynchronously, at the cost of using more GPU and CPU memory (about 3x as for this one node). The extra memory usage is only a concern if you are using Async Upload on a lot of nodes (many Movie File In TOPs for example, which use the same mechanism).

Note: Syncing is only supported by DataPath cards currently.

Syncing allows multiple nodes using multiple inputs and capture cards on a single system to ensure they are outputting frames in sync. Without this each node will be free running and will possible be outputting frames that came it at different times due to internal queuing. It's important the input sources are GenLocked to ensure all of their data arrives to all of the inputs at the same time, otherwise syncing will not work.

Sync Inputs syncinputs - Enabling syncing of multiple Video Device In TOPs.

Sync Group Index syncgroupindex - There can be multiple sync groups active in a .toe file. Nodes will only sync to other nodes that are part of the same sync group.

Max Sync Offset maxsyncoffset - Specified in milliseconds. The maximum difference in time two image could have arrived at be considered in-sync. Images that arrive at times more different than this offset will be considered to be part of different 'frame'.

Sync Timeout synctimeout - Specified in milliseconds. Any of the inputs have not yet received a new image for the next required output, this controls how long the system can stall for, waiting for the missing images to arrive. It will freeze the playback for this number of milliseconds, so this number should be kept very small unless dropping frames is preferable to being out of sync.

Parameters - Common Page

Output Resolution - quickly change the resolution of the TOP's data.

  • Use Input - uses the input's resolution.
  • Eighth, Quarter, Half, 2X, 4X, 8X - multiply the input's resolution by that amount.
  • Fit Resolution - Resizes the input to the size specified in Resolution using the best possible match that does not crop any of the input. It will resize the image to be larger than the input resolution if a larger resolution is specified. It's a "fit inside", Aspect Ratio is maintained.
  • Limit Resolution - Limits the input to the size specified in Resolution using the best possible match that does not crop any of the input. It will NOT resize the image to be larger than the input resolution if a larger resolution is specified. It's a "fit inside", Aspect Ratio is maintained.
  • Custom Resolution - enables the Custom Res parameter below, giving direct control over res in the X and Y axis.

Resolution - enabled only when the Output Resolution parameter is set to Custom Resolution. Some Generators like Constant and Ramp do not use inputs and only use this field to determine their size. The drop down menu on the right provides some commonly used resolutions.

Use Global Res Multiplier - Uses the Global Resolution Multiplier found in Edit>Preferences>TOPs. This multiplies all the TOPs resolutions by the set amount. This is handy when working on computers with different hardware specifications. If a project is designed on a desktop workstation with lots of graphics memory, a user on a laptop with only 64MB VRAM can set the Global Resolution Multiplier to a value of half or quarter so it runs at an acceptable speed. By checking this checkbox on, this TOP is affected by the global multiplier.

Output Aspect - sets the image aspect ratio, which is the visible width vs height, independent of the pixel resolution. If the pixels are not square, the aspect ratio is not the resolution's width/height. Watch for unexpected results when compositing TOPs with different aspect ratios.

  • Input - uses the input's aspect ratio.
  • Resolution - uses the aspect of the image's defined resolution (ie 512x256 would be 2:1), whereby each pixel is square.
  • Custom Aspect Ratio - lets you explicitly define a custom aspect ratio in the Aspect parameter below.

Aspect - Use when Output Aspect parameter is set to Custom Aspect.

Fill Viewer - determine how the TOP image is displayed in the viewer.

  • Input - uses the same Fill Viewer settings as it's input.
  • Fill - stretches the image to fit the edges of the viewer.
  • Fit Horizontal - stretches image to fit viewer horizontally.
  • Fit Vertical - stretches image to fit viewer vertically.
  • Fit Best - stretches or squashes image so no part of image is cropped.
  • Fit Worst - stretches or squashes image so image fills viewer while constraining it's proportions. This often leads to part of image getting cropped by viewer.
  • Native Resolution - displays the native resolution of the image in the viewer.

NOTE: To get an understanding of how TOPs works with images, you will want to set this to Native Resolution as you lay down TOPs when starting out. This will let you see what is actually happening without any automatic viewer resizing.

Viewer Smoothness - This controls pixel filtering in the viewers.

  • Nearest Pixel - uses nearest pixel or accurate image representation. Images will look jaggy when viewing at any zoom level other than Native Resolution.
  • Interpolate Pixels - uses linear filtering between pixels. This is how you get TOP images in viewers to look good at various zoom levels, especially useful when using any Fill Viewer setting other than Native Resolution.
  • Mipmap Pixels - uses mipmapfiltering when scaling images. This can be used to reduce artifacts and sparkling in moving/scaling images that have lots of detail. When the input is 32-bit float format nearest filtering will always be used, regardless of what is selected in the menu.

Pixel Format - format used to store data for each channel in the image (ie. R, G, B, and A). Refer to Pixel Formats for more information.

  • Input - uses the input's pixel format.
  • 8-bit fixed (RGBA) - uses 8-bit integer values for each channel.
  • sRGB 8-bit fixed (RGBA) - uses 8-bit integer values for each channel and stores color in sRGB colorspace.
  • 16-bit float (RGBA) - uses 16-bits per color channel, 64-bits per pixel.
  • 32-bit float (RGBA) - uses 32-bits per color channel, 128-bits per pixels.


  • 10-bit RGB, 2-bit Alpha, fixed (RGBA) - uses 10-bits per color channel and 2-bits for alpha, 32-bits total per pixel.
  • 16-bit fixed (RGBA) - uses 16-bits per color channel, 64-bits total per pixel.
  • 11-bit float (RGB), Positive Values Only - A RGB floating point format that has 11 bits for the Red and Green channels, and 10-bits for the Blue Channel, 32-bits total per pixel (therefore the same memory usage as 8-bit RGBA). The Alpha channel in this format will always be 1. Values can go above one, but can't be negative. ie. the range is [0, infinite).
  • 8-bit fixed (Mono) - Single channel, where RGB will all have the same value, and Alpha will be 1.0. 8-bits per pixel.
  • 16-bit fixed (Mono) - Single channel, where RGB will all have the same value, and Alpha will be 1.0. 16-bits per pixel.
  • 16-bit float (Mono) - Single channel, where RGB will all have the same value, and Alpha will be 1.0. 16-bits per pixel.
  • 32-bit float (Mono) - Single channel, where RGB will all have the same value, and Alpha will be 1.0. 32-bits per pixel.
  • 8-bit fixed (RG) - A 2 channel format, R and G have values, while B is 0 always and Alpha is 1.0. 8-bits per channel, 16-bits total per pixel.
  • 16-bit fixed (RG) - A 2 channel format, R and G have values, while B is 0 always and Alpha is 1.0. 16-bits per channel, 32-bits total per pixel.
  • 16-bit float (RG) - A 2 channel format, R and G have values, while B is 0 always and Alpha is 1.0. 16-bits per channel, 32-bits total per pixel.
  • 32-bit float (RG) - A 2 channel format, R and G have values, while B is 0 always and Alpha is 1.0. 32-bits per channel, 64-bits total per pixel.
  • 8-bit fixed (A) - An Alpha only format that has 8-bits per channel, 8-bits per pixel.
  • 16-bit fixed (A) - An Alpha only format that has 16-bits per channel, 16-bits per pixel.
  • 16-bit float (A) - An Alpha only format that has 16-bits per channel, 16-bits per pixel.
  • 32-bit float (A) - An Alpha only format that has 32-bits per channel, 32-bits per pixel.
  • 8-bit fixed (Mono+Alpha) - A 2 channel format, one value for RGB and one value for Alpha. 8-bits per channel, 16-bits per pixel.
  • 16-bit fixed (Mono+Alpha) - A 2 channel format, one value for RGB and one value for Alpha. 16-bits per channel, 32-bits per pixel.
  • 16-bit float (Mono+Alpha) - A 2 channel format, one value for RGB and one value for Alpha. 16-bits per channel, 32-bits per pixel.
  • 32-bit float (Mono+Alpha) - A 2 channel format, one value for RGB and one value for Alpha. 32-bits per channel, 64-bits per pixel.


Scripting

oppardetails -p /videoin1 (for example) lists all the dynamic menu choices (as does oppardetails -p /videoin1 device). If there are no choices listed it might be an indication no cameras are installed.