Login | Register 
FEATURES
APPLICATIONS
DOWNLOADS
EDUCATION
BLOG
WIKI
FORUM
STORE

Two kinects(v2)

Discuss graphics hardware, computer hardware, and input/output devices to use with TouchDesigner

Re: Two kinects(v2)

Postby CutMod » Thu Mar 23, 2017 6:22 pm

I've been playing with streaming a second Kinect pointcloud from an Intel NUC.

phpBB [video]


The best way I could figure out so far is Pack TOP -> Touch Out TOP -> [LAN] -> Touch In TOP -> Pack... it has decent quality uncompressed, but super laggy. Maybe it's just the little i3 in my NUC, but I imagine that super high res image is heavy for a single stream. Any suggestions?
CutMod
 
Posts: 134
Joined: Thu Jul 03, 2014 10:44 am
Location: Los Angeles

Re: Two kinects(v2)

Postby barakooda » Fri Mar 24, 2017 1:35 am

if reduce resolution whats happens .?

what hardware you use for the network?
User avatar
barakooda
 
Posts: 546
Joined: Fri Nov 27, 2009 5:08 pm

Re: Two kinects(v2)

Postby L05 » Fri Mar 24, 2017 8:48 am

Hey Will,
I actually have a Kinect hooked up to an Intel NUC (i7) setup for a current project. I'm running a lot of the computations I need on the NUC itself, but I'm sending RGB, IR, and tracking data feeds over Cat6 with no problems. I think at one point I had tried sending the packed depth data and it worked fine, but I'll do a more intentional test later today and report back.

## EDIT ##
So I tested the following setup:
Kinect v2 on Intel NUC (i7) -> Cat6 -> Desktop

TEST:
On the Intel NUC...
Kinect TOP (sending Color Point Cloud) -> Pack TOP -> Touch Out TOP.

On the Desktop...
Touch In Top -> Pack TOP (set to Unpack RGB) -> Malcom's example pointcloud.toe (http://derivative.ca/Forum/viewtopic.php?f=17&t=6587&hilit=kinect+point+cloud).

Observations:
- The Desktop only received usable data when the NUC's Touch Out was set to Uncompressed. I'm assuming this is because the Pack TOP uses the alpha channel and HAP does not support alpha.
- It was laggy, similar to the video you posted.
- Maybe the 5760 x 1080 resolution from the Pack TOP is crushing the NUC's iGPU? Performance seems to tank on the NUC once I add that to the mix. Also, sending the uncompressed 32-bit float RGB texture of the Color Point Cloud seems to slow down at the network level.
- Halving the resolution somewhat improved performance, but it still wasn't smooth (and that's not ideal).

Hope this is helpful!
User avatar
L05
 
Posts: 201
Joined: Thu Jan 05, 2012 2:36 pm
Location: Los Angeles, CA

Re: Two kinects(v2)

Postby ben » Tue Mar 28, 2017 4:26 am

What about trying the new NDI OPs since they transit alpha?
User avatar
ben
Staff
 
Posts: 2369
Joined: Thu Nov 08, 2007 6:55 pm
Location: Toronto

Re: Two kinects(v2)

Postby L05 » Tue Mar 28, 2017 3:05 pm

The NDI In and Out TOPs are really exciting. From an initial look at the NDI TOPs, it seems like they send an 8-bit fixed RGBA texture, which may still require the Pack TOP on the NUC and therefore the 5760x1080 resolution. We just deinstalled and are on another project now, but I'll give it a shot on the NUC as soon as I get a chance. Regardless, I'll definitely be using the NDI TOPs moving forward. :D

Ben, is there any way to access the shader code and/or matrices used to align the Kinect v2's 512x424 depth image with the 1920x1080 RGB image for the color point cloud image? I'm wondering if it might be possible to send the smaller depth and RGB textures over the network and do the alignment after receiving it, since the NUC seems to handle those alright.
User avatar
L05
 
Posts: 201
Joined: Thu Jan 05, 2012 2:36 pm
Location: Los Angeles, CA

Re: Two kinects(v2)

Postby malcolm » Fri Mar 31, 2017 11:26 am

NDI is a lossy compressed format as far as I can tell so you can't pack pixels with it. Pixel packing only works with uncompressed or lossless compression.
User avatar
malcolm
Staff
 
Posts: 4228
Joined: Tue Nov 13, 2007 1:11 am

Re: Two kinects(v2)

Postby malcolm » Fri Mar 31, 2017 11:34 am

I just checked and unfortunately no you can't do the remapping on a second machine. The code that does this is hidden by Microsoft and comes from the sensor directly (since they would all likely have slightly different remappings).
User avatar
malcolm
Staff
 
Posts: 4228
Joined: Tue Nov 13, 2007 1:11 am

Re: Two kinects(v2)

Postby CutMod » Fri May 26, 2017 1:30 pm

So far it seems we've all been talking about sending the second pointcloud over network, but what about via capture card? With a high quality capture card, could you pack 1080p 32bit to 4K 8bit on sending machine then capture that with a 4K capture card and unpack back to 32bit?

I only have some 1080p Blackmagic Intensity cards so I can't test. I remember using a Datapath card once that had a "force 444" mode we used to ensure accurate checkerboard mask for a 3D screen. Is there a capture card that would preserve the 8bit image well enough to unpack to 32bit?

My understanding of the lower level process is lacking, but I'm wondering: Is there a way to do this with pixel math as opposed to bitwise math? I.e. create 4x 8bit images that would additively composite together to represent the original 32bit image? Maybe this would be more noisy but I've had times when unpacking failed altogether because the bits didn't line up(?), giving a blank image. I'd take noisy over nothing... just an idea.
CutMod
 
Posts: 134
Joined: Thu Jul 03, 2014 10:44 am
Location: Los Angeles

Re: Two kinects(v2)

Postby yeataro » Mon May 29, 2017 10:07 pm

I got some way, but it is not final answer,
and very busy for this project now, maybe I will sharing code after this week(after refine it) and expect someone to re-refine it.

phpBB [video]

phpBB [video]


(inverted colors is the second kinect)
User avatar
yeataro
 
Posts: 8
Joined: Tue Sep 28, 2010 2:31 pm
Location: Taipei

Re: Two kinects(v2)

Postby CutMod » Tue May 30, 2017 12:24 am

very nice! Would be very interested to learn more about your workflow when you have the time. :geek:
CutMod
 
Posts: 134
Joined: Thu Jul 03, 2014 10:44 am
Location: Los Angeles

PreviousNext

Return to Hardware

Who is online

Users browsing this forum: No registered users and 2 guests