Login | Register 
FEATURES
APPLICATIONS
DOWNLOADS
EDUCATION
BLOG
WIKI
FORUM
STORE

Hitting bottleneck controlling close to 4,000 LEDs !

General discussion about anything TouchDesigner

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby lucasm » Sun Oct 19, 2014 2:57 pm

Glad it helped!

Sounds like a lot of splitting and re ordering. If you can isolate the part of your TOE that you're talking about I could take a look at it. I'm not hugely experienced at optimizing but there might be a few things you could do.

For example, if your rgb data is coming from a single top at some point in your network before splitting, you could try a reorder TOP and switch the channels there, it might be more efficient.

Any step you're doing multiple times on multiple streams, see if there's an alternate way of doing it in another OP type further back in your stream when things are merged.
User avatar
lucasm
 
Posts: 223
Joined: Sat Apr 28, 2012 7:55 pm
Location: Dallas, TX

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby lucasm » Sun Oct 19, 2014 3:24 pm

FM64,
Actually you mind if I ask what your hardware setup is like?

What kind of Artnet controllers are you using? Leds?
User avatar
lucasm
 
Posts: 223
Joined: Sat Apr 28, 2012 7:55 pm
Location: Dallas, TX

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby FM64 » Mon Oct 20, 2014 5:58 am

Hi Lucas,
thanks for your support, here is a very simplified and commented .toe, it's great if you have the time to take a look at it !
I'm currently using WS2812b ledstrips, I know it's kind of slow and innacurate but it does the job.
This is the controller : http://www.electrondes.com/an6u1903_00/an6u1903_00.pdf
it looks cheap but works incredibly great !
Attachments
LEDmapper_v2.10.toe
(9.03 KiB) Downloaded 236 times
User avatar
FM64
 
Posts: 28
Joined: Thu Nov 28, 2013 5:44 pm

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby rob » Mon Oct 20, 2014 11:49 am

Hi Everyone.
Thanks for the great analysis and working solutions.

Some thoughts on the python script:

Lucasm, can you describe your issue with:
"A side note here, I have to split the led bytes into chunks or packets of 255 or less due to a built in limit that sendBytes has.. "

Looking at your sample script, I notice you do things like:

exec('op("%s").sendBytes(%s)'%(serialConnectorName, str(subArray).strip('[]')))

Instead of:

op(serialConnectorName).sendBytes(subArray)

Though I suspect: .sendBytes(*subArray) may work in your case.

Minimizing the amount of scripting altogether would likely improve performance.
(ie, using a Quantize CHOP to avoid the int() calls, and a shuffle CHOP to get everything in the right format).

It might boil down to:

n = op('some_resulting_chop') #everything interleaved and with shuffle CHOPs
v = n[0].vals #put everything into one float array
op('serial_dat').sendBytes(v)

Also, agreed, FM64, those reorder CHOPs seem unnessarily slow.
Thanks for the examples, we will try to optimize them further.

We're also researching the idea of changing the formats accepted by the DMX CHOPs to avoid the interleaving altogether.

Cheers,
Rob.
User avatar
rob
Staff
 
Posts: 1547
Joined: Tue Nov 13, 2007 1:14 am

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby rob » Mon Oct 20, 2014 1:48 pm

A couple more points:

The latest official, Shuffle CHOP has a relatively recent option (August 11th):
"Sequence All Samples"
Which would take three long channels, (example: r, g, b), and reshuffle it into one
single interleaved channel: (r0, g0, b0, r1, g1, b1, ....)
This may be of use in some networks.

Also, we've just optimized the Reorder CHOP to be 5 to 10 times faster when dealing with a few hundred channels, which is often the case with these LED networks.
That will be in build 25440 or later.

Cheers,
Rob.
User avatar
rob
Staff
 
Posts: 1547
Joined: Tue Nov 13, 2007 1:14 am

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby FM64 » Mon Oct 20, 2014 6:19 pm

I'ts awesome Rob, thanks a lot for your reactivity, can't wait to see my network run @60FPS ;)
User avatar
FM64
 
Posts: 28
Joined: Thu Nov 28, 2013 5:44 pm

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby lucasm » Tue Oct 21, 2014 1:30 am

Rob, thanks a bunch for sharing your thoughts on that part of the send code. Ill be honest i was looking to optimize everywhere else but there.. Everything you suggested did LOADS of good..

Precisely the bits about shuffle CHOP to get the samples ordered correctly.
Building that ordered array via loop in python was also eating up a ton of time. The append function was probably being used WAY too much.

My FPS is around the 60 range and solidly sticking. All of it is split between two instances of Touch Designer with out the help of any other programs . Woo!

Not quite sure why I was approaching it the way I was with strings, but using a direct array with that asterisk in front of it did perfectly..... and loads faster.
What's with the asterisk anyways? I noticed it stripped away the commas when sent through a print statement, I'm curious, is that all it does?

-------

All that being said, here's the new Serial send code, it's LOADS faster and WAY simpler.
You're the man Rob:

Code: Select all
def valueChange(channel, sampleIndex, val, prev):

   n = op('topto1') # CHOP to pull rgb data from
   serialConnector = op("serialConnector") # serial DAT
   
   vals = [int(i) for i in n[0].vals] # returns a list with values converted to ints.
   serialConnector.sendBytes(*vals) # Send data to leds!
   
   return
   


I've attached a new TOE file to reflect the changes and updates.
Attachments
simpleLedMappingAndDrivingWorkflow.6.toe
Modified to reflect the optimizations Rob suggested and some other simplifications in the TOE
(9.35 KiB) Downloaded 212 times
User avatar
lucasm
 
Posts: 223
Joined: Sat Apr 28, 2012 7:55 pm
Location: Dallas, TX

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby lucasm » Tue Oct 21, 2014 2:10 am

FM64,

I took a look at your file, made a few changes and attached it.
I switched the order of operations up a bit and moved a few things to your ledstrips container.

I was able to cut it down by almost about half, but there might be a way to squeeze a few more MS out of it. It's getting by at just over 60 though.

I suspect that won't hold if you have more going on in that file but if you split it off to another touch process I imagine it would hold fine!


You may already be doing this but when you test fps, open your performance monitor and then go into performance mode and then hit analyze. Touch's UI takes a good chunk it's self understandably and that can make it seem worse than it is.
Attachments
LEDmapper_v2.12.toe
(8.98 KiB) Downloaded 222 times
User avatar
lucasm
 
Posts: 223
Joined: Sat Apr 28, 2012 7:55 pm
Location: Dallas, TX

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby rob » Tue Oct 21, 2014 12:33 pm

That's awesome. Im glad all our bits are making one fast system.
One more thing:

vals = [int(i) for i in n[0].vals] # returns a list with values converted to ints.
serialConnector.sendBytes(*vals) # Send data to leds!

The .sendBytes will cast each entry to int anyways, but to be extra sure, you can use the Limit CHOP to quantize/round a channel to whole numbers, eliminating that line entirely.

As for the python * operator, apparently called the 'splat' operator, which expands a list or tuple into separate arguments in a function call.
User avatar
rob
Staff
 
Posts: 1547
Joined: Tue Nov 13, 2007 1:14 am

Re: Hitting bottleneck controlling close to 4,000 LEDs !

Postby snaut » Tue Oct 21, 2014 12:55 pm

You can save a bit more by not using a select CHOP but a constant CHOP with a replace CHOP in the dmx_output COMPs.
The expression you are using in the select CHOP can be copied and pasted into the name parameter of the constant CHOP, then use the constant CHOP as the first input to the replace and the select2 CHOP as the second input. This little trick saves you another ~0.18 ms per dmx output...

cheers
Markus
User avatar
snaut
Staff
 
Posts: 846
Joined: Mon Nov 12, 2007 6:21 pm
Location: Toronto

PreviousNext

Return to General TouchDesigner Discussion

Who is online

Users browsing this forum: Google [Bot] and 7 guests