Optical Flow

I had a small case of keeping up with the Jones’s and decided to port an Optical Flow patch I saw on the C*******4 board. I added a few more parameters that they didn’t have :stuck_out_tongue: Its rudimentary, yet the basics are pretty cool. As always, Im sure could use some improvement. I built it utilizing a webcam, and might need to be optimized for higher rez for instance.

Feel free to hack it apart, I tried to focus on the Flow aspects, yet its pretty great at glitchy stuff as well. If that interests you, negative numbers for some parameters works awesome.

As an aside, I was SO SO thankful for Touch while building this :smiley:

Jeffers
opticalflow.tox (57.7 KB)

nice one jeffers. I tried to read you network, but didn’t get very far. Can you explain a little what’s happening in there? I’m trying to figure out if this optical flow stuff could be used for motion blur.

thx
Achim

Achim,

hahha, yeah I need to clean up the mess I made in there… I’ll get some better documentation, repost another TOX and report here as well. I realized I didn’t get to having a ‘default’ or ‘reset’ mode which would be nice. The values I posted the TOX with prob don’t make much sense. For starters setting Threshold=0 and High-Range=1 should get you more normalized.

In terms of using it as a motion blur, I was curious about this as well - follow the discussion here, tiny.cc/6uR7Z . In message 1802 there is some code and a brief explanation of what to do. Haven’t tried this yet.

Jeffers

nice, that VADE name pops up a lot. Have you seen his shader collection 001.vade.info/?page_id=20

nice - they are easy to implement in TouchDesigner. Just tried out a couple…

yeah pretty sweet. I put a TOX of the technicolor/film effects in the TOX forum.

Achim, there is also v002, tinyurl.com/ckc8mv , which includes a motion blur… I had no dice with it, same problem as last one I posted. maybe you find something else.

Jeffers

Here is a newer version of the Optical Flow, as I realized belatedly that there were some issues with relative/absolute paths(bad for sharing). The TOX defaults to using your webcam as input, so you should see something when you start up. I cleaned up, somewhat :slight_smile:, the insides, so poking around should be better. Also new is I put a line to demarcate where the pure color flow starts, so moving the RGB sliders to the right of this line puts you into pure color territory. There is a default/original button if you get lost.

I could use some help on the defualt/original buttons. I can override the slider values, tho how to override the RGB values eludes me.

Achim, for a brief, unscientific explanation of what is going on in there… the three main parts in order are,

  1. HSFLOW
    This breaks down the input image into R,G,B values, similar to a constant RGB delay.

  2. NORMAL
    This adds a motion blurry effect up to a value of 1, then the color separation starts.

3)FLOW Reposition
Offsets the HSFLOW values by the NORMAL motion blurry values

For Motion Blur, I might try isolating the Normal portion, go in an replace the RGB cubes with a single cube, so you don’t get color shift… set the values at around .75 … actually let me try something… :wink:

Also any luck with the blur function on the boards? I can’t seem to figure it.

Someone let me know if this works!

Jeffers
OpticalFlow1.tox (59.1 KB)

Achim,

I’ve isolated the Motion Blurry functions. I hesitate to call it Motion Blur… maybe Motion Blurish :wink: Perhaps it works for you, or you can use it as a base point.

Jeffers
motionblurry.tox (26.8 KB)

Here’s a version which is a little easier to read (cleaner layout and removed some duplicate/less important nodes). Hope I didn’t mess up anything.

flow.toe (94.4 KB)

jeffers,

this thing produces some neat images!

in the blur version, the hsflow MAT is in error, complaining about missing luma_TOP sampler. And the render TOP only shows an empty image (probably because of the missing texture).

I’m also wondering about the infinite recursion between the “flowrepos” and “normal” TOPs (error appears when modifying parameters)

Finally, what is the “raw output” of the optical flow analysis? Is there a way to see just that?

Thanks for sharing this!

Achim,

For Motion Blur, I realized the only necessary part was the NORMAL function, so I removed everything except this. There is still gives the recursion error, tho doesn’t seem to matter and necessary for this feedback loop. I also edited down the GLSL code, its VERY basic now. Again I’m not sure this is accurate, or traditional.

Jeffers
Motion_Blur4.tox (23.7 KB)

thanks jeffers

Hey,

I just udated these 2 components with the latest TUIK.

opticalFlow.jpg
opticalflow.tox (48.5 KB)
motionblur.jpg

cheers
Markus
motionBlur.tox (950 Bytes)

I just found the implementation is not correct. If my understanding is correct “the “raw output” of the optical flow analysis” should be R = right, G = left, B = up, A = down. Unfortunately I couldn’t get such a result with the Jeffers’s component. So, I revised it and I’d like to share these comp here. Thanks to Jeffers’s comp I noticed some GLSL based Max patch are not very difficult to port to Touch.
HSFlowDistortion.7.toe (68.8 KB)
HSFlowBasic.7.toe (54.5 KB)
HSFlow.tox (22.6 KB)

Thanks Itaru-san,
makes for a scary mirror!

Rodney

Thanks for playing with it Rod-san,
Actually the HSFlow shader is very useful we can make many interesting stuff not only the scary mirror ; )

Kinda new to TD. All I get is a black output. Do I need to do something to hook up my webcam to this thing?

hi sunspider, first try a blank touch project and put down a ‘videoIn’ TOP. Is it working okay? If it’s blank, it may be that you have some other program hogging the camera somewhere in the background?

rod.

Hmmm. No… VideoIn works fine.

Hi sunspider, I attached another toe example. You might try it. Hope it works.
HSFlowBasic.8.toe (54.2 KB)

Man, that’s cool. The movie works for me. Webcam still doesn’t, strangely. But I did get a version message when I ran the .toe, so maybe that is the problem. Anyway this works well enough for me to grok what’s happening! Thanks!