Optical Flow

I’ll dig the optical flow thing more later.

Fore now I was able to get the one from the pdf working (which is also in latest nvidia sdk docs.nvidia.com/gameworks/index. … sample.htm)

I get nice results but since it’s using normals to stretch meshes it doesn’t work with flat or hard edge geometry, so I’m trying to get the advanced sample to work now.
docs.nvidia.com/gameworks/index. … sample.htm

Are there any updates on this working in 088?
thanks

I wrote a research paper on hardware accelerated motion blur in 2003 for 3D using the vertex data. There is a pdf and video on my website.

clementshimizu.com/hardware- … eneration/

This full image space optical flow is very exciting.

Clement

Hey,

just spend a little time with this and fixed it up to work (or so I believe) in the experimental branch of 088.

All Parameters are now on the component as custom parameters.
I think this is what it’s suppose to do - feedback welcome.

cheers
Markus
opticalFlow.jpg
opticalFlowFx.tox (4.01 KB)

Hi Markus,

Thanks, just having a very quick look, I don’t think it’s supposed to be blue, is it?

edit : I guess I was still half asleep it’s actually working as it should, just the parameters saved with the tox make it blue.

Hey guys,

Sorry to revive an old thread.

With this example, is it possible to pull out the vector information into chop channels.

I saw this implementation here: vimeo.com/114542945 and would love to be able to use the vector data for driving other animations / visualizations.

Cheers,

O.

Hey, yes you can always extract the vectors information using a TOPtochop, make sure to reduce image resolution and use the next frame mode, otherwise it can be pretty slow.

Vinz, I would love to see an example of how you might do that. I’ve been scratching my head all afternoon trying to work it out.

Hey Jason,

Sure I’ll post the setup in the next couple days.

Yes! indeed vinz , seeing an example of how to execute that would be great!

10 years later, here you go!

Cheers,
Vincent
optical_vectors_display.toe (6.96 KB)

Thank you, thank you, thank you! Will test it out when I’m back in the studio :smiley:

Hi Vinz

Is this meant to be the vectors of pixel movement like optical flow or something else? Maybe I just don’t get it, but if I connect a more simple source to this setup to see it more clearly(a circle top for instance, moving left to right) it’s showing vectors going in almost all directions, when I would expecct it to be only the axis of movement. Am I missing something or is the shader not actually designed to show this?

Hi, keep in mind the original optical flow shader by Andrew Benson which is used here is fast but a rough approximation, compared to more sophisticated but much slower algorithm found in opencv.

It doesn’t do too well on hard edges, and you’ll get better results if you blur your circle beforehand. Even if vectors are a bit off the overall direction is correct.

Cheers

Funny timing, just saw it was added to the palette, same algorithm though. (though the palette one needs its format forced to 16/32bit float or it will clamp results to positive values)

Build 2017.7180 - Jun 05, 2017
New Palette
opticalFlow added to Tools folder in Palette.

Thanks for pointing this out! Will be 32bit float texture in next release.

Best
Markus

Could somebody be so kind as to provide me an example of practical usage of this technique? Its inclusion in the palette suggests that this is of value, yet my previous comments about inaccuracy have led me to pigeon-hole it as an inaccurate, purely aesthetic technique. Is this correct?

Ha :wink: depends what you mean by “practical”, I’m mostly in this business to make pretty useless things to look at and play with, this is a recent example with this optical flow technique : instagram.com/p/BVSaPBonvKN … cent.houze

Also this one uses the same optical flow technique to push the fluid around : vimeo.com/218695680

love it - especially the ‘practical’ bit.

Hey Vincent - is there a reason you used texture with a calculated UV offset rather than textureOffset?

edit - forgot my new sig

thanks in advance. real money slots.

Thanks! I think historical reason for not using textureOffset, like this was probably adapted from an old shader.