Binary and Morphology Operations

[svanderbeck]

We are working on a project where it would be beneficial to perform some binary operations in touch designer. I was wondering if anyone had any suggestions on some general strategies for performing these in Touch Designer.

I know the Convolution TOP can be used to perform some operations, but any suggestions for operations such as Erosion or Dilation?

Also, we are looking to be able to perform "hole" filling on blobs. Is there some strategy to perform a flood fill operation from a given pixel?

[malcolm]

There is a component in the palette that does erosion and dilation. It may not be quite what you want so you may need to edit the shader a bit (avoid doing operations 'inside' your image for example).

Floodfill is an operation that doesn't really lend itself to GPU acceleration. It would require many, many passes to perform the operation.
Perhaps a blur and a threshold will do the trick though?

[svanderbeck]

Thanks Malcom! I didn't even think to check the palette for that. I will have to give it a try.

I will also have to give the blur and threshold a try for filling holes. I think that should be sufficient for eliminating some of the smaller holes from noise. I think it might run into issues with some of the larger holes generated by shadows, etc. I don't know to much about GPU programming... Would the best approach be to hand off to c++ where it could do a linear time depth first search (or BFS) to perform the fill?

Also, any way you know of to perform a min or median filter?

[malcolm]

Ya if you have to do the floodfill then it's possible the C++ TOP is the way to go.
If the resolution is low then it may be doable in a GLSL TOP with multiple passes.

The Analyze TOP can do minimum or average but that's for the entire image (output is a single pixel). You would need a GLSL TOP if you are trying to do an minimum or median over a small filter size.
I'm happy to help you along the way if you want to learn how to write the GLSL yourself.

[svanderbeck]

Thanks! Yeah, I was thinking of the min and median filter for each pixel for a small neighborhood. I suppose you could break the image into many smaller images and then do it for each small image, and then re-assemble the many results back into the original resolution but I am guessing that is way more inefficient then just writing it in GLSL.

I would love to learn more about about writing GLSL. Do you have a suggestion for the best starting point? It would save us from having to bounce between software in some cases.

[malcolm]

I wrote an article that is TouchDesigner specific:
http://www.derivative.ca/wiki/index.php ... a_GLSL_TOP

There are loads of tutorials you'll find on google that will get you started quite quickly in a more general sense.

[svanderbeck]

Great article for getting started quickly. So would something like the code below work for a min filter in a 3x3 neighborhood. I will need to do some homework to see how this gets handled on boundary pixels.

Also, in general is there any specific way to deal with binary images... or is best to just pick a channel to work with and use 0.0 for false, and 1.0 for true and render the pixel to black or white accordingly?

Code: Select all

uniform vec4 uInputRes1;
uniform sampler2D sInput1;

vec2 input1Offset(int xOffset, int yOffset)
{
  return vec2(gl_TexCoord[0].s + (float(xOffset) * uInputRes1.s), gl_TexCoord[0].t + (float(yOffset) * uInputRes1.t));
}

void main()
{
 vec4 colorSum = vec4(0.0);
 colorSum = texture2D(sInput1, input1Offset(0, 0));
 colorSum = min(colorSum,texture2D(sInput1, input1Offset(-1, -1)));
 colorSum = min(colorSum,texture2D(sInput1, input1Offset(0, -1)));
 colorSum = min(colorSum,texture2D(sInput1, input1Offset(1, -1)));
 colorSum = min(colorSum,texture2D(sInput1, input1Offset(1, 0)));
 colorSum = min(colorSum,texture2D(sInput1, input1Offset(1, 1)));
 colorSum = min(colorSum,texture2D(sInput1, input1Offset(0, 1)));
 colorSum = min(colorSum,texture2D(sInput1, input1Offset(0, -1)));
 colorSum = min(colorSum,texture2D(sInput1, input1Offset(-1, 0)));

 gl_FragColor = colorSum;
}

[malcolm]

Yup, that's the idea, although it's operating per-channel. So you'll get the min red, min green, min blue and min alpha in your output. Not sure if that's what you want. If you are looking for overall pixel brightness you may want to get the luminance of the pixel

Code: Select all

dot(color, vec3(0.3, 0.6, 0.1))

and do a minimum of that. There is no general 'correct' way to do this, it totally depends on what you need.

The boundary pixel values you get can be controlled by the Extend Mode menu in the GLSL TOP.

[svanderbeck]

Great! Yeah I was figuring it was already operating on a grayscale or binary image where R=G=B and alpha=1 so it wouldn't impact the result. I suppose I could probably save some runtime by just doing the operation on one of the channels and then copying the result to the others.

Is the issue with doing a floodfill in GLSL that you could do the recursive search to find all the pixels connected to a given pixel to be filled, but ultimately only return the result of that single pixel (without changing all the other pixels that you identified as being in that fill region)... Hence the need for the many passes?

It seems like it would be possible to do one of the following in a single pass:

1. If the current pixel is foreground, determine the area of all foreground pixels in this pixels connected region. If that area is less than some threshold, move it to background. This should eliminate "small" holes

2. For each pixel in a list of pixels to be floodfilled, if there is a connected path of foreground pixels between this pixel and the floodfill pixels, move this pixel to background. (Assuming you can pass data into the shader which is still something I need to learn)

Thanks again!