Book of Shaders and TouchDesigner

I can’t say enough good things about The Book of Shaders by Patricio Gonzalez Vivo & Jen Lowe. If you’re looking to get a handle on how to write shaders, or find some inspiration this is an incredible resource.

For TouchDesigner programmers who are accustomed to the nodal environment of TD, working with straight code might feel a bit daunting - and making the transition from Patrico’s incredible resource to Touch might feel hard - it certainly was for me at first. This repo is really about helping folks make that jump.

Here you’ll find the incredible examples made by Patricio and Jen ported to the TouchDesigner environment. There are some differences here, and I’ll do my best to help provide some clarity about where those come from.

This particular set of examples is made in TouchDesigner 099. In the UI you’ll find a list of examples below the rendered shader in the left pane, on the right you’ll find the shader code and the contents of an info DAT. You can live edit the contents of the shader code, you just have to click off of the pane for the code to be updated under the hood. If you hit the escape key you can dig into the network to see how everything is organized.

Each ported shader exists as a stand alone file - making it easy to drop the pixel shader into another network. When possible I’ve tried to precisely preserve the shader from the original, though there are some cases where small alterations have been made. In the case of touch specific uniforms I’ve tried to make sure there are notes for the programmer to see what’s happening.

Download the repo and start getting your GL on.

github.com/raganmd/BOS-in-TouchDesigner

2 Likes

Being hungry for this topic, I have started collecting the very few bits and pieces regarding GLSL I could find (for beginners) in TD and I accidentally stumbled upon The Book of Shaders referenced in Michael Walczyk tutorial: youtube.com/watch?v=AWhJpD2 … utu.be&t=1
Sadly, he hasn’t continued his series, which seemed very promising, but TBOS has given my happy goosebumps as I am starting my GLSL learning path!
I was stuck, away from my computer, with nothing but an iPad for a few days. Because of the books online, interactive examples, I could continue learning :slight_smile:

Anyways, I am really happy you ported these Matthew! Another valuable asset for the community. I will not look through them yet though, as I want to work through them myself first :wink:

I really hope they will work their way through the whole book. So I urge you guys, If anybody else find the book valuable, please donate, so that these guys can write up the next chapters.

Ok… I couldn’t help myself and I had to look through it after all, hehe :wink:

I must congratulate you Matthew! This is such a nicely wrapped up of The Book of Shaders examples! WOW!!! Really well presented for people who want to get their hands dirty with GLSL code!

Glad it makes sense and is interesting to pull apart. I went through the book of shaders several times and this was the result of one of my final passes through the examples. If it can help others overcome the steep learning curve it will have been well worth my time.

I do have one question to ask:
When going through the gradient and green line examples, the code for assigning the color values to the pixels became a bit complicated. Coming from the layered world of Adobe and the way you composite layers in TD, it would have been so much clearer for me as a beginner to have a function to comp these two results together. Just like we can add, multiply, divide and subtract two vec3 results, it would be great to add the line on top somehow.
Would anyone have an idea of how to do that?
(Or is this maybe a separate topic?)

As much as possible I’ve tried to faithfully copy the examples from the original source - which is why those bits are all tied together here.

Long story short - it is a slightly different topic, as the line in these examples is really about showing the shape of y.

plot is the function that’s being used to describe the line - where y is the value describing the slope of the line.

plot is defined above and outside of the main loop on line 22:

float plot(vec2 st, float pct){ return smoothstep( pct-0.02, pct, st.y) - smoothstep( pct, pct+0.02, st.y); }

smoothstep is a built in function that gives us a nice gradient:

https://www.khronos.org/registry/OpenGL-Refpages/es3.1/html/smoothstep.xhtml

The color of the line in the first example can be found on line 51:

// plot our line float pct = plot( st, y ); color = (1.0-pct) * color + pct * vec3( 0.0, 1.0, 0.0 );

It’s the vec3 on the color line:

// plot our line float pct = plot( st, y ); color = (1.0-pct) * color + pct * vec3( 0.0, 1.0, 0.0 );

Another way to write this would be to declare our color outside of this operation:

[code] // plot our line

float pct 		= plot( st, y );
vec3 lineColor	= vec3(0.0,1.0,0.0);
color 			= (1.0-pct) * color + pct *lineColor;[/code]

This would make it a little more easy to see where the line color is actually coming from. If you wanted to be able to change the color from a set of parameters, you could declare this as a uniform outside of the main loop:

[code]// uniforms
uniform vec3 uLineColor;

// functions

// Plot a line on Y using a value between 0.0-1.0
float plot(vec2 st, float pct){
return smoothstep( pct-0.02, pct, st.y) -
smoothstep( pct, pct+0.02, st.y);
}

out vec4 fragColor;
void main()
{
// TouchDesigner provides us with a built in variable
// that already holds the uvs for our texutre. Normally we’d
// you’ll see this done other places with fragcoord and the
// resolution of the scene. We could similarly derive this value
// like this:
// vec2 st = gl_FragCoord.xy / uTD2DInfos[0].res.zw;
// here gl_FragCoord provides the pixel value, and uTD2DInfos[0].res.zw
// provides the xy resolution of our first input.
vec2 st = vUV.st;

// we're going to create another variable out of our st vec2. This
// time we only want the x value, we don't need to recalculate anything
// instead we can use the vec2 we've just declared. 
float y 		= st.x;

// we can create a vec3 with the same values in all three positions
// with the shorthand below. This is functionally equivilane to"
// vec3 color 	= vec3( y, y, y );
vec3 color 		= vec3(y);

// plot our line
float pct 		= plot( st, y );
color 			= (1.0-pct) * color + pct * uLineColor;

// TDOutputSwizzle is a TouchDesigner function that helps ensure 
// consistent behavior between mac and pc versions of touch. What's
// important to know here is that you need to provide this function
// with a vec4. Because our example above doesn't consider alpha, 
// we can construct a vec4 out of our variable color, and an additional
// value of 1.0 for the alpha channel.
fragColor 		= TDOutputSwizzle(vec4( color, 1.0 ));

}[/code]

You’d then need to add the uniform on the GLSL TOP’s uniform page - but then you’d be able to change the color through the pars on the op - as these uniforms are being passed to the shader.

What these first examples are showing is the shaping nature of functions, the addition of the line is more about how you can visualize that process. If look at the color examples that will start to show how color can be manipulated.

I’d also highly recommend looking at the 2D ShaderToy examples and port:
shadertoy.com/view/Md23DV

ported to touch here:
github.com/raganmd/learningGLSL

Seems like it might be worth doing a similar repo that’s just the 2D examples in a similar fashion to the Book of Shaders. :slight_smile:

Thanks for explaining that. I totally understand that you made a faithful port of TBOS, so it is not a criticism or a suggestion to change your code. My question was just related to the fact that I had a hard time understanding what was actually going on on the coloring part of the code. At first the code looked like this to me:

(something - something) * why is that there + that variable again * green

But after breaking it down, operation by operation, it now makes sense.
If anybody else is confused, try adding one math operation at the time and then you start seeing the logic of why you need to turn certain pixels to white or black before you can multiply back the green from the line.

To make the code cleaner, at least for a beginner, I would have loved to wrap all of this inside it’s own function to help keep the focus on the what is happening to the line and the gradient. I think I will do that while I am going through the rest of these examples.

Also, thanks for linking us to that Shadertoy tutorial and for the TD port :slight_smile:

Started in on the 2D Tutorial port today:

[url]https://github.com/raganmd/glsl2dTuts-in-TouchDesigner[/url]

Cool! GLSL learning resources overload! :smiley:

And you also made a comment just for me, how nice :wink:

completed the GLSL 2D Tutorials port this evening:

viewtopic.php?f=27&t=10676

Admirations Matthew, I started to use TD and grew up in it because of your tutorials.

Thank you so much Matthew!
I’ve been going through your GLSL files for the last couple of weeks. They have been extremely helpful!

I put together a tox that can hopefully help others as they’re collecting shaping functions
shaping_tool.tox (2.24 KB)

Very much looking forward to check this out!!!

so the green line one is giving me lots of trouble, too. matthew’s explanations have helped a bit, but there’s still a lot i do not understand. in lieu of asking 500 questions, i’ll just ask these two:

  1. what do “st” and “pct” stand for? I understand (i think?) that these are variables that we define, but I really don’t understand their function, or what they’re referring to. commenting things out, and changing some numbers, i see interesting changes in the code’s behavior, but none that i find predictable.

  2. this probably has a complicated answer, but why are we plotting a line twice? there’s a function defined outside the main loop that’s commented as plotting a line… then in the main loop we plot our line… huh?

this is definitely not the cakewalk that was Python.

Hey Corbin,

Hang in there - it gets easier, it just takes some time.

1 - wtf is ‘st’ and ‘pct’?
There’s a lot to that question. Swizzling is a technique that’s used to grab some portion of a larger variable. A vec3, for example, is a variable with three positions. In pythonic terms is tuple with three positions. You can both access and update those values with dot notation. Let’s use rgb as a starting idea. So let’s say you have a variable like this:

vec3 color = vec3( 0.0, 0.0, 0.0 );

You want to update your color to be red… so you could do this

vec3 color = vec3( 0.0, 0.0, 0.0 ); color = vec3(1.0, 0.0, 0.0);

or you could just choose to update the red value of your variable by swizzling:

vec3 color = vec3( 0.0, 0.0, 0.0 ); color.r = 1.0;

There are three sets of swizzle masks you can use:
.xyzq
.rgba
.stpq

From the glsl documentation:

khronos.org/opengl/wiki/Data_Type_(GLSL#Swizzling

Okay… so back to st… because we still don’t know what that is. The big picture thing to remember is that anything .someSwizzle is just grabbing values from a variable. So line 38 that defines st as vUV.st is where we really have to dig in.

vUV is a variable provided by touch that’s the normalized coordinates for the texture - you might otherwise have to do this by hand by looking at the gl_FragCood of a given fragment (mostly you can think of fragment = pixel). My attempt in porting these was to preserve as much of the original BOS code as possible, which is why I go to the trouble of using the BOS variables, even if that only points back to something provided by touch. In this case, we don’t have to do the math to create a set of normalized coordinates, since they’re already ready already… instead we just create a new variable that points to our existing built-in.

To understand what pct is, however, we need to pull apart the function smoothstep().

smoothstep() takes three arguments - you might think of these as the gradient start, gradient end, and the fragment you’re evaluating. The use of smoothstep is to produce a linear gradient that starts at arg1, ends at arg2, by evaluating arg3. What the function returns is a 0 to 1 value based on that evaluation. In order to get a soft edge plotted line in this example we have to write a function to do that… plot(). This function is going to return a float value. The syntax for construction of the function means that we need to declare its data type fist, then provide its name, followed by the data type for the arguments it will take…

Let’s look at our function blog that’s outside of the for loop:

float plot(vec2 st, float pct){ return smoothstep( pct-0.2, pct, st.y) - smoothstep( pct, pct+0.02, st.y); }

plot will return a float value, its first argument is a vec2 (a tuple with two positions) called st, and a float value called pct. It’s going to return the difference of two different smoothstep functions - we might think of this as the left and right side of our line - since what we’re after is a line with a soft edge on both sides. In our function, pct is a variable that’s going to be used for the second argument. In the main loop, pct describes the returned value from the plot() function. I think that’s part of what might be confusing here.

2 - why are we plotting this line twice?!
We’re not. We’re just defining the function that we’ll use in our main loop. The pieces in main() are executed for every fragment, the functions we define outside of main aren’t executed until they’re used in the main loop somewhere. This is similar to how you can define a function in a dat, but not actually see the results until you call the function:

[code]# here we define our function
def print_my_str(my_str):
print(my_str)
return

but it’s not executed until we call it here

print_my_str(“hello world”)
[/code]

Does that help? or just make for more questions?

“In our function, pct is a variable that’s going to be used for the second argument. In the main loop, pct describes the returned value from the plot() function. I think that’s part of what might be confusing here.”

Yep, that’s the does-not-compute part.

but i think i’m starting to get it… a little.

can the void main() be thought of as a for loop? you said float pct = plot(st, y); is evaluated for each fragment. so is that sort of like saying:

for frag in allFrags: execute this function with these variables

and then each time we run that plot() function, it refers to the declaration prior to the main loop to know how the arguments st and y relate to each other?

i’m still having trouble seeing how exactly this all comes together to create the gradient with the green line. line 48 vec3 color = vec3(y) is what creates the black to white gradient; that makes all the sense in the world, and i’m loving it.

line 52 loses me a bit. is it sort of cordoning off a group of pixels according to the plot() function and then coloring them green? that line of code looks like it should be pretty easy to pick apart, but this pct variable just keeps teasing me. why don’t we have to declare what type of data '“color” is, i.e. float or vec? and how is it that we can use its name in its definition?

what i’m really hearing is that i need to go back and learn Python functions again; i remember it taking me a hot second for those to click, and i never really write my own.

So for the sake of legibility we can swap around a few things on lines 51 and 52 to see if this makes more sense:

// plot our line float line = plot( st, y ); color = (1.0-line) * color + line * vec3( 0.0, 1.0, 0.0 );

If we do this, than pct only shows up as the variable that’s in the function defined outside of the main loop. Does that help?

main() is a function that’s run for every fragment in your texture - the idea close to a for loop, the big divergence being that we can’t really use the same ideas from for loops when thinking about main(). For loops are sequential, which lets us do things like accumulate values. For example, let’s say that I want to add together all of the values in a list. I could do this in python with something like:

[code]values_to_be_summed = [1, 3, 4, 3, 1]
result = 0

for each_val in values_to_be_summed:
result += each_val

print(result)[/code]

That’s great, and I know that I’m adding to result sequential as I go through the list. It’s easy to take for granted that you have access to the whole list, and the process is ordered.

Shaders don’t function that same way. You’ll execute the main() function on each fragment, but you don’t know what order that’s happening in, nor do you know what’s happening in the fragment next to you, or the one that was evaluated before or after you. All to say that glsl is looking to evaluate the results for each fragment simultaneously and in parallel, rather than sequentially. This matters because it changes the logic we use for how to think about drawing.

and then each time we run that plot() function, it refers to the declaration prior to the main loop to know how the arguments st and y relate to each other? 

It’s better to think that you’re running the plot() function for every fragment - and into that function you’re passing a vec2 and a float value. That function is only going to return a value that’s between 0.0 and 1.0. You could do all of this without defining a function outside of main() by changing line 51-52 to be:

float line = smoothstep(y - 0.02, y, st.y) - smoothstep(y, y+0.02, st.y); color = (1.0-line) * color + line * vec3( 0.0, 1.0, 0.0 );

That does the exact same thing.

Better still, if you want to take out all the arguments from that function, and just look at the smoothstep() function without any other funny business you could re-write that to be:

float line = smoothstep(st.x - 0.02, st.x, st.y) - smoothstep(st.x, st.x + 0.02, st.y);

Okay, so this brings us to your next question -

Looking at line 52:

color = (1.0-line) * color + line * vec3( 0.0, 1.0, 0.0 );

Important to consider here is that our color variable refers to our output color for the texture. Okay, let’s start with (1.0-line). line is float that’s the returned value of our plot() function. It’s going to be a value between 0.0 and 1.0. The operation 1.0 - line will return a result that’s going to be from 1.0 - 0.0. Sanity check:

[code]f(line) = 1 - line

f(0) = 1.0 - 0.0 = 1
f(0.5) = 1.0 - 0.5 = 0.5
f(1.0) = 1.0 - 1.0 = 0.0
[/code]

We’ve essentially just remapped our result to be from 1.0 to 0.0 rather than 0.0 to 1.0. Next we take that result and multiply it by color. Right now color is a variable that’s the x value of our normalized fragment coordinate - we see this in the shader with float y = st.x, and then later when we see color is a vec3 of y vals. Our plot() function, however, uses normalized y value (you’ll also see this called v), we see this as st.y. We’ve gotta go back to smoothstep() for a minute to understand what’s happening.

Smoothstep’s args look like:

smoothstep(start_of_gradient, end_of_gradient, value_we_are_checking);

Try this in a glsl TOP:

[code]out vec4 fragColor;
void main()
{
vec3 color = vec3(1.0);
float alpha = 1.0;

float gradient = smoothstep(0.25, 0.75, vUV.s);
color *= gradient;

vec4 outCol = vec4(color, alpha);
fragColor = TDOutputSwizzle(outCol);

}[/code]

Here are gradient starts at 0.25 and ends at 0.75 - before 0.25 the value returned from smoothstep() is is 0.0, after 0.75 it’s 1.0.

So far so good.

Okay, but what if I want a result that’s a gradient that goes up to white, then back to black? Or from 0.25 to 0.5 is a gradient going up to white, then from 0.5 to 0.75 it ramps back down to black. Try this:

[code]out vec4 fragColor;
void main()
{
vec3 color = vec3(1.0);
float alpha = 1.0;

float gradient = smoothstep(0.25, 0.5, vUV.s) - smoothstep(0.5, 0.75, vUV.s);
color = color * gradient;

vec4 outCol = vec4(color, alpha);
fragColor = TDOutputSwizzle(outCol);

}[/code]

What if instead of white I want that color of that to be green?

[code]out vec4 fragColor;
void main()
{
vec3 color = vec3(1.0);
float alpha = 1.0;

float gradient = smoothstep(0.25, 0.5, vUV.s) - smoothstep(0.5, 0.75, vUV.s);
color = color * gradient;	
color = color * vec3(0.0, 1.0, 0.0);

vec4 outCol = vec4(color, alpha);
fragColor = TDOutputSwizzle(outCol);

}[/code]

The biggest difference in getting from the above to the BOS examples is that in the BOS we’re thinking in 2 dimensions instead of just 1. So finally we can get back to your question…

The last thing that’s missing here is order of operations… pink elephants must die at sunrise - PEMDAS:
parenthesis
exponents
multiplication
division
addition
subtraction

It’s implied but not explicit in this:

code * color + pct * vec3( 0.0, 1.0, 0.0 )
[/code]

if we add some parenthesis to clear it up we’ll see:

((1.0-pct) * color) + (pct * vec3( 0.0, 1.0, 0.0 ))

The left hand side is added to the right hand side… the right side gives us a green gradient that’s shaped like our line and that’s added to our left hand side where we’ve created a feathered black gradient for our green to fit into. WHAT?!

Try this - replace line 52 with:

color  = (1.0-pct) * color;

now try:

color = pct * vec3( 0.0, 1.0, 0.0 );

Hopefully you’ll be able to see that the combination is about getting the green line over the gradient in the space where we’ve soft black line:

color  = (1.0-pct) * color + pct * vec3( 0.0, 1.0, 0.0 );

We actually do declare color up on line 48.

vec3 color = vec3(y);

Shaders are a real brain twister… most of the ideas are grounded more in mathematics so going back to python might help you think about building functions, but going back and brushing up your linear algebra will also be a huge help. You might also look at the glsl2D tutorials that were ported from shader toy… that’s where I started before BOS, and that might help you get grounded a little better.
matthewragan.com/2017/08/02/gls … hdesigner/

Hope that helps at least a little.