Augmented Reality and Touch Designer

[rodberry]

Dear all,
here are some screen shots of my Augmented Reality authoring with Touch project.

interface overview01800px.jpg (260.8 KiB) Viewed 2622 times

Above is the overall interface. You drag the marker icons onto your workspace then use them to parent the geometry you want to use in the scene. The extra null is to move the object up a bit so it is not place inside the marker but on top of it.

imagesurface_600px.jpg (180.54 KiB) Viewed 2625 times

here the real world image is used as both a texture and surface in real time. This kind of stuff you can't do with most AR authoring tools out there without a lot of hand coding.

It works in a similar way to Jörg's PrevizART application. There are a couple of extra tricks used to read the ARToolkit's camera calibration file to adjust for lens distortion etc. and reconcile the openGL matrix representation of the camera with Touch's camera parameters.

If anyone wants to try it out, you need the pro version of Touch077 in order to access shared memory.

um... video ... once I get around to joining vimeo ...

Rod.

[girBringerOfDoom]

super nifty!

[kamwong3]

Hi Rod,

It's cool! I have tried ARToolkit, but don't understand how you make it works in Touch. Did you use OSC to connect to ARToolkit result? Or you write the tracking code by yourself using scripts in Touch?

Kam

[rodberry]

Yep we cooked up our own program.

first it looks in the camera frame for black squares.

when i finds one, it then looks inside it to see if there's a glyph it's been taught to recognize.

if it sees one, then it checks which way up it is to get z rotation
then checks how big the black square is to get Z translation
checks where it is in the frame to get x and y translation
then checks if the square is more trapeziod than square to get the x and y rotations

then it puts the original frame into shared memory and at the same time puts the tracking data for that frame into shared memory as well.

touch designer then brings in the data using shared memory TOPs (pro version only, sorry)
a TOPtoCHOP operator and a bunch of chops then sort out the data to send to the translation fields of the geometry.

I'm cooking up a nice GUI to make this a drag and drop operation and also a control panel with which to launch the various camera calibration and marker training etc. I also hope to cook up a control panel to actually make and print markers ready for use (maybe even an automatic pattern data file generator).

If you have the pro version of Touch077, then I can let you try it out. ... but give me a month to tidy it up a bit first.

Rod.

[kamwong3]

Rod,
I see. I don't have the Pro version, but thanks anyway!
Kam

[obscura]

i'd love to see how you handled the matrix transforms... i'm working on a project that requires some projection translation functions and am having a tough time figuring out how to do it in touch!

[rodberry]

Short answer is I got someone smarter to handle it.

the registration is tidied up by reading the ARToolkits camera parameters file and using the lens distorion info to bend a mesh around before sticking the 'real' image onto it as a texture.

the matrix is translated into rotations and translations in degrees in the tracking program before feeding it into touch. owever, if you look in any null or geometry component, you can see a pre-transform tab which has a 'matrix chop' input' that lets you use a chop to feed in the 4 x 4 matrix. ARToolkit only needs 4 x 3 but same thing. you should be able to feed it to a chp then to the pretransform chop field.

rod.

[quine]

looks great! would love to give it spin in pro

[rodberry]

Hi,
I've put it on the backburner for a bit while trying to get my PhD work under control. However, I plan to get some interface improvements into it soon and use the pallette so you can just select a AR project and drop it ready to start work.

I've also cooked up some things to link with the marker-training and camera callibration programs so they get launched from a panel inside touch.

problem is, a student helped me with it and now I have to reverse-engineer his project to see whay it doesn't quite work yet!

rod.

[wolfmanbofb]

We are looking into to creating Augmented reality projects.
pros- clients are really excited
cons- none of us knows programming

I came across touchdesigner searching for augmented reality for non-programmers but reading this forum it looks like I need the pro version.

Is TouchDesigner Pro capable to create augmented reality or do I required different softwares? And can I get them publish over the net for the my clients?

thanks
Wolfman