Our 'man in Berlin' Achim Kern has been busy as ever working on TouchDesigner projects for Derivative clients in Europe. HUGE undertakings that are really pushing the boundaries of existing technologies and emerging some new ones as we speak. Achim's report follows.
Client: Deutsche Telekom AG
Concept and realization: q~bus Mediatektur GmbH
TouchDesigner Programming: Achim Kern (www.achimkern.de)
Right on the heels of the job at Rockheim Museum in Trondheim (article coming soon) I was contracted by Berlin based q~bus Mediatektur to help in creating parts of an installation for the Deutsche Telekom booth at IFA 2010. The task was to create the real-time part of an interactive experience displayed on a 15 x 4.5 meter LED Wall with a resolution of 2560 x 768.
Working closely with the art director, technology group, conceptional and interaction designers as well as software developers, we created a system that features 3 different interactive scenarios interspersed with alternating sequences of Telekom commercials or interactive typography. All the scenarios were visually connected via seamlessly looping transition sequences, which automatically adapted their length to the current overlay commercial.
We had access to a backlit tracking bar, a custom tracker created by q-bus that was able to track the persons’ movements as well as the up/down-position of their arms. The tracking data was sent to TouchDesigner via OSC. We decided to limit the amount of concurrent users to 15 and therefore had to develop a system which automatically selects and matches a new user to one of the 15 available subsystems. For instance, there were 15 particle components, each connected to one of the tracked IDs/persons.
When a person left the tracking area, their attached component/particle system became available and would consequently be attached to the next person entering the tracking area (or to a person already in the tracking bar but lacking the attached component due to the fact that the allotted 15 were full). It would have been interesting to see how touch’s blob tracker would have performed in this scenario, but unfortunately we had no time to test it.
The 3 interactive scenarios were a mixture of life action footage and TouchDesigner generated “overlays”.
Sunrise is a particle based scenario which attaches a particle system to each user in the tracking bar. Using just the particle SOP and metaball force fields did not yield enough control over the movement of the particles and so, due to the lack of advanced particle tools (like POPs), we had to come up with an alternative approach to mimic the look & feel of a swarm. After testing various approaches, it turned out that it is possible to approximate swarm-like motions by applying noise fields, various rotations and well timed scaling effects to a simple sphere.
While it’s not actual flocking, the overall effect works very well and the movements are reminiscent of a fish-swarm. In order to achieve the trailing effect, a GPU based version of the trail SOP was created in addition to some feedback TOP based trails. This served to create nice long trails without much of a performance impact. By lifting and lowering their arms, the users were able to scale the particle system which followed them as they moved. When a user left the tracking bar, the attached particles quickly shot off-screen leaving visibly long trails – an effect extremely popular with the younger audience, who were jumping in and out of the tracking bar with abandon.
The sunrise scene was followed by transition sequences featuring either a Telekom commercial or interactive typography.
q-bus interaction designer: Before I knew TouchDesigner I have been developing interactive applications and artworks solely text based. I really liked the way how TouchDesigner let me bring in this experience without pushing me to work graphically only. With some practice and good tips from Achim I was able to replace more and more TextDATs with other nodes and become a true Touch Designer. :-)
Next up was the forest scene, a composite of live action footage in 3 layers and, again, 15 particle systems set to follow the users. The key task here was to allow particles to move behind as well as in front of some trees in order to create a realistic impression of taking a walk through the forest. As it was impossible to use only a single particle system to reliably handle attractors for up to 15 users and ensure that each person had an even amount of particles, I used 15 separate particle systems - each with its own set of force fields and collision objects (the trees, so particles would flow around them).
As this introduced a lot of overhead, a custom point-sprite shader was developed to do the coloring and the time dependent scaling/transparency effects without the use of a costly point SOP. In order to create the impression that the particles emit light, a lightning layer was calculated with the help of long b/w feedback based trails that were heavily blurred.
The last interactive piece – the sunset scene – followed another round of commercials. This scene also consisted of some real-time footage and overlaid lightning effects. Subtle light effects would creep along the building edges while people moved up and down the tracking bar. When they lifted their arms or took a step towards the screen, one of 12 different lightning effects shot up in the sky. Needless to say, this was another favorite with the kids.
All in all, this was an extremely exciting project and seeing the kids running around with big happy smiles on their faces was definitely rewarding – especially after all the hard work everybody at q~bus had put into the realization.