Jun.13.19 Deepfake Salvador Dalí Interacts with Museum Visitors, Takes Selfies!
Image: The Dalí Museum
Salvador Dalí once wrote, "If someday I may die, though it is unlikely, I hope the people in the cafes will say, ‘Dalí has died, but not entirely." Now 30 years after his alleged death, in a precedent-setting exhibition where a museum has used artificial intelligence-based techniques and TouchDesigner to bring an artist back to life, it turns out that Dalí's prescience was on point!
With the opening of the Dalí Lives exhibitionat the Dalí Museum in St. Petersburg, Florida, Dalí has quite authentically been resuscitated with a 'deepfake' where the man himself appears to banter pleasantly with museum visitors, taking selfies with people and sending them text messages!
Dalí Lives is unique in that it offers a new way to delight and inspire museum visitors and is a remarkable example of art meeting artificial intelligence and showcasing the possibilities created by this sometimes controversial technology.
We spoke to creative technologist Shan Jin of the San Francisco-based agency Goodby Silverstein & Partners (GS&P) who partnered with the museum to create the immersive exhibit. Shan Jin works at GS&P Labs, an internal innovation department at GS&P that experiments with emerging technology and collaborates with other creative departments to develop prototypes and fun experiences. Shan managed the Dalí Lives project and was responsible for the TouchDesigner development aspects of the implementation.
The Mau5head makes an appearance at Toronto rehearsal. Photo credit: Leah Sems
Less than 5 months before the premiere of his new Cube V3 show Canadian electronic music producer and DJ deadmau5 aka Joel Zimmerman paid a visit at the Derivative office in Toronto with the plan to build the entire show from scratch in a real time environment, specifically with TouchDesigner.
He had already invested solid – and very public time – learning the software in mind-boggling 18-to-48-hour work sessions on his Twitch.tv channel punctuated by cathartic bouts of gaming, his opinionated and adoring clan in tow. You could not look away… but also Zimmerman's self-reliance, agility and sheer stamina was perhaps what was most impressive as was his swift handling of TouchDesigner's infamous 'learning curve'.
Long story short, a few months and a sh*t ton of WORK later - "I put in the hours" he flatly states, emerged a 1.5 hour spectacle centred around a 21-foot-tall x 22-foot-wide, mechanical feat of engineering clad in pulsating, twisted visuals that singularly originates from him and turns E:DM on its head (so to speak ;). Whichever way you want to look at it, Cube V3 is something that hasn't been done before and it is remarkably good.
And as Zimmerman put it to the Miami New Times before the show's premiere at the Ultra Music Festival March 30th "I need everyone to know how much of a fucking insanely stupid technological feat this is… That's important, because we have all this tech and no one's fucking using it."
Before getting into that, one of the many take-aways from having seen this beast come together is that deadmau5 is a truly great role model and from a kind of 'disenfranchised' standpoint that makes it even cooler and a lot more real. Look no further. Massive thanks to the head mau5 for so publicly supporting TouchDesigner and for talking to us here.
Mar.14.19 TouchDesigner Drives Real-Time Digital Game-Mechanics in Foxconn Frequency
Foxconn Frequency (no.3) - For Three Visibly Chinese Performers is a work of "algorithmic theatre" that incorporates real-time game mechanics, piano pedagogy, 3D-printing, and the poetry of former Foxconn worker Xu Lizhi.
Three performers work with and against each other as they move through a series of testings and mini-games sourced from piano training. They succeed and fail in real-time, forced to correct their mistakes before they can continue. In each performance, 3D-printers output an object documenting the performer's competency over time.
Fully automated and generative, the work is different every performance, ranging from 50 minutes to 80 minutes: an experiment in digital game-mechanics as a way of meaning-making in the performing arts.
TouchDesigner was used in all aspects of the design and development process: to program all the game mechanics, the audio/visual feedback for the performers, and what the audience sees.
This is the last iteration in the Foxconn Frequency series from Hong Kong Exile, the Vancouver-based interdisciplinary arts company. We spoke with project lead Remy Siu who was responsible for all the TouchDesigner programing to learn more about the making of as well as the experience of staging and performing such an impressively original and ambitious work. Thank-you for talking to us Remy!
Live video feed of 3D printed objects from Foxconn Frequency premiere at the PuSh Festival 2018. Photo Credit: Sepehr Samimi + Daniel O'Shea
Foxconn Frequency premiere at the PuSh Festival 2018. Photo Credit: Sepehr Samimi + Daniel O'Shea
Feb.06.19 Prolific Artist Duo 404.zero Release Zerror a Control UI for TouchDesigner
Photo credit: Andreas Keller (http://www.keller-fotografie.de)
We caught up with artists and tool-makers Kirstina Karpysheva and Alexander Letcius of 404.zero who are also consummate voyagers and appreciably prefer to be absorbed doing these things rather than discussing them.
Despite a healthy reticence to the media spotlight, the duo within just a few years of joining forces, achieved bona fide "art-star status" for their signature painterly style of moody and opulent TouchDesigner generated visuals and modular audio.
The only thing impeding their creativity was a deep-seated frustration with the amount of time wasted on the boring task of optimizing patches and then "optimizing optimization" for every single project.
And so doing what they do admirably, 404.zero took matters into their own hands and created ZERROR a customizable controller UI component for TouchDesigner with multi-touch/midi/osc support. Hosting a comprehensive feature-set including drag and drop rigging, preset management and recall system, a timeline editor, multi-touch interface and the freedom to control, arrange and customize parameters Zerror is according to it's discerning makers "crazy comfortable, believe us. Make art now it's easy." With the release of Zerror v1.6 this week bringing even more features and refinements to the mix with the Pro version selling for the very affordable price of $50 it's a win/win scenario.
Dec.03.15 Christopher Bauder's Mesmerizing Kinetic Lights System Soars
2015 Vodafone - CeBIT, Hanover.
Brainchild of Christopher Bauder, the modular Kinetic Light System is an award-winning product embedding TouchDesigner which choreographs and moves not just light and pixels but actual physical objects, high above one’s head via steel wire and powered cable DMX winches to which large arrays of light elements are tethered.
It is an amazing system that has produced stunning results for a variety of installations both temporary and permanent while without fail catching people by surprise with its organic, mesmerizing movement.
In early 2015 TouchDesigner was brought into the Kinetic Lights ecosystem to provide an environment for show creation and to control movement and lights. We started a conversation with Christopher Bauder shortly thereafter which after almost 3 years to the day is at last ready to share! It’s a marathon of an interview which had to be put on hold several times due to Kinetic Lights’ success and the ensuing demand it ignited all over the world. We’d like to thank Mr. Bauder for spending so much time answering our questions and sharing his wealth of insight and experience and trust our readers will enjoy!
GRID performance with Robert Henke (Monolake) at the Fête des Lumières Light Art Festival 2013 in Lyon.
Derivative:Looking back through your succesion of work what stands out is a defiance of constraints - spatially, dimensionally, by medium, movement, forms, and certainly by scale and complexity! You don't like flat or 'box' anything! You design and produce complete systems that seem to live unhinged (well technically not quite!) in environments-at-large. (System defined as: (1.) a set of connected things or parts forming a complex whole, in particular; systems as (2.) a set of things working together as parts of a mechanism or an interconnecting network; and (3.) systems as a way of thinking that is based on a big idea?)
Christopher Bauder: Yes, I like that! Describes pretty well what I love and what I am after!
D:Can you tell us a bit about your background then, how you came to realize this and do what you do?
CB: I studied "Visual Communication" at the University of the Arts in Berlin, Germany. I was specialising in "media design" and the newly emerging "interaction design". It was all 2D screen design at the time and I always imagined that there should be something more to interact with than a screen, mouse and keyboard. Everything felt so flat and unemotional.
One semester we were given the task to use an everyday household item and extend it with a digital function. I chose an ordinary stepladder and used some sensors and midi connection to turn it into a musical instrument. When hardware and software finally came together and I triggered my samples on the computer, I was thrilled by the newly found option to hook up stuff to a computer as alternative input or output device. We even performed as a kind of band with several so called ToneLadders and toured extensively for two years all over Europe from media festival to media festival.
Since then I am always trying to question and reinvent the principles of how we interact with computers and how computer generated information can be displayed outside the computer in our physical world.
Sep.21.15 An Interactive Sonic Play-Space in a Maze of Netting
"When you walk in, you'll see a series of netting, mesh material… We want you to touch it. It's all very hands-on. There's six sections within this maze of netting that people can go up to each station. Within those stations my song has been, essentially, dissected and spread out across 44 speakers going around the rectangle of the space. So if you're in section one, and you are interacting with the netting and the material, the Microsoft Kinect cameras are essentially following your movements. Whatever you're doing with your arms and your hands, if you're pushing on the netting, the depth is being translated to data that goes into the master mainframe computer, which is running my song in real time on Ableton Live." Matthew Dear Discusses His Installation at the New Museum, Pitchfork
photo Drew Reynolds
photo Drew Reynolds
photo Drew Reynolds
DELQA is a collaboration between Matthew Dear and a talented Brooklyn-based creative team that puts visitors "inside Dear's music" with the ability to change and affect the composition by interacting with arrangements of netting in a gallery space. It is a playful and very clever interactive piece conceived of and produced in-part by Gabe Liberti and Dave Rife of Dave and Gabe, multimedia artists who design and build interactive sound and light installations.
The musical composition is multi-layered and dynamic such that the arrangement and sound synthesis can be transformed on the fly. In an interview with Microsoft, Dear explained he "wanted to compose a sonic dreamscape, fully realizing that dreams are weird and not always black and white. The music rests between peaceful and chaotic states, allowing the listener to come up with their own story while engaging with the piece."
We had the pleasure of meeting Gabe and Dave at a recent TouchIn NYC meetup organized by Kamil Narwatil of VolvoxLabs where Gabe presented on the project. Afterwards we had a chance to learn more about how the project came about and how the team combined skills in architecture, acoustics, interaction design, and music to create an experience that allowed the audience to be participants in this musical realization.
Gabe Liberti: It was the most immersive experience we've been able to create yet and leveraged all of our skills from spatial sound design to real-time interactive lighting. The crowd reaction was magical because they were listening, physically exploring, and also performing the music and visuals happening all around them. It was an opportunity for us to design and install a 44-channel spatial audio system that transformed the gallery into a living world of sound.
Last year we wrote about the brilliant multimedia dance performance PIÈCE À VIVRE (Living Room), a collaborative work between Moscow-based artists and new media producers Evgeniy Afonin and Yan Kalnberzin, who also co-founded the creative company Russian Visual Artists. We were understandably excited when Evgeniy contacted us to share his most recent project EXHIBIT ITEM/AWAKENING.
The contemporary dance performance strikes out in bold and innovative directions. Most notably, the entire production takes place on a semitransparent glass stage platform comprised of a complex lighting and video setup. LED strips and other lighting devices are embedded inside the stage under the glass which is projection-mapped from above. Finally "hydrogel", a substance resembling shaved ice or slushy snow covers the stage forming a medium upon which the performer - well... slides.
Almost all control, video, and LED content production and synchronization was achieved with TouchDesigner.
EXHIBIT ITEM/AWAKENING was recently awarded two prizes from the prestigious "Golden Mask Award", the National Theatre Award established in 1993 by the Theatre Union of the Russian Federation. The Golden Mask Award is the most prestigious award presented to productions in all genres of theatre art: drama, opera, ballet, modern dance, operetta, musical, and puppet theatre."
Receiving their awards: Evgeniy Afonin, Anna Abalikhina, Galina Solodovnikova, Yuriy Chulkov, Sergey Vasilyev
The first prize was awarded to the entire team for "Contemporary Dance/Best Production", and a remarkable achievement (in the traditional theatrical world where LED and projection mapping rarely make an appearance), Evgeniy and his partner Sergey Vasilyev received a second award for "Best Light Designer in Musical Theatre".
The play is now in the repertory of stately Alexandrinskiy Theatre in Saint Petersburg where the theatre staff handles the TouchDesigner-based lighting system via an interface Evgeniy designed specifically for this purpose.
It's a stunning work and a spectacular achievement! Evgeniy tells us about the making of the production below, thank-you Evgeniy!
Aug.19.15 Maotik's Light Bearers: 2,500 Fireflies Animate and Delight
"On a summer evening, the air is filled with the flash of fireflies calling to one another. Intrigued by their dialogue we created a poetic representation of their conversation.
Our immersive environment is a multmedia interpretation of a natural ecosystem that combines thousands of original crafted objects with state-of-the-art technology such as lasers. Visitors are invited to participate in the experience by creating their own audiovisual show inspired by the frail beauty of fireflies, they themselves become light bearers."
There can be no immunity to the surprise and delight experienced when suddenly in the darkness of a hot summer night, one encounters the glimmering pulses of fireflies in the tall grass. Mathieu Le Sourd, a.k.a. Maotik, together with multidisciplinary artist Etienne Paquette have designed Light Bearers, a very magical installation that simulates this wondrous experience indoors at Montreal's Insectarium.
Furthermore this swarm of 'fireflies', which are in fact 2,500 light bulbs filled with light-diffusing tonic water, can be controlled at one's whimsy, by activating an invisible hand-guided laser. "Point at different spots across the elegant glass field and Le Sourd's clever combination of Leap Motion sensors and TouchDesigner software will set them aglow like the lightning bugs that inspired them." we learn at The Creators Project.
Feb.24.15 Vincent Houzé on Rapid-Prototyping and Working in Real Time
My first steps with Arduino and a little proof of concept. Rotating the little white knob drives the cardboard box rotation and the virtual box rotation the rocks collide with. Graphics and sim generated with TouchDesigner. Mapping is a bit approximate, adhesive tape and 12$ servo can only take you so far.
Paper + pico projector + TouchDesigner
We first became aquainted with Vincent Houzé through the very compelling and often amusing videos he shares of his artistic and R&D experiments. Made with TouchDesigner, these short video proofs-of-concept for dynamic simulations, video mapping and GLSL shaders typically also incorporate paper, cardboard, glue and sometimes Arduino and rotors. They appreciably make their often complex point simply and elegantly with easy to find resources, showing an excellent aptitude for designing and prototyping with TouchDesigner.
Vincent currently lives in New York and works on large scale architectural installations and interactive installations. He is currently employed by award-winning company AV&C and also collaborates with local artists and designers like VolvoxLabs for example, where he brought his expertise in real time 3D graphics to shaping the Dubfire show.
Most recently we were intrigued by the teaser for Hybrid, a dance performance with immersive projections of motion and sound reactive visuals where Vincent designed and developed the visuals. We thought this would be an opportune time to catch up and have a conversation about Vincent's work.
Feb.16.15 Luminosity - Keith Lostracco's TouchDesigner Media Server
Blizzcon StarcraftII Stage Mapped
There exists an impressive and growing range of TouchDesigner-built tools and applications authored by the community to perform and operate live shows. The most recent and likely ambitious we've seen to date is Keith Lostracco's brand new TouchDesigner media server 'Luminosity'.
No stranger to taking on the building of dauntingly complex systems (see Keith Lostracco's Mindbending Fractal World "Escape Time"), Keith tells us Luminosity was built to create a front end for TouchDesigner aimed at providing operators with an easy-to-use interface to perform, control, and distribute realtime media while retaining the open-ended programmability of TouchDesigner as well as its speed.
Keith describes Luminosity's functionality as combining a high-end media server, a VJ program, and TouchDesigner. The same types of audio routing and distribution methods found in digital audio workstations and mixers were implemented here to process video and data as well as audio.
The initial idea - which was to facilitate the playing of clips (video, TouchDesigner Components and audio clips) in a similar way that popular VJ softwares or Ableton Live does, quickly evolved to taking on the functionality of high-end media server systems.
A staggering undertaking with spectacularly comprehensive results, Luminosity was debuted for V Squared Labs' Blizzcon Starcraft where 12 projectors 3d mapped the complex stage with full previs, and again at the Superbowl this weekend to manage a 24-projector ESPN event.
With a number of features pending, this tool is not quite 'finished' but it's a fair prediction that Luminosity will prove to be useful in many places and that it will continue to evolve.