Jun.13.19 Deepfake Salvador Dalí Interacts with Museum Visitors, Takes Selfies!
Image: The Dalí Museum
Salvador Dalí once wrote, "If someday I may die, though it is unlikely, I hope the people in the cafes will say, ‘Dalí has died, but not entirely." Now 30 years after his alleged death, in a precedent-setting exhibition where a museum has used artificial intelligence-based techniques and TouchDesigner to bring an artist back to life, it turns out that Dalí's prescience was on point!
With the opening of the Dalí Lives exhibitionat the Dalí Museum in St. Petersburg, Florida, Dalí has quite authentically been resuscitated with a 'deepfake' where the man himself appears to banter pleasantly with museum visitors, taking selfies with people and sending them text messages!
Dalí Lives is unique in that it offers a new way to delight and inspire museum visitors and is a remarkable example of art meeting artificial intelligence and showcasing the possibilities created by this sometimes controversial technology.
We spoke to creative technologist Shan Jin of the San Francisco-based agency Goodby Silverstein & Partners (GS&P) who partnered with the museum to create the immersive exhibit. Shan Jin works at GS&P Labs, an internal innovation department at GS&P that experiments with emerging technology and collaborates with other creative departments to develop prototypes and fun experiences. Shan managed the Dalí Lives project and was responsible for the TouchDesigner development aspects of the implementation.
The Mau5head makes an appearance at Toronto rehearsal. Photo credit: Leah Sems
Less than 5 months before the premiere of his new Cube V3 show Canadian electronic music producer and DJ deadmau5 aka Joel Zimmerman paid a visit at the Derivative office in Toronto with the plan to build the entire show from scratch in a real time environment, specifically with TouchDesigner.
He had already invested solid – and very public time – learning the software in mind-boggling 18-to-48-hour work sessions on his Twitch.tv channel punctuated by cathartic bouts of gaming, his opinionated and adoring clan in tow. You could not look away… but also Zimmerman's self-reliance, agility and sheer stamina was perhaps what was most impressive as was his swift handling of TouchDesigner's infamous 'learning curve'.
Long story short, a few months and a sh*t ton of WORK later - "I put in the hours" he flatly states, emerged a 1.5 hour spectacle centred around a 21-foot-tall x 22-foot-wide, mechanical feat of engineering clad in pulsating, twisted visuals that singularly originates from him and turns E:DM on its head (so to speak ;). Whichever way you want to look at it, Cube V3 is something that hasn't been done before and it is remarkably good.
And as Zimmerman put it to the Miami New Times before the show's premiere at the Ultra Music Festival March 30th "I need everyone to know how much of a fucking insanely stupid technological feat this is… That's important, because we have all this tech and no one's fucking using it."
Before getting into that, one of the many take-aways from having seen this beast come together is that deadmau5 is a truly great role model and from a kind of 'disenfranchised' standpoint that makes it even cooler and a lot more real. Look no further. Massive thanks to the head mau5 for so publicly supporting TouchDesigner and for talking to us here.
Mar.14.19 TouchDesigner Drives Real-Time Digital Game-Mechanics in Foxconn Frequency
Foxconn Frequency (no.3) - For Three Visibly Chinese Performers is a work of "algorithmic theatre" that incorporates real-time game mechanics, piano pedagogy, 3D-printing, and the poetry of former Foxconn worker Xu Lizhi.
Three performers work with and against each other as they move through a series of testings and mini-games sourced from piano training. They succeed and fail in real-time, forced to correct their mistakes before they can continue. In each performance, 3D-printers output an object documenting the performer's competency over time.
Fully automated and generative, the work is different every performance, ranging from 50 minutes to 80 minutes: an experiment in digital game-mechanics as a way of meaning-making in the performing arts.
TouchDesigner was used in all aspects of the design and development process: to program all the game mechanics, the audio/visual feedback for the performers, and what the audience sees.
This is the last iteration in the Foxconn Frequency series from Hong Kong Exile, the Vancouver-based interdisciplinary arts company. We spoke with project lead Remy Siu who was responsible for all the TouchDesigner programing to learn more about the making of as well as the experience of staging and performing such an impressively original and ambitious work. Thank-you for talking to us Remy!
Live video feed of 3D printed objects from Foxconn Frequency premiere at the PuSh Festival 2018. Photo Credit: Sepehr Samimi + Daniel O'Shea
Foxconn Frequency premiere at the PuSh Festival 2018. Photo Credit: Sepehr Samimi + Daniel O'Shea
Feb.06.19 Prolific Artist Duo 404.zero Release Zerror a Control UI for TouchDesigner
Photo credit: Andreas Keller (http://www.keller-fotografie.de)
We caught up with artists and tool-makers Kirstina Karpysheva and Alexander Letcius of 404.zero who are also consummate voyagers and appreciably prefer to be absorbed doing these things rather than discussing them.
Despite a healthy reticence to the media spotlight, the duo within just a few years of joining forces, achieved bona fide "art-star status" for their signature painterly style of moody and opulent TouchDesigner generated visuals and modular audio.
The only thing impeding their creativity was a deep-seated frustration with the amount of time wasted on the boring task of optimizing patches and then "optimizing optimization" for every single project.
And so doing what they do admirably, 404.zero took matters into their own hands and created ZERROR a customizable controller UI component for TouchDesigner with multi-touch/midi/osc support. Hosting a comprehensive feature-set including drag and drop rigging, preset management and recall system, a timeline editor, multi-touch interface and the freedom to control, arrange and customize parameters Zerror is according to it's discerning makers "crazy comfortable, believe us. Make art now it's easy." With the release of Zerror v1.6 this week bringing even more features and refinements to the mix with the Pro version selling for the very affordable price of $50 it's a win/win scenario.
the installation unidisplay offers an examination of semiotics and the laws of perception. the work operates with a number of modules of different visual effects that interfere with the viewers' perception, through optical illusion, jitter, flicker, after-image, movement, complementary colour effect, and so on. the installation unfolds against a long projection wall with two mirror walls on the side thus visually expanding like a universe. the basic visual, made up of sequences, motifs and graphic translations of various units of time measurement acts as a world clock and evokes the intertwining of time, between past, present and future. the installation is created with derivative's touchdesigner software which has been used for alva noto live performances with a triple-screen projection. cn
For the last four years we have had the unmatchable pleasure of working with carsten nicolai, a clear and discreet presence in the contemporary art world well known for his minimalist aesthetic and for his conceptual work. nicolai has produced an influential and critical body of work as visual and installation artist, in the realm of electronic music (as alva noto and as cofounder of the much respected raster noton label) and has published several design-based books with the prestigious publisher Gestalten.
The collaboration which has seen Derivative's Markus Heckmann work closely with the artist for a few weeks once or twice each year, began in 2009 when nicolai expressed the desire to model the visuals for live performances of unitxt on the TouchDesigner user interface itself. The resulting work has been refined, finessed and performed at an impressive list of festivals and venues around the world including: Ars Electronic (Linz and seen in the video below), Transmediale (Berlin), Mutek (Montreal), Sonar (Barcelona), Nemo (Paris), icograda world design congress (Beijing), and also in Athens, Moscow, Tokyo and London.
Nov.27.12 EYE VAPOR visualizes MICKEY HART'S "Brain on Drums"
Jeff Smith of Eye Vapor recently brought to our attention a project the studio has been working on in collaboration with legendary ex-Grateful Dead drummer Mickey Hart and Dr. Adam Gazzaley of the UCSF Neuroscience Imaging Center.
The research project is centered around the interplay of rhythm on brain function and the healing power of music and mental health, a life-long focus of Hart's. Eye Vapor became involved in the project to visualize this interplay and used TouchDesigner to produce real time visualization and sonification of brain activity with live data coming from an Emotiv headset.
Eye Vapor's TouchDesigner Performance Interface
Mickey Hart has long believed in the healing and regenerative power of music and asks the question: "What if in making music we were able to select and refine the rhythms, instruments and amplitude of that music to target and heal neural disorders?". “Breaking the rhythm code” as Mickey Hart describes this work, is his holy grail. Hart believes that knowing how rhythm affects the human brain will enable us to control and apply it medicinally, therapeutically and for diagnostic purposes. Hart believes this would allow us to reconnect synapses that are broken with Parkinson’s and alzheimer’s diseases. In short, the aim of this research and the ensuing visualization is to have medical science realise and embrace the healing powers of music - to bring “the power of rhythm to neuroscience” Hart says.
The research findings were revealed this past September during a live event at the AARP convention in New Orleans. UCSF cognitive neuroscientist Dr. Adam Gazzaley displayed the legendary drummer’s "brain on drums" as Hart's brain waves were visualized and projected onto a screen while he led the audience in a 1,000-piece drum circle.
"It all comes down to the vibrations and rhythm of things and how they interact, and now we have the real science," Hart says of his collaboration with Dr. Gazzaley. "Before, it was anecdotal. But every musician knows it works. When you get off the stage and your consciousness is elevated, there's a whole different kind of priorities in your body. We've never really been able to see the brain on music, this is a handshake between science and art."
We spoke to Eye Vapor about their involvement in this project but before continuing, watch the video above where Hart and Gazzaley give a demonstration of the system at AARP.
Aug.30.12 THR3HOLD Visualizes VITOR JOAQUIM'S Homage To GEOGRAPHY
"Sometimes a small event happens and it makes me open my eyes. (...) It's as if a light has been turned on. Suddenly there's something. Something that I know is present, that I must respond to.(…) It's like an adventure, but with no map." - Pina Bauch (quotation provided by Vitor Joaquim)
Geography is the name of a complex and beautiful new work by the sound and visual artist Vitor Joaquim slated for release this fall on the Kvitnu label. The 8-track CD is in many ways Joaquim's contemplation of author and professor of geography and physiology Jared Diamond's book Guns, Germs and Steel: The Fates of Human Societies. Diamond's chief question is: "Why Did Human History Unfold Differently On Different Continents For The Last 13,000 Years?" Through a layered and unfolding musical score Joaquim reflects upon the environmental factors and unanswered questions that have shaped patterns of human evolution with such variation. The journey unfolds in 8 tracks with titles like Cantino, Tordesilhas and Cargo that are all founded on the history that connects humankind and geography. But regardless as to the listener being informed or not about Diamond's work, Vitor Joaquim's Geography stands alone, fascinating and haunting.
We were doubly intrigued when we caught wind that Spanish collective Thr3hold were collaborating with Joaquim to produce live visuals for Geography performances. Thr3hold has intrigued us since coming on our radar with TouchDesigner-based work which has from the outset been somewhat unexpected - abstract, graphic, glitchy - even visually shocking at times, but always accomplished flawlessly.
So, when stills for the Geography visuals started popping up on the internet we got in touch to see what was brewing. Vitor and Thr3hold partners Maria Fernandez and Rodolfo Lillo were very gracioius and keen to have a long-distance conversation over the last few weeks about the many aspects of their work and collaboration. This article then serves a two-fold function: a look at Geography and the collaboration between Joaquim and Thr3hold, followed by an inerview with Thr3hold about their TouchDesigner work - past, present, and in development.
Geography Performance at Kino Šiška Centre for Urban Culture. Photo credit: An?e Kokalj
Aug.01.12 ARKAM'S LIVE VISUAL FEAT for GOOGLE I/O's AFTERPARTY
We recently wrote about TouchDesigner making a rather spectacular appearance at Google I/O 2012 in Bot & Dolly’s remarkable Kinetisphere, an interactive installation (with robots) designed to celebrate the launch of the Nexus Q.
We were doubly excited a few days later to get word from Mark Wells and Adam Jenkins of Arkam who as it turned out, used TouchDesigner to design, build and perform a spectacular, elaborate and hugely FUN 90 minute real-time interactive visual set for the Google I/O afterparty.
When Google came calling, Mark and Adam were excited by both the prospect and the context (Google I/O being all about awe-inspiring technology and hands-on experiences) and so challenged themselves to create something “beyond the realm of traditional animation.” What started off as a job producing a few minutes of traditional motion graphics work evolved into the delivery of real-time motion graphics that broke the 2-3 minute mold and could stay dynamic for hours if need be.
Naturally we were very impressed and had a few more questions - about the project, Mark and Adams backgrounds, their practice, motivations, working with TouchDesigner and so on, which Mark and Adam kindly answered. The following is an account of our conversation and of the fantastic visual experience Arkam created for the Google I/O afterparty.
Jul.15.12 TOUCHDESIGNER MUTEK WORKSHOP 2012 in 6 HOURS of VIDEO
The workshop was held in Montreal during MUTEK at SAT Transform and was designed for people who want to make TouchDesigner their professional production tool. It was led by Steve Mason of Obscura Digital along with Barry Threw - also from Obscura - and Greg Hermanovic and Markus Heckmann of Derivative. Needless to say there was a gargantuan wealth of knowledge, insight and practised experience shared over the course of the workshop that is now available to everyone.
The workshop's focus was on practicalities of production and participants worked along with the instructor replicating whatever was being demonstrated and produced. We have taken the content of these 2 days and edited 18 chronological videos to be watched and replicated in the same way.
In producing the final videos of the live sessions we've tried as much as possible to keep the content intact while 'tidying' up only the bits that seemed to really need editing. In a few cases (and never for too long) the audio volume may be a bit low with the presenter turning away from the recording source, or one of the other people in the room speaking far from the mic. Generally speaking, it's very clear.
For those of you about to tackle the videos there should be ample time to work along with them.
Watch the videos with HD on, if possible. Make sure you get to watch Steve Mason's "Demo of CineChamber Perspective Pre-viz" perspective techniques.
Date: Thursday May 31 & Friday June 1, 2012 (2 days) Location: SAT Transform Time: 10:30 am - 4:30 pm Title: Learning Derivative's TouchDesigner with Steve Mason (Obscura Digital) Instructors: Led by Steve Mason (Obscura Digital) with Barry Threw (Obscura Digital), Greg Hermanovic and Markus Heckmann (Derivative) Organizer: Isabelle Rousset (Derivative)
Abstract: Learn from some of its most accomplished and daring users how TouchDesigner, the real-time visual development platform driving Amon Tobin's ISAM, Plastikman Live and Cinechamber, can be used for creating interactive media systems, immersive/projection mapping environments, music visuals, and rapid-prototyping creative impulses. Master the TouchDesigner fundamentals with Steve Mason and Barry Threw of Obscura Digital and Greg Hermanovic and Markus Heckmann of Derivative.
Jul.17.12 TOUCHDESIGNER in the KINETISPHERE at Google i/O 2012
Kinetisphere, an interactive installation designed and built by San Francisco-based creative engineering studio, Bot & Dolly to celebrate the launch of Google’s Nexus Q seemed to steal the show at the recent Google I/O 2012.
Based on what we've seen it certainly is memorable: An industrial Kuka robot deftly articulating an 8-foot, 300-pound diameter fiberglass replica of the Nexus Q with a visualizer ring of 6 mm pitch LEDs controlled by a signal coming out of TouchDesigner that’s reacting to the music as well.
Jul.05.12 What WHITE KANGA is building with TOUCHDESIGNER to alter our sense of REALITY
We were recently taken by surprise by something rather out of the ordinary that challenged our most liberal interpretation of what can be categorized as ‘real’ and certainly as ‘real-time’. The projected image seemed not to be sitting on the surface of the object but rather to be embedded into it. A formation of suspended blocks, bigger than a breadbox and smaller than Amon Tobin's ISAM, moving in space with a surface movement sticking to it giving the impression of a single unified artifact that couldn't really exist by definition of what is known. But there it was.
A little investigation led us to White Kanga, a talented crew from Poland who have expertly been building their own custom applications with TouchDesigner to pull off the otherworldly and it could be said unprecedented fabrication seen here.
Following the video below which illustrates what we’re talking about, is an interesting and highly inspiring interview with White Kanga that details their experience working with TouchDesigner from a seasoned VFX background (Houdini pros) and the making of the Modeling Projection System (MPS) project.
Derivative:With the MPS you’ve produced a very ‘advanced’ body of work in a very specific direction and it seems to have come together really quickly. Can you tell us what the process of learning and adopting TouchDesigner has been from the standpoint of VFX designers with Houdini expertise.
White Kanga: Rafal Bielski has been a Houdini user since 2008 when he was looking for tools to help realize complicated tasks in the Polish animation "Switez" at Human Ark Studios where he was employed as pipeline engeener and CG supervisor. Arek Rekita on the other hand is all about procedural solutions so Houdini is his natural habitat. It’s where he developed techniques for J.C. Avatar's flora.
So our decision to use TouchDesigner was made because of similarities between Houdini and TouchDesigner’s familiar philosophy.
Rafal B was first to be hooked by TouchDesigner and used it to produce a prototype model of an interactive music game-with full working game play, 3D visuals and sound.
April.09.12 FROM RUSSIA with LUMINOUS INTENSITY, THE SILA SVETA SHOW
SILA SVETA, which literally translates as 'Luminous Intensity', is a Moscow-based visual label who have been casting some very bright lights and producing jaw-droppingly ambitious projection mapping events this last year using TouchDesigner. Its principles are Alex Rozov (founder and engine), Alexander Us (founder and directing), Dmitry Napolnov (tech lead) and Masha Roslavskaya (production manager).
Self-taught and deicidedly self-motivated, Sila Sveta was formed in 2008 with the idea, as Alexander Us explains, "to create astonishing things which make people a little bit happier."
"In Russia and in Moscow almost 6 months is bad weather.” Alexander continues “We grew up in residential neighbourhoods, surrounded by high rise building blocks. They are all horrible grey colour and since childhood we dreamt of turning them into canvases for painting. What a joy it was to light up one of those with a slide projector. That was the beginning of it all"
"Another motivation is to point out that Russia doesn’t only have oil and gas stations. It has young individuals who want and can create original and cool things, developing and participating the international community.”
Sila Sveta team: Alexander Us, Dmitry Napolnov, Masha Roslavskaya and Alex Rozov
Inspiring, impressive and also very true as the following interview conducted long distance and between projects with the very busy crew makes clear. We are grateful not only for their taking the time to talk to us, but also for the enthusiasm and community spirit that comes across lound and clear.
"The Viewing of Six New Works is a light projection composition derived from the essentialized movements of eyes and head, that a possible person might make in looking at a rectangular object on the wall (i.e., a "painting", a "photograph"). Each hypothetical wall rectangle is perceived differently. This is shown by the different "personal" gestures involved in the revealing of the rectangle. When attention is not being paid to it the object/rectangle is not there.
The work is an attempt to present only the movements of perception, not perception itself. The art of looking." - Michael Snow
Canadian filmmaker-artist Michael Snow came to the Derivative studio a few months ago to see Greg Hermanovic about a way to realise a piece he had in mind for an upcoming solo show in New York.