Jun.13.19 Deepfake Salvador Dalí Interacts with Museum Visitors, Takes Selfies!
Image: The Dalí Museum
Salvador Dalí once wrote, "If someday I may die, though it is unlikely, I hope the people in the cafes will say, ‘Dalí has died, but not entirely." Now 30 years after his alleged death, in a precedent-setting exhibition where a museum has used artificial intelligence-based techniques and TouchDesigner to bring an artist back to life, it turns out that Dalí's prescience was on point!
With the opening of the Dalí Lives exhibitionat the Dalí Museum in St. Petersburg, Florida, Dalí has quite authentically been resuscitated with a 'deepfake' where the man himself appears to banter pleasantly with museum visitors, taking selfies with people and sending them text messages!
Dalí Lives is unique in that it offers a new way to delight and inspire museum visitors and is a remarkable example of art meeting artificial intelligence and showcasing the possibilities created by this sometimes controversial technology.
We spoke to creative technologist Shan Jin of the San Francisco-based agency Goodby Silverstein & Partners (GS&P) who partnered with the museum to create the immersive exhibit. Shan Jin works at GS&P Labs, an internal innovation department at GS&P that experiments with emerging technology and collaborates with other creative departments to develop prototypes and fun experiences. Shan managed the Dalí Lives project and was responsible for the TouchDesigner development aspects of the implementation.
The Mau5head makes an appearance at Toronto rehearsal. Photo credit: Leah Sems
Less than 5 months before the premiere of his new Cube V3 show Canadian electronic music producer and DJ deadmau5 aka Joel Zimmerman paid a visit at the Derivative office in Toronto with the plan to build the entire show from scratch in a real time environment, specifically with TouchDesigner.
He had already invested solid – and very public time – learning the software in mind-boggling 18-to-48-hour work sessions on his Twitch.tv channel punctuated by cathartic bouts of gaming, his opinionated and adoring clan in tow. You could not look away… but also Zimmerman's self-reliance, agility and sheer stamina was perhaps what was most impressive as was his swift handling of TouchDesigner's infamous 'learning curve'.
Long story short, a few months and a sh*t ton of WORK later - "I put in the hours" he flatly states, emerged a 1.5 hour spectacle centred around a 21-foot-tall x 22-foot-wide, mechanical feat of engineering clad in pulsating, twisted visuals that singularly originates from him and turns E:DM on its head (so to speak ;). Whichever way you want to look at it, Cube V3 is something that hasn't been done before and it is remarkably good.
And as Zimmerman put it to the Miami New Times before the show's premiere at the Ultra Music Festival March 30th "I need everyone to know how much of a fucking insanely stupid technological feat this is… That's important, because we have all this tech and no one's fucking using it."
Before getting into that, one of the many take-aways from having seen this beast come together is that deadmau5 is a truly great role model and from a kind of 'disenfranchised' standpoint that makes it even cooler and a lot more real. Look no further. Massive thanks to the head mau5 for so publicly supporting TouchDesigner and for talking to us here.
Mar.14.19 TouchDesigner Drives Real-Time Digital Game-Mechanics in Foxconn Frequency
Foxconn Frequency (no.3) - For Three Visibly Chinese Performers is a work of "algorithmic theatre" that incorporates real-time game mechanics, piano pedagogy, 3D-printing, and the poetry of former Foxconn worker Xu Lizhi.
Three performers work with and against each other as they move through a series of testings and mini-games sourced from piano training. They succeed and fail in real-time, forced to correct their mistakes before they can continue. In each performance, 3D-printers output an object documenting the performer's competency over time.
Fully automated and generative, the work is different every performance, ranging from 50 minutes to 80 minutes: an experiment in digital game-mechanics as a way of meaning-making in the performing arts.
TouchDesigner was used in all aspects of the design and development process: to program all the game mechanics, the audio/visual feedback for the performers, and what the audience sees.
This is the last iteration in the Foxconn Frequency series from Hong Kong Exile, the Vancouver-based interdisciplinary arts company. We spoke with project lead Remy Siu who was responsible for all the TouchDesigner programing to learn more about the making of as well as the experience of staging and performing such an impressively original and ambitious work. Thank-you for talking to us Remy!
Live video feed of 3D printed objects from Foxconn Frequency premiere at the PuSh Festival 2018. Photo Credit: Sepehr Samimi + Daniel O'Shea
Foxconn Frequency premiere at the PuSh Festival 2018. Photo Credit: Sepehr Samimi + Daniel O'Shea
Feb.06.19 Prolific Artist Duo 404.zero Release Zerror a Control UI for TouchDesigner
Photo credit: Andreas Keller (http://www.keller-fotografie.de)
We caught up with artists and tool-makers Kirstina Karpysheva and Alexander Letcius of 404.zero who are also consummate voyagers and appreciably prefer to be absorbed doing these things rather than discussing them.
Despite a healthy reticence to the media spotlight, the duo within just a few years of joining forces, achieved bona fide "art-star status" for their signature painterly style of moody and opulent TouchDesigner generated visuals and modular audio.
The only thing impeding their creativity was a deep-seated frustration with the amount of time wasted on the boring task of optimizing patches and then "optimizing optimization" for every single project.
And so doing what they do admirably, 404.zero took matters into their own hands and created ZERROR a customizable controller UI component for TouchDesigner with multi-touch/midi/osc support. Hosting a comprehensive feature-set including drag and drop rigging, preset management and recall system, a timeline editor, multi-touch interface and the freedom to control, arrange and customize parameters Zerror is according to it's discerning makers "crazy comfortable, believe us. Make art now it's easy." With the release of Zerror v1.6 this week bringing even more features and refinements to the mix with the Pro version selling for the very affordable price of $50 it's a win/win scenario.
Dec.08.09 TOUCHDESIGNER POWERS UP "CO2 CUBE" at SUMMIT in COPENHAGEN
How big is 1 metric ton of carbon dioxide (CO2)? A weighty question that can now be more readily addressed thanks to the ?CO2 CUBES - Visualize a Tonne of Change? project unveiled on the opening day of the UN Climate Summit in Copenhagen with a special presentation by San Francisco-based Obscura Digital and powered by TouchDesigner.
The CO2 Cube is a giant multimedia arts installation that aims to assist in visualizing what one metric tonne of carbon dioxide (CO2) stored at standard atmospheric pressure looks like which is the size of a three-story building, or, 8.2m x 8.2m x 8.2m (27ft x 27ft x 27ft). This staggering quantity is the amount of CO2 the average person in industrialized countries emit on a monthly basis, and in Canada and the US, every two weeks!
The project is intended to "bring together the ?open minds? of artists, architects and scientists through the universal language of art." said Mia Hanak, Executive Director of Millennium ART, creators of platforms for public engagement on themes tied to the Millennium Development Goals.
The cube will be used for projections showing CO2 emissions’ impact providing a real time stream of events that will be transmitted via YouTube for the duration of the conference. The YouTube “Raise Your Voice” channel simultaneously provides a forum for people to send in their thoughts and questions through videos that will also be projected on the cube in Copenhagen.
Dec.05.09 DERIVATIVE'S MARKUS HECKMANN PERFORMS WITH FALKO BROCKSIEPER
Markus Heckmann performed an impeccable visual set with Berlin's Falko Brocksieper at Break & Enter's steamy 2nd year birthday jam in Toronto last weekend. Click below for YouTube clip.
Dec.05.09 BRYANT PLACE in RESIDENCE
Bryant Place (also known as the visual artist CPU) was recently in residence at Derivative HQ to beef up his TouchDesigner skills, show us the ropes with the Jazzmutant Lemur, and be the first to try out Ableton's Max for Live to TouchDesigner via OSC - and all this in 2 weeks! He even found the time for a side trip to Miami where he performed for Heineken. Bryant is based in LA where he VJ's at the infamous Droid Behavior parties and has performed live visuals for artists including the Black Eyed Peas, HARD parties,and Carl Cox.
Update Dec.22.09 : Bryant recently shared these work-in-progress videos produced while working with the Derivative development team. The first provides a few examples and some discussion surrounding the Touch environment and Lemur data OSC integration and the second gives a brief overview of Ableton's Max for Live to TouchDesigner via OSC. Have a look, very informative.
Update Jan.18.10 : More visual spectacle from Bryant/CPU, this time with Richard Devine at the WHAM BAM THANK YOU AFTERHOURS presented by Friendly Integration, Electronic Creatives and Droid Behavior last Friday.
Scott Pagano continues to impress with TouchDesigner alongside Speedy J and the first 'real live' remixed performance of their Umfeld collaboration at the Hypersound festival in Madrid this November. (Hypersounds' mandate is to showcase artists that have "produced outstanding work in the field of research into sound".)
Scott's post-show feedback to the Derivative crew clearly relayed his excitement at being able to move towards an all-live show with TouchDesigner enabling. He described the performance at HyperSounds as remarkably smooth with a huge 5.1 sound and a solid projection/screen setup. "This was my first real system with usable audio analysis and I have to say it is such a joy to have this work so perfectly and smoothly. Jochem kept commenting on how "smooth" everything worked and looked in Touch."
Scott's next performance with TD will be a special event at Art Miami with Richard Devine on dec 1st. Stay tuned
Sep.30.09 SCOTT PAGANO at TED TALKS (Oxford) and SPEEDY J (Berlin)
new: Meeting of the Minds - Achim Kern interviews Scott Pagano and Jörg Unterberg on their experiences using TouchDesigner.
Scott Pagano, the mind behind the beautiful, rhythmically attuned visuals of Neither Field performed with TouchDesigner at the Absolut Late Nights event of TED Talks Global 2009 conference in Oxford, England in July. At TED, Scott performed alongside sound/video artist Barry Moon who built a max/msp audio/midi analysis system that feeds OSC to TouchDesigner. The jazz musician Eric Lewis gently ripped his piano to pieces.
On its heels July 30 he re-convened with Speedy J in Berlin's Berghain for the Native Instruments Maschine & Traktor event and further evolve their collaboration, this time propelled by the real-time abilities of TouchDesigner. His 3D video mixer was constructed and re-engineered with Jarrett Smith of Derivative.
"It's a series of nano robotic sculptural objects that animate in some really cool ways. running 1280x720 @ 60fps...." Full article at Meeting of the Minds.
Sep.24.09 OBSCURA DIGITAL'S CUELIGHT 'INTERACTIVE POOL TABLE'
Featured at the Hard Rock Hotel & Casino in Las Vegas and Esquire's SoHo "Ultimate Bachelor Pad".
Oct.02.09 DERIVATIVE at MEDIAN GALLERY, TORONTO NUIT BLANCHE
From October 3 to Sunday October 11, Toronto's Median Contemporary Gallery, (1142 Queen St West) shows 10 light and video-based art pieces, of which three are produced with TouchDesigner.
On October 3 for Nuit Blanche, Derivative's Markus Heckmann and Greg Hermanovic performed their individual brands of live visuals on the rooftop adjoining the Median gallery, a part of Median's all-night show/opening of light and video-based art in the gallery.
Among the three TouchDesigner gallery pieces, Markus Heckmann describes Datachart as "a collection of random datasets pressed into the shape of a recognizable pattern". Audio for Datachart is produced by Sherry Kennedy.
Also on show will be new work from the internationally acclaimed filmmaker, motion designer, and spatial reconstructionist Scott Pagano. The work on display will be part of the visuals Scott designed using TouchDesigner for his and Speedy J's Umfeld project.
Derivative's Rob Bairos will be unveiling the next-gen of his enfant terrible :) "Total Internal Reflection", the classic 1940's oscilloscope resuscitated from the depths of obscurity (Rob's parents' basement). Ingeniously interfaced with an iPod shuffle it displays imagery processed with TouchDesigner's new image to audio-signal conversion techniques to persuasively convey the eerie impression of being a little too observant and canny. The green-eyed relation of another mono-orbed computing rogue - you could say. These bad boys sell like hot-cakes so be sure to check it out now.
Jul.29.09 DIGITAL ENVIRONMENT DESIGN, Nuova Accademia di Belle Arti, ITALIA
TouchDesigner as used in education.
The final showing was TouchDesigner running on 2 PCs over 6 screens, synched via OSC (Open Sound Control).
May.01.09 GUITAR ANTI-HERO
Prototyping ideas with TouchDesigner - guitar-image at Eye Vapor
Sep.05.09 PARUL IO / DJ BLNDR
Toronto-based producer Parul Io recently spent some time at Derivative HQ preparing her special brand of visuals. Io blends the life of a DJ and VJ and plays a major part in the electronic and experimental music events in the city. She spins jungle, breakcore, minimal techno, and speedcore while complementing her performance with videos she's prepared earlier with Derivative?s experimental video mixer, Mixxa. She is pictured below getting ready for another show.
Parul arrived with a vhs deck and an armful of vhs tapes featuring a rare collection of source material that included bollywood phantasmagoria, early early animation classics, '70's schlock horror and a handful of other prize items found in the dollar bins of video stores across the country.
Parul's technique is to import VHS source material into Mixxa where she can layer footage into lush, intricate and fast moving compositions. She then alters frame rates, creates patterns, and applies effects as she mixes the video in realtime. The computer stores the material as quicktime movies, but Parul chooses to play them back in sequence onto VHS tapes. Why VHS? With a couple of copies Parul can avoid any unforeseen technical difficulties that can occur when VJing live. And there's something about the quality of those old VHS tapes that brings out the true grit in her hypnotic and shocking video mix.
Here's a sample of Parul's visual work using Mixxa. Click below for a clip on youtube.
Jun.10.09 JARRETT SMITH at the AVALON, Los Angeles
Jarrett Smith's Painter experimental application, for Damian Lazarus, Los Angeles, June 2009
May.12.09 RASTER-NOTON with DERIVATIVE'S TOUCHDESIGNER, TRANSMEDIALE and OFFF
In January of this year Derivative's Greg Hermanovic and Markus Heckmann spent an intensive week working with critically exalted German procurers of minimal photosonic waveforms raster-noton and the prolifically sonal Sheffield-based SND to produce the visual facet of their shows at Transmediale 09 in Berlin. The success of the CTM09 collaboration marked the beginning of a rare and significant relationship with raster and co. Lots more about this on the raster-noton page.
May.03.09 SHELLFISH AND ALFAMA WITH RICHIE HAWTIN AND ALI DEMIREL, Lisbon
The day following our arrival in Lisbon Richie Hawtin and Ali Demirel fly in from Berlin for the night to meet with Greg Hermanovic and Markus Heckmann to discuss previous and future collaborations. Lively conversation though jet-lag in effect. Heckmann demonstrates TD's evolution and development since the last Plastikman collaboration in 2004.
The Toronto crew is starving - our last 'meal' eaten more than 24 hours ago somewhere above the Atlantic. It's 6pm, not even close to dinner time in Lisbon but Rich makes a call to a local promoter who endorses a 'real portuguese' seafood resto - real as in authentically portuguese, family run, locally endorsed, fresh etc etc.
For the next couple of hours we sample every type of crustacean and shellfish in the place and possibly in existence including alien looking black barnacles that are a rare delicacy due to a harvesting process that's extremely dangerous - divers having to negotiate crashing waves and fierce undertow to pry these off of jagged treacherous rocks at some undisclosed location on the Portuguese coastline to the North. Richie loves the seafood!
Dinner and the conversation are... lively, enjoyable, and... substantial. After a couple of hours of steady consumption we decide a long walk is a really good and necessary idea and head towards Alfama, the kasba-like neighbourhood that is the oldest part of Lisbon and quite literally built on top of a rock (having for the most part survived the quake of 1755 buttressed by the rock its built on). Houses of Moorish design many of them clad in colourful, geometrically patterned tiles or azulejos (derived from the Arabic for 'small stones') spill down from the walls of the Castle of Saint Jorge.
We head up up up along a maze of narrow alley-like cobbled streets and meandering stairs with the intention of making it to the top for a view of the setting sun over the city. The area is mostly residential and like most of Lisbon the smell of fresh air-drying laundry is pervasive. Tucked between the houses are tiny cafes and bars, some quite well known for the traditional Portuguese Fado music.
Reaching the summit as the sun sinks beneath the horizon we find the castle gates slamming shut for the night but the view is spectacular and so we sit on the old walls looking out over the city before meandering back down the hill.
We stop at one of the cafes for water and some local brew and continue the earlier discussions. Ali pulls out his laptop and shows some of their latest work and ideas. The cafe is full of locals and young children. It doesn't feel like Sunday night. What appears to be the Portuguese version of American Idol blasts from an old TV. A caged bird chimes in and out of the general noise.
Richie and Ali have a 7am flight back to Berlin in the morning and the Toronto crew much work to do in preparation for the upcoming Raster marathon. We head back to the hotel full of shellfish and good feelings in agreement that this has been a very good day.