Jun.13.19 Deepfake Salvador Dalí Interacts with Museum Visitors, Takes Selfies!
Image: The Dalí Museum
Salvador Dalí once wrote, "If someday I may die, though it is unlikely, I hope the people in the cafes will say, ‘Dalí has died, but not entirely." Now 30 years after his alleged death, in a precedent-setting exhibition where a museum has used artificial intelligence-based techniques and TouchDesigner to bring an artist back to life, it turns out that Dalí's prescience was on point!
With the opening of the Dalí Lives exhibitionat the Dalí Museum in St. Petersburg, Florida, Dalí has quite authentically been resuscitated with a 'deepfake' where the man himself appears to banter pleasantly with museum visitors, taking selfies with people and sending them text messages!
Dalí Lives is unique in that it offers a new way to delight and inspire museum visitors and is a remarkable example of art meeting artificial intelligence and showcasing the possibilities created by this sometimes controversial technology.
We spoke to creative technologist Shan Jin of the San Francisco-based agency Goodby Silverstein & Partners (GS&P) who partnered with the museum to create the immersive exhibit. Shan Jin works at GS&P Labs, an internal innovation department at GS&P that experiments with emerging technology and collaborates with other creative departments to develop prototypes and fun experiences. Shan managed the Dalí Lives project and was responsible for the TouchDesigner development aspects of the implementation.
The Mau5head makes an appearance at Toronto rehearsal. Photo credit: Leah Sems
Less than 5 months before the premiere of his new Cube V3 show Canadian electronic music producer and DJ deadmau5 aka Joel Zimmerman paid a visit at the Derivative office in Toronto with the plan to build the entire show from scratch in a real time environment, specifically with TouchDesigner.
He had already invested solid – and very public time – learning the software in mind-boggling 18-to-48-hour work sessions on his Twitch.tv channel punctuated by cathartic bouts of gaming, his opinionated and adoring clan in tow. You could not look away… but also Zimmerman's self-reliance, agility and sheer stamina was perhaps what was most impressive as was his swift handling of TouchDesigner's infamous 'learning curve'.
Long story short, a few months and a sh*t ton of WORK later - "I put in the hours" he flatly states, emerged a 1.5 hour spectacle centred around a 21-foot-tall x 22-foot-wide, mechanical feat of engineering clad in pulsating, twisted visuals that singularly originates from him and turns E:DM on its head (so to speak ;). Whichever way you want to look at it, Cube V3 is something that hasn't been done before and it is remarkably good.
And as Zimmerman put it to the Miami New Times before the show's premiere at the Ultra Music Festival March 30th "I need everyone to know how much of a fucking insanely stupid technological feat this is… That's important, because we have all this tech and no one's fucking using it."
Before getting into that, one of the many take-aways from having seen this beast come together is that deadmau5 is a truly great role model and from a kind of 'disenfranchised' standpoint that makes it even cooler and a lot more real. Look no further. Massive thanks to the head mau5 for so publicly supporting TouchDesigner and for talking to us here.
Mar.14.19 TouchDesigner Drives Real-Time Digital Game-Mechanics in Foxconn Frequency
Foxconn Frequency (no.3) - For Three Visibly Chinese Performers is a work of "algorithmic theatre" that incorporates real-time game mechanics, piano pedagogy, 3D-printing, and the poetry of former Foxconn worker Xu Lizhi.
Three performers work with and against each other as they move through a series of testings and mini-games sourced from piano training. They succeed and fail in real-time, forced to correct their mistakes before they can continue. In each performance, 3D-printers output an object documenting the performer's competency over time.
Fully automated and generative, the work is different every performance, ranging from 50 minutes to 80 minutes: an experiment in digital game-mechanics as a way of meaning-making in the performing arts.
TouchDesigner was used in all aspects of the design and development process: to program all the game mechanics, the audio/visual feedback for the performers, and what the audience sees.
This is the last iteration in the Foxconn Frequency series from Hong Kong Exile, the Vancouver-based interdisciplinary arts company. We spoke with project lead Remy Siu who was responsible for all the TouchDesigner programing to learn more about the making of as well as the experience of staging and performing such an impressively original and ambitious work. Thank-you for talking to us Remy!
Live video feed of 3D printed objects from Foxconn Frequency premiere at the PuSh Festival 2018. Photo Credit: Sepehr Samimi + Daniel O'Shea
Foxconn Frequency premiere at the PuSh Festival 2018. Photo Credit: Sepehr Samimi + Daniel O'Shea
Feb.06.19 Prolific Artist Duo 404.zero Release Zerror a Control UI for TouchDesigner
Photo credit: Andreas Keller (http://www.keller-fotografie.de)
We caught up with artists and tool-makers Kirstina Karpysheva and Alexander Letcius of 404.zero who are also consummate voyagers and appreciably prefer to be absorbed doing these things rather than discussing them.
Despite a healthy reticence to the media spotlight, the duo within just a few years of joining forces, achieved bona fide "art-star status" for their signature painterly style of moody and opulent TouchDesigner generated visuals and modular audio.
The only thing impeding their creativity was a deep-seated frustration with the amount of time wasted on the boring task of optimizing patches and then "optimizing optimization" for every single project.
And so doing what they do admirably, 404.zero took matters into their own hands and created ZERROR a customizable controller UI component for TouchDesigner with multi-touch/midi/osc support. Hosting a comprehensive feature-set including drag and drop rigging, preset management and recall system, a timeline editor, multi-touch interface and the freedom to control, arrange and customize parameters Zerror is according to it's discerning makers "crazy comfortable, believe us. Make art now it's easy." With the release of Zerror v1.6 this week bringing even more features and refinements to the mix with the Pro version selling for the very affordable price of $50 it's a win/win scenario.
Dec.19.11 OBSCURA DIGITAL Shines 940,000 Lumens in the Desert and it is Spectacular
Earlier this month Obscura Digital used TouchDesigner to once again raise the bar of large scale architectural mapping and projection design, this time for the 40th Anniversary of the United Arab Emirates National Day Celebration at the Sheikh, Zayed Grand Mosque in Abu Dhabi.
The sheer scale of this production was mindboggling with 49 projectors used to project on a total surface measuring 19,474 square meters which included 4 minarets and 12 domes necessitating "detailed accuracy of 3-D mapping and content quality on incredibly complex geometry of flat walls with intricate hand carved details and arcades with rows of columns."
Interactive developer and artist Mary Franck who worked on the project for the duration had this to say:
"It was certainly a dream projection-mapping job: incredible architecture, all white marble, completely artistic and cultural content. It was absolutely an honor to be a part of it... continue reading>>
Nov.29.11 DIGITAL ENVIRONMENT DESIGN at NABA with TOUCHDESIGNER
We recently posted an article about an opera and a ballet being staged with 3D stereo-vision virtual sets for the first time. Andrew Quinn working with 3DLive Live was responsible for this landmark achievement and used TouchDesigner to develop the 3D technology employed in this way for the first time.
Andrew, one of the first TouchDesigner users, is based in Milan where he teaches a TouchDesigner course as part of a masters program in Digital Environment Design at NABA. The masters program was founded in 2007 by Paolo Atzori with the goal of providing a 360° view in the field of Interaction and Digital Environment Design and to evolve the technical and critical skills needed to create environments that integrate the digital scenarios of the information society in new representations of space characterised by dynamics of pervasiveness and interaction.
Andrew and teaching assistant graduate student Nima Gazestani have for some time now produced impressive live audio-reactive visuals to accompany classical music performances. They have also graduated proficient and inventive TouchDesigner users from the Digital Environment Design program over the past three years.
You can read about how TouchDesigner was used to make opera 3D here. This article focuses on the teaching and learning of TouchDesigner at NABA and the rather facinating work produced by students and faculty. What follows are interviews with Andrew, Nima and graduate students Carolina Travi and Elena Castellini who were very generous in sharing their work and experiences. We hope it's both valuable and inspiring to would-be TouchDesigner students and instructors.
If anything could possibly add to the experience of Frank Lloyd Wright's iconic Fallingwater, a house famously and defiantly situated in Bear Creek Run - right in the creek, that is - then this could be it.
It was a very nice surprise to discover this beautiful sound and light show from Chicago-based Luftwerk, produced for Fallingwater's recent 75th Anniversary Gala, September 17th. Luftwerk used TouchDesigner for this "rather challenging technical set-up" which included eight video projectors, six of of these mounted in two large maple trees and all running on one computer! According to the designers, the software worked flawlessly.
We spoke with Luftwerk founding partner Petra Bachmaier to learn a bit more...
Oct.10.11 Obscura Digital use TouchDesigner to make CONNECTIONS for FACEBOOK at F8
Obscura Digital used TouchDesigner to create a "physical, social, augmented reality experience dubbed "Connections" at F8, Facebook's developer's conference. Attendees swipe in to the experience using their RFID enabled event badge. Multiple overhead projectors map visuals to the floor and an array of 3D cameras are used to reliably track any number of people within the space.
Once "logged in" to Connections, a radial visualization, constructed from the user's social graph data, surrounds them creating a unique "fingerprint". Colored lines extend from the circles connecting people who share one or more of the observed metrics (mutual friends, interests, workplaces, schools, locations, birth sign, or non-English languages). When two or more people, who have mutual connections, stand within close proximity, a slideshow of mutual friends and interests appear between them.
Positioned behind the Connections space, a large screen shares aggregate data about the collective group- surfacing common interests and profiling the most connected of the group."
For the first time ever, an opera and a ballet performance using 3D stereo-vision virtual sets were staged at the Hungarian State Opera in September. The interest generated by, and the response to these performances were overwhelmingly positive, with 3D technology seen as having huge potential for a world typically fond of traditions.
The 3D effects in Bluebeard's Castle (by Andrew Quinn) worked beautifully. Indeed, arguably it is hard to stage Bartók's opera satisfactorily, as it depicts vivid pictures of the castle and its seven hidden rooms yet remains a psychological drama. In Komlósi's concept we get the best of both words: two people on stage trying to make sense of their relationship while 3D effects suggest what they may experience, although possibly only in their minds. - MusicalCriticism.com
Bartók's Bluebeard’s Castle in 3D, Hungarian State Opera, Budapest
Ravel's Boléro in 3D, Hungarian State Opera, Budapest
Andrew Quinn, one of the first TouchDesigner users, worked with the newly-formed company 3DLive Live and usedTouchDesigner to develop the 3D technology employed in this way for the first time.
Andrew is based in Milan where he teaches a TouchDesigner course as part of a masters program in Digital Environment Design at NABA. He and graduate Nima Gazestani have for some time now produced live audio-reactive visuals to accompany classical music performances. So if at first the notion of opera in 3D was a bit foreign, when we learned that Andrew and Nima were involved the concept gelled. Nima and Andrew were kind enough to speak with us about how these precedent-setting and masterfully designed productions came into being.
Jul.12.11 ON VISUALIZING AMON TOBIN'S ISAM with the V SQUARED TEAM
A Behind the Scenes Report on the Making of the Show Visuals and Delivery Systems by Isabelle Rousset
photo by Valerio Berdini - liveon35mm.com
Amon Tobin's show ISAM premiered at MUTEK June 1st to incredulity and awe from fans and the media alike. The general consensus from all camps was that this show was something new, futuristic, not experienced before; it exceeded all possible expectations and was very, very exciting. Suspension of disbelief had been achieved and audiences transported with it.
Our friends at V Squared Labs and Leviathan responsible for the show visuals "killed it" - that's pretty much the word.
The team made up of V Squared director Vello Virkhaus, designer and programmer Peter Sistrom and Leviathan chief scientist Matt Daly along with the help of Bryant Place built a custom application that runs the visuals of the entire Amon Tobin show end to end. The application is made in TouchDesigner, Derivative's visual programming environment, and does the projection mapping, video playback, Kinect response, realtime effects and more. How did they do that..?
The concensus on Amon Tobin's show at MUTEK last night was that it exceeded expectations which were very, very high and it's easy to see how. Our friends at V Squared Labs responsible for the visuals: "killed it" - that's the word today!
Watching the extended trailer released by the artist a few days ago gives some idea of scale and complexity of this production and what and who it took to pull it off.
The VSquared team built a custom TouchDesigner application that runs the visuals of the entire Amon Tobin show. The application, made in TouchDesigner’s visual programming environment, does the projection mapping, video playback, Kinect response, realtime effects and more.
We'll have an interview with V Squared director Vello Virkhaus, designer and chief programmer Peter Sistrom and Leviathan chief scientist Matt Daly to expand on the making of when the dust settles but in the interim here is some media coverage of the event.
May.26.11 RICHARD BURNS in the house making TOUCHDESIGNER TUTORIAL VIDEOS
Meet the talented and industrious Richard Burns from Newcastle upon Tyne. He is about to become instrumental in how TouchDesigner is learned.
Over the last two years we started to see some rather exceptional new work being made by students at Northumbria University in the UK where Program Leader Paul Goodfellow had introduced TouchDesigner into the University's Motion Graphics & Animation Design Department. We were impressed and delighted at the range and profusion of work - some finished projects, some works in progress and others performances of finished works - all produced with inventiveness, a certain polish, and at times a good deal of humour...
May.18.11 VSQUARED LABS produces AMON TOBIN 'ISAM'premiering at MUTEK!!
Very, very exciting, this video sneak preview of the Amon Tobin's 'ISAM' produced using TouchDesigner by our friends Vello Virkhaus and Co at VSqaured Labs just released today!! This has to be the most under-wraps music event of the summer and also one of the most anticipated.
So it's with some excitement and pleasure that we are finally able to share this video trailer for "Amon Tobin's all new, highly ambitious, breathtaking 'ISAM' live show. Tobin will be stepping away from previous DJ centric performances, and instead will be providing a large scale live audio/visual experience to select cities around the world. Designed in conjunction with Blasthaus, VSquared Labs, Vita Motus Design, Leviathan and others."
Watch this page for a full feature on the show and the making of as we'll be at MUTEK and talking with Vello and crew.
1st June - MUTEK, Montreal
9th June - Astra, Berlin
10th June - AB, Brussels
15th June - Melkweg, Amsterdam
17th June - Roundhouse, London
May.13.11 WILL ALVES, Content Programmer, IM COMMUNICATION
We asked Will Alves, content programmer at the young and highly energetic Swedish company IM Communication -specialists in largescale projection, reactive surfaces and interactive environments - about his experience using TouchDesigner for the first time. As it happened, this project, a tradeshow exhibit for the firm Mekonomenon, was also the first project for IM using TouchDesigner. From what Will had to say and the work they produced it looks like a very good fit!
Will, please tell us a bit about your background-- what you do and the tools you use.
I'm a Brazilian who lives and works in Sweden. I currently work with a company that creates and deploys interactive media solution. Before that I used to live in the USA where I worked as an enginner tech for a company named Teradyne Connection Systems. In Teradyne, I worked mostly as a quality assurance technician for circuit manufacturing.
Had you ever used TouchDesigner before?
I had never heard of TouchDesigner before I started working for my current company.
How did you find/discover it?
Our creative director has interest in the VJ and electronic music scene so he has some experience in Houdini, Pilgrim and some other similar software. He had known about TouchDesigner for a while but never really used it for his projects. Those other software packages were limited in the amount of outputs they could handle so we decided to take a leap of faith and use TouchDesigner to get the job done for one of our clients.
Tell us about some of the motivations and concepts driving what you do...
I'm lover of technology, gadgets, and interactive media. Instead of going to a show and wondering how people achieve the cool effects I just witnessed, I want to be the one creating and deploying it.
Can you describe your experience using TouchDesigner, the process? If and how it might differ from the tools you normally use?
Using TouchDesigner was quite the wake up call for me. I had been a long time since I did any kind of programming since my previous job didn't require it. But TouchDesigner's visual style of programming made it possible to build a network with as little code writing as I have ever seen. Pretty amazing. Learning how to use the program itself was very fun since you see the results as things change in real time. The learning curve was also amazingly small considering we used it to create a project in 6 weeks without any knowledge of how to use the programs before hand. The main lesson I learned is that TouchDesigner is easy to learn but difficult to master. The possibilities of what it can do is staggering, you have to be able to think outside the box to really use it to its full potential.
With your experience now how else do you see using TouchDesigner... things you see being able to do with it? Has it opened any doors?
I see us using TouchDesigner for many of our projects in the future, specially in conjunction with the Microsoft Kinect camera which has been a huge hit with the media industry. We currently work mostly with projection media and TouchDesigner will be one of our main tools to get the job done. Reliability is of key importance when creating media for an audience. It's not good for business when things crash in middle of a show, but TouchDesigner performed flawlessly running real time for 12 hours a day for several consecutive days in our first project with it, which is a testament to its stability.
Mar.18.11 OBSCURA DIGITAL Turns the SYDNEY OPERA HOUSE Inside Out & Outside In
Obscura Digital is at it again- wrapping the world's architectural modernist icons in unbelievable feats of moving imagery. Last October Obscura produced the exterior and interior projections for YouTube’s first Biennial awards, using TouchDesigner to seamlessly wrap Frank Lloyd Wright's 1959 Guggenheim Museum in NYC and on Sunday night they will be upping the ante at the Sydney Operahouse.
Visual Design and Exterior Projections by Obscura Digital. Western Sails, Sydney Opera House - YouTube Symphony Orchestra 2011.
We were recently blown away by Dev Harlan's Suffolk Deluxe Electric Bicycle, a collaboration with artist Olek which turns a men's road bike into a kinetic canvas for projected crocheted patterns and textures. The self-taught multidisciplinary artist, designer and CG director used TouchDesigner to mimic with an uncanny level of authenticity and much playfulness the effect of the bicycle continually being crocheted, unravelled and re-crocheted through projected images and the realtime evolution of the patterns.
Dev's first solo exhibition INFINITY LANDING at Rouge58, NYC till March 17, is a site-specific installation consisting of new works in sculpture, projection and light manipulation which according to the artist is "intended to be viewed in a "leisurely" fashion, whilst presenting optically challenging work at a larger than life scale."
Dev was kind enough to answer a few questions we had about his work and experience using TouchDesigner that we find both informative and inspiring and believe you will too.
The chamber itself is an 8 x 12 metre space walled by 10 large screens featuring 8.8.2 multi-channel sound as well as custom-applied
kinetic floor vibration units creating an immersive, self-contained mobile environment for audio-visual media works and performances.
How it works: The heart of CineChamber is TouchDesigner on one PC driving all 10 screens of CineChamber. Some content is generated within
TouchDesigner as realtime 3D mixed with video playback, image effects and compositing. Other content is generated on other systems and is
fed into two HD input cards and processed through TouchDesigner and applied to the 10 screens.