Login | Register 
FEATURES
APPLICATIONS
DOWNLOADS
EDUCATION
BLOG
WIKI
FORUM
STORE

088: Resonate 2013 workshop example file

Share your components here. First one to 100 wins!

088: Resonate 2013 workshop example file

Postby snaut » Wed Apr 17, 2013 1:01 pm

Image
Read more about the workshop and the Resonate Festival here

The workshop was led by Dimitry Napolnov (Sila Sveta) with Barry Threw (Obscura Digital), and Markus Heckmann (Derivative) with support from Greg Hermanovic and Isabelle Rousset (Derivative).

For the workshop we prepared a file in TouchDesigner 088 (Resonate.toe) which contains 3 parts:
  1. the Echo Nest component in /resonate/echonest which sends a music track for analysis to echonest.com and converts the detailed results to animation channels
  2. the section which creates the visuals for the mapping
  3. CamSchnappr to map the visuals onto the physical object

The Echo Nest Component

Echo Nest describes itself on the website as: “…offer[ing] an incredible array of music data and services for developers to build amazing apps and experiences.” (Source: http://developer.echonest.com/)

Its developer API accepts music uploads for analysis and returns a detailed description of the track including time signature, key, tempo, timbre, pitch and sequenced information on beats, bars, sections and much more. The documentation of the analyzer can be found here: http://developer.echonest.com/docs/v4/_ ... tation.pdf

To start using Echo Nest developers will have to acquire an API key: https://developer.echonest.com/account/register

The Echo Nest component comes with a simple user interface where the API key has to be entered and a URL to a music track has to be specified.

TouchDesigner will download the music file into temporary storage and start playback (this might take a while depending on the size).

Clicking the Fetch button on the Echo Nest component user interface will start the process of analyzing the track.

After a short while the analysis should be done and all tables in the Echo Nest component should be filled.

This is a multi-stage process:

  • First Echo Nest is told were the track is located via its Track API Method (http://developer.echonest.com/docs/v4/track.html#upload). Inside TouchDesigner this is done via a Web DAT (/resonate/echonest/webUpload), using the POST Method to Submit and Fetch the information to and from Echo Nest. The Web DAT is fed with the API key and the music file's URL.
  • TouchDesigner eventually receives a Track ID back from Echo Nest as part of a JSON formatted response also containing meta-information like Artist, Title and more. The Track ID is used in the next stage to receive a detailed audio analysis.
  • Using the Track Profile API Method (http://developer.echonest.com/docs/v4/t ... ml#profile) with the Web DATs Fetch Method (/resonate/echonest/profile), Echo Nest returns info on a Track given its ID. The information contains beside links to previews, images and more artist information, a URL to a complete analysis file. TouchDesigner parses the returned JSON for this URL and uses it in yet another Web DAT (/resonate/echonest/audio_summary) to fetch the complete analysis.
  • This audio_summary is a JSON formatted package containing sequenced information on beats, bars, sections and more which is parsed in TouchDesigner with a python script (/resonate/echonest/decode) and its content is passed on to a collection of Table DATs
  • The Table DATs are being fed into an Animation Component and via a Script DAT (/resonate/echonest/animation1/script1) converted into animation channels and keyframes.
  • The output from the Animation Component now can be used in the synth as an animation channel.

Echo Nest has a lot more to offer in regards to song analysis and track recognition, the documentation at http://developer.echonest.com/docs/v4/index.html has a good overview of what else is possible.

The Geometry and Animation

The sample installation works with a fairly simple setup of 7 cardboard boxes. The geometry for the 7 boxes is generated with 2 techniques:

  • Instancing, with the channels for the instances created in the instanceData Base Component (/resonate/instanceData) and
  • Using the Copy SOP to create the object.

A series of Render Pass TOPs are used to render the different visualizations. Besides showing various techniques on how to apply textures to geometry it is also shown how shadows are created with the Geometry components shadows and shadows1, the shadowLight and littleHelper Light components.

The component soundAnalysis (/resonate/soundAnalysis) explores one way on how to convert the audio waveform to via the Spectrum CHOP to meaningful animation data.

CamSchnappr

Eventually CamSchnappr is used to map the output onto the physical object.

The latest version of camSchnappr can be found in the Palette Usage directions on the Wiki.
Attachments
resonate.9.toe
build with ver. 088.52320
(705.43 KiB) Downloaded 754 times
User avatar
snaut
Staff
 
Posts: 795
Joined: Mon Nov 12, 2007 6:21 pm
Location: Toronto

Re: 088: Resonate 2013 workshop example file

Postby alless » Thu Apr 18, 2013 7:03 am

Is it possible to use it in 077 version ?! I opened it ... gave API code and media Link but it didn't Fetch :(
my A/V Portfolio Blog :
alllesss.wordpress.com
User avatar
alless
 
Posts: 25
Joined: Wed Jan 30, 2013 4:57 pm

Re: 088: Resonate 2013 workshop example file

Postby snaut » Thu Apr 18, 2013 12:00 pm

Hi,

this file makes extensive use of python which is not available in 077. Required build is TouchDesigner 088 build 7400.

cheers
Markus
User avatar
snaut
Staff
 
Posts: 795
Joined: Mon Nov 12, 2007 6:21 pm
Location: Toronto

Re: 088: Resonate 2013 workshop example file

Postby rurik » Wed Apr 24, 2013 5:07 pm

Fantastic Markus
Thanks so much for posting this.
---
Touch Designer Pro 088
Windows 7 - 8 Core- 8 gigs RAM - Nvidia GTX 560
rurik
 
Posts: 258
Joined: Thu Dec 29, 2011 1:04 am

Re: 088: Resonate 2013 workshop example file

Postby Lebowski3cx » Wed May 01, 2013 4:04 pm

Anyone know if the resonate workshop example can analyze music I have in a folder on my computer or does it need a url. I tried using a path like D:\folder\file.mp3, with no luck.

-Mark
Lebowski3cx
 
Posts: 15
Joined: Mon May 14, 2012 10:51 pm

Re: 088: Resonate 2013 workshop example file

Postby delray » Wed May 08, 2013 5:07 pm

great, thanks - this was a fun course in some techniques I wasn't familiar with!
User avatar
delray
 
Posts: 139
Joined: Thu Jun 30, 2011 9:38 am
Location: Montreal, QC

Re: 088: Resonate 2013 workshop example file

Postby jonathank » Sat Jul 27, 2013 2:35 pm

Is an internet connection always required for Echo Nest functionality in TD? i.e. for Perform mode with an installation venue without internet?
User avatar
jonathank
 
Posts: 174
Joined: Thu Apr 25, 2013 5:27 pm

Re: 088: Resonate 2013 workshop example file

Postby DG2011 » Tue Aug 13, 2013 3:04 pm

For all of you trying to manipulate this component... echoprint OTA is a bust! I spoke with an Echonest Python developer and basically OTA is no can do... they'd get sued unfortunately :(

echoprint is still pretty cool and worth checking out, python peeps:
http://echoprint.me/
DG2011
 
Posts: 31
Joined: Mon Nov 28, 2011 7:00 pm

Re: 088: Resonate 2013 workshop example file

Postby armonico » Fri Oct 25, 2013 10:04 am

Hi all,

I get an error with this file.
I click okay but the window error doesn't close.
Touchdesigner 10780 on windows seven.

Salvatore
Attachments
toucherror.png
toucherror.png (19.16 KiB) Viewed 16147 times
armonico
 
Posts: 45
Joined: Mon Feb 25, 2013 2:34 am

Re: 088: Resonate 2013 workshop example file

Postby armonico » Fri Oct 25, 2013 10:58 am

It works!
I was without an echonest API Key as you can read within the file:
"To use this component you first need to get a
echonest API Key:
https://developer.echonest.com/account/register"

Salvatore
armonico
 
Posts: 45
Joined: Mon Feb 25, 2013 2:34 am

Next

Return to Shared .tox Components

Who is online

Users browsing this forum: No registered users and 3 guests