088: Resonate 2013 workshop example file


Read more about the workshop and the Resonate Festival here

The workshop was led by Dimitry Napolnov (Sila Sveta) with Barry Threw (Obscura Digital), and Markus Heckmann (Derivative) with support from Greg Hermanovic and Isabelle Rousset (Derivative).

For the workshop we prepared a file in TouchDesigner 088 (Resonate.toe) which contains 3 parts:

[]the Echo Nest component in /resonate/echonest which sends a music track for analysis to echonest.com and converts the detailed results to animation channels[/]
[]the section which creates the visuals for the mapping[/]
[]CamSchnappr to map the visuals onto the physical object[/]

The Echo Nest Component

Echo Nest describes itself on the website as: “…offer[ing] an incredible array of music data and services for developers to build amazing apps and experiences.” (Source: developer.echonest.com/)

Its developer API accepts music uploads for analysis and returns a detailed description of the track including time signature, key, tempo, timbre, pitch and sequenced information on beats, bars, sections and much more. The documentation of the analyzer can be found here: developer.echonest.com/docs/v4/_ … tation.pdf

To start using Echo Nest developers will have to acquire an API key: developer.echonest.com/account/register

The Echo Nest component comes with a simple user interface where the API key has to be entered and a URL to a music track has to be specified.

TouchDesigner will download the music file into temporary storage and start playback (this might take a while depending on the size).

Clicking the Fetch button on the Echo Nest component user interface will start the process of analyzing the track.

After a short while the analysis should be done and all tables in the Echo Nest component should be filled.

This is a multi-stage process:

[]First Echo Nest is told were the track is located via its Track API Method (developer.echonest.com/docs/v4/track.html#upload). Inside TouchDesigner this is done via a Web DAT (/resonate/echonest/webUpload), using the POST Method to Submit and Fetch the information to and from Echo Nest. The Web DAT is fed with the API key and the music file’s URL.[/]
[]TouchDesigner eventually receives a Track ID back from Echo Nest as part of a JSON formatted response also containing meta-information like Artist, Title and more. The Track ID is used in the next stage to receive a detailed audio analysis.[/]
[]Using the Track Profile API Method (developer.echonest.com/docs/v4/t … ml#profile) with the Web DATs Fetch Method (/resonate/echonest/profile), Echo Nest returns info on a Track given its ID. The information contains beside links to previews, images and more artist information, a URL to a complete analysis file. TouchDesigner parses the returned JSON for this URL and uses it in yet another Web DAT (/resonate/echonest/audio_summary) to fetch the complete analysis.[/]
[]This audio_summary is a JSON formatted package containing sequenced information on beats, bars, sections and more which is parsed in TouchDesigner with a python script (/resonate/echonest/decode) and its content is passed on to a collection of Table DATs[/]
[]The Table DATs are being fed into an Animation Component and via a Script DAT (/resonate/echonest/animation1/script1) converted into animation channels and keyframes.[/]
[]The output from the Animation Component now can be used in the synth as an animation channel.[/]

Echo Nest has a lot more to offer in regards to song analysis and track recognition, the documentation at developer.echonest.com/docs/v4/index.html has a good overview of what else is possible.

The Geometry and Animation

The sample installation works with a fairly simple setup of 7 cardboard boxes. The geometry for the 7 boxes is generated with 2 techniques:

[]Instancing, with the channels for the instances created in the instanceData Base Component (/resonate/instanceData) and[/]
[]Using the Copy SOP to create the object.[/]

A series of Render Pass TOPs are used to render the different visualizations. Besides showing various techniques on how to apply textures to geometry it is also shown how shadows are created with the Geometry components shadows and shadows1, the shadowLight and littleHelper Light components.

The component soundAnalysis (/resonate/soundAnalysis) explores one way on how to convert the audio waveform to via the Spectrum CHOP to meaningful animation data.

CamSchnappr

Eventually CamSchnappr is used to map the output onto the physical object.

The latest version of camSchnappr can be found in the Palette Usage directions on the Wiki.
resonate.9.toe (705 KB)

Is it possible to use it in 077 version ?! I opened it … gave API code and media Link but it didn’t Fetch :frowning:

Hi,

this file makes extensive use of python which is not available in 077. Required build is TouchDesigner 088 build 7400.

cheers
Markus

Fantastic Markus
Thanks so much for posting this.

Anyone know if the resonate workshop example can analyze music I have in a folder on my computer or does it need a url. I tried using a path like D:\folder\file.mp3, with no luck.

-Mark

great, thanks - this was a fun course in some techniques I wasn’t familiar with!

Is an internet connection always required for Echo Nest functionality in TD? i.e. for Perform mode with an installation venue without internet?

For all of you trying to manipulate this component… echoprint OTA is a bust! I spoke with an Echonest Python developer and basically OTA is no can do… they’d get sued unfortunately :frowning:

echoprint is still pretty cool and worth checking out, python peeps:
echoprint.me/

Hi all,

I get an error with this file.
I click okay but the window error doesn’t close.
Touchdesigner 10780 on windows seven.

Salvatore
toucherror.png

It works!
I was without an echonest API Key as you can read within the file:
“To use this component you first need to get a
echonest API Key:
developer.echonest.com/account/register

Salvatore

What are you guys using to upload your .MP3s to?

I did a little google, tried Picosong - but it didn’t work. I think it’s because it gave me a tiny URL and not a .mp3 URL?

Does this sound right?

So right now I’m looking for somewhere to upload my mp3s and recieve a URL in exchange.

Thanks!

Nothing?

I’m still stuck on the same issue.

Not a clue how to get a song in a URL .mp3 format… any advice?

Hi,

I just used my webspace for it. Not sure if you can use dropbox for it as well?

cheers
Markus

Has anyone successfully used the tox provided here?

When I attempt to map the realworld object using the camschnapper it doesn’t seem to make a difference. The same perspective/position is displayed (linked to cam1). I’m not sure I understand how camschnapper is to be used in this instance. I’m using F1, changing monitor/display to 1 (my projector) and then selecting the points and putting them in their realworld place. Even after 6 points nothing changes (I know with mapamok it would adjust after 6 points).

I tried the camschnapper tox and it worked fine - meaning the animation would update the perspective/position as I moved points. Which leads me to believe maybe this tox is broken?

So… after making some progress I’m realizing that when I align my projector with the camschnapper component, the wireframe is upside down?!? and maybe inside out?

Any explanation for this?

When I right click and ‘view camschnapper’ and then I set my 6 - 8 points, and slide 'wireframe: over to the right I can see the wireframe is on the ceiling - upside down from where my object is…

Anyone else having the same experience? I’m trying to figure out where/how this is happening - I think it’s within’ the camschnapper component - but I haven’t figured out where…

Any help is always appreciated.
Thanks.

Do you have a example file I can have a look at?

thx
Markus

See attached.
BCResonate2013.1.1.toe (551 KB)

Any ideas?

Can anyone confirm I’m the only one experiencing this? Or is it the patch?

I have learned a TON from this file…

one thing I cant get working tho is echonest… I have the API key… I enter it… but for some reason my audio in chop still gives me this error…

any thoughts on whats wrong in the network?