Login | Register 

088: Map an object with CamSchnappr

Share your components here. First one to 100 wins!

088: Map an object with CamSchnappr

Postby snaut » Wed Jan 09, 2013 3:01 pm

Note: The latest CamSchnappr moved to the palette starting with TouchDesigner version 088.50180. For older builds the last working version is attached at the bottom of this post.

CamSchnappr has its own wiki page.

CamSchnappr is a interactive mapping application entirely inspired by MAPAMOK, created by Kyle McDonald at the YCAM Interlab. With Kyle publishing the source code for MAPAMOK, we had a chance to look at it and convert it into TouchDesigner 088. For the original tool and source have a look here: https://github.com/YCAMInterlab/ProCamToolkit/wiki/mapamok-%28English%29

Kyle describes the significance of the tool as the use of openCVs cameraCalibrate to calibrate a projector via a model of the to be mapped structure instead of using a checkerboard. With TouchDesigner 088’s integration of Python, we were able to implement the same functionality into a standalone tool which lets you map complex structures in a small amount of time.


What it does:
If you have a physical 3D structure plus a virtual 3D model of that structure, you will be able to project a rendered virtual model onto that physical structure perfectly-aligned. You will be selecting points of your 3D Model and align them with the real world positions as you see them on the projector output.

Using your 6 or more alignment points, CamSchnappr will compute the position, rotation, scale and viewing angle of a specified TouchDesigner Camera component used for the rendering. Internally, an openCV function called calibrateCamera will be run and the “intrinsics and extrinsics” of the camera will be calculated and stored in the Camera component.

What you need:
  • TouchDesigner 088
  • a recent version of CamSchnappr
  • a projector
  • a physical structure to project on
  • a 3D Model of the to-be-mapped structure

What's in the package:
The attached zip file contains:
  • a recent version of CamSchnappr complete with an integrated test model and render setup
  • the documentation as pdf file
  • a cutout plan of the integrated model as pdf file

Open the latest CamSchnappr.toe, right-click on the CamSchnappr Component and select “View”:

basicInterface.jpg (5.72 KiB) Viewed 67456 times

The initial interface will show an interactive viewport with a default geometry and some input elements at the bottom.

The Viewport:
You can interact with the viewport as follows:

  • left-mouse click and drag to tumble
  • middle-mouse click and drag to zoom
  • right-mouse click and drag to pan
  • h to home the geometry
  • select points by left-mouse clicking on the blue spheres (blue sphere turns yellow, inactive selected points are orange)
  • make selected points active by left-mouse clicking on orange spheres
  • de-select points by right-mouse clicking onto spheres
  • tab to cycle through selected points

When you select points a little flag with the point number will be displayed:

selectedPointsViewport.jpg (6.13 KiB) Viewed 67456 times

Below the viewport are separate controls to specify
    an external Camera COMP
  • Initially an internal default camera will be used for the mapping, yet normally you would want to use your own Camera COMP. Be advised that all of your camera settings will be overwritten by the application.

    an external Render TOP
  • You will want to also drag in your own Render TOP to be used by the CamSchnappr. The application will change the resolution of the your Render TOP to the projectors output resolution.

    an external SOP
  • Drag your own SOP (surface operator) into this field. From testing on a decent machine good results were achieved with geometries of up to 200 points but a considerable slowdown when using the viewport occurred. Yet, if you minimize the number of points of the geometry you want to map, it all should be fine. In the end the result is saved in the camera and you can disable or delete the CamSchnappr Component entirely.

    an output Monitor selector
  • select the output Monitor number here. Depending on which Monitor you choose, the resolution of your specified render will be adjusted to match that resolution.
  • a button to enable the fullscreen Monitor output and Mapping Viewport.

The Mapping Viewport
Once you selected the correct output Monitor on the Monitor Selector, click the Fullscreen Output Button

mapperInterface.jpg (9.76 KiB) Viewed 67456 times

The interface is quite simple:
  • a pink crosshair indicates your current cursor position on the screen
  • a blue crosshair with number indicates an inactive point previously selected to be mapped in the main Viewport
  • a yellow crosshair with number indicates an active point previously selected to be mapped in the main viewport
The numbers beside each point correspond to the numbers of the selected points in the main viewport

You can drag points to their real-world position on the screen or use the arrow keys on the keyboard. For faster movement, use shift+arrowkey to move points.

Tab will cycle through the available points.

A faint outline will show the virtual model representation.

  • Create a simple render setup with the model of the object you want to map onto.
  • Right-click on CamSchnappr and select view.
  • Drag your Camera COMP onto the Camera COMP field at the bottom of the CamSchnappr Viewport.
  • Drag your Render TOP to onto the Render TOP field at the bottom of the CamSchnappr Viewport.
  • Drag the SOP containing the geometry of the to be mapped object onto the Geometry SOP field at the bottom of the CamSchnappr Viewport.
  • Select the number of the output your Projector is connected to via the Monitor selector at the bottom of the CamSchnappr Viewport.
  • Open the Mapping Viewport by clicking the open projection output button, if somehow the output now overlays your main viewport, just hit escape and select a different Monitor Output number via the Monitor selector at the bottom of the CamSchnappr Viewport.
  • In the main viewport align the geometry to the camera so it’s similar to what the projector sees.
  • Select a point in the main viewport and move it in the mapper viewport to the corresponding position on the real world object.
  • Repeat the last step for at least 6 points total.
  • After you have aligned 6 points, the camera should be calibrated and you should see the projection mapped onto the object.
  • If everything is correct you can disable the CamSchnappr or delete it. The Camera Calibration values are saved inside your Camera COMP as 2 Table DATs.
  • CamSchnappr can be run on multiple render setups making it possible to calibrate multiple projectors or multiple objects.
mapperInterfaceMapped.jpg (18.03 KiB) Viewed 67456 times
build with version 088.26200
(282.92 KiB) Downloaded 3466 times
User avatar
Posts: 904
Joined: Mon Nov 12, 2007 6:21 pm
Location: Toronto

Re: 088: Map an object with CamSchnappr

Postby Achim » Wed Jan 09, 2013 3:52 pm

Awesome. Thx markus!!!

Cologne area TD artists? Let’s get in touch

Posts: 1711
Joined: Wed Nov 14, 2007 1:52 pm
Location: Cologne

Re: 088: Map an object with CamSchnappr

Postby keithlostracco » Wed Jan 09, 2013 5:00 pm

Sweet!!! now we don't have to build our own. Thanks for the extremely useful component.

Posts: 875
Joined: Tue Dec 09, 2008 2:39 pm

Re: 088: Map an object with CamSchnappr

Postby DG2011 » Wed Jan 09, 2013 5:04 pm

YAY! it's like my dreams came true! :D
Posts: 31
Joined: Mon Nov 28, 2011 7:00 pm

Re: 088: Map an object with CamSchnappr

Postby elburz » Wed Jan 09, 2013 6:30 pm

Wow, very nice work! Markus strikes again!
Elburz Sorkhabi
Creative + Technology
nVoid Art-Tech Limited
Posts: 2017
Joined: Fri Jun 01, 2012 6:55 pm
Location: Toronto, Canada

Re: 088: Map an object with CamSchnappr

Postby heidrogen » Wed Jan 09, 2013 6:31 pm

this made my day, thanks for putting the work in!
Posts: 12
Joined: Wed Sep 19, 2012 9:54 pm

Re: 088: Map an object with CamSchnappr

Postby archo-p » Wed Jan 09, 2013 10:21 pm

just tested out the isam model with a virtual scenario, works like a charm, just had to reduce the map point model so it had only the points i'd consider using to keep the UI speed up

very excellent :)
User avatar
Posts: 469
Joined: Thu Nov 19, 2009 3:03 pm
Location: Los Angeles

Re: 088: Map an object with CamSchnappr

Postby puck » Wed Jan 09, 2013 10:49 pm

Legend! that takes the sting outta mapamok

Posts: 48
Joined: Thu Jun 28, 2012 1:52 am

Re: 088: Map an object with CamSchnappr

Postby matthewwachter » Thu Jan 10, 2013 12:04 am

Sweet! Great job man. This is amazing!
Posts: 296
Joined: Fri Jul 08, 2011 3:18 pm
Location: Los Angeles

Re: 088: Map an object with CamSchnappr

Postby depthcharge » Thu Jan 10, 2013 6:14 am

Thank you for using your amazing skills to help us lesser mortals Markus. Your awesome as usual.
Posts: 6
Joined: Fri Dec 17, 2010 12:04 pm


Return to Shared .tox Components

Who is online

Users browsing this forum: No registered users and 7 guests