088: Mapping Tool / Vert Pusher

Hi All,
Am making available a projection mapping tool I’ve developed and used in production for awhile, and recently dubbed “VertPusher”.

VertPusher is a single purpose tool for per-vertex mesh distortion, specifically for projection mapping applications. Having developed naturally over the course of my practice I use it constantly and feel that it may be useful to others besides myself, so just thought I’d share.

Recently rebuilt in the new Touch 088, the UI was revamped with direct inspiration (and code!) from the “CamSchnappr” tool. In fact the tool could be used as a companion to CamSchnappr, where after performing a camera calibration you still need to refine some points in the alignment.

This video shows the basic concept and workflow with the publicly available project (.TOE) and example geometry. It assumes you already have a properly imported mesh, UV’s and calibrated camera:

vimeo.com/75380915

The project files are available here:
VertPusher_Public.zip (30.7 KB)

Feedback or improvements are appreciated. I feel the video is self explanatory but if further documentation is helpful let me know.

Dev

1 Like

Amazing! I can’t wait to mess around later! Thank you.

Thanks for sharing, Dev, looks fantastic.

Dev, thanks for sharing this awesome tool.

There are some bugs that I’ve been running into.

After some time, the points move back to the original file.

The navigation keys ASDW will stop working after some time.

Any ideas ?

Thanks !

CPU

Hey CPU,
Thanks for giving it a try.

First question:

  • All point adjustments are made in a Table DAT in the project file, so make sure your’e saving frequently. (From personal experience I’ve never lost Table Info, knock on wood)
  • Make sure not to accidentally click the “Reset All Points” button in the “SetupTools” panel
  • Make sure a copy of the original project is not still open (for instance if you are doing a copy/paste into a new project)

Second question:

  • You are probably experience a loss of window focus by clicking somewhere else outside the panel control where the point mesh is displayed. Simply re-click in the canvas, or reselect any point. (See “gotcha” note in the Usage notes forthcoming for more info!)

Cheers,
Dev

VertPusher, Usage Notes
Here’s some additional workflow things for getting started with this project. Again feel free to hit me with any questions, and would love to know about any significant improvements or alterations.

PRE-FLIGHT:

  • Import your geo (make sure mesh has no transforms, for instance freeze transformations in Maya)

  • Import camera, or create one in Touch by hand or using CamSchnapper

  • Set prefs in Win node, i.e. set to full screen and your chosen monitor ID

  • Set the desired output resolution in both the panel and the render node:

    • Here “geoMapSetup5” (layout W & H)
    • And Here “geoMapSetup5/render01”
      (These two should be the same resolution, or at least should be the same aspect ratio. They could be expression linked, though in some cases you may want to render at a different resolution than the output screen size.)
  • Check for the correct path to your MAT texture on the rendered Geo here:

    • “geoMapSetup5/geo1”

MULTI SCREEN USAGE:
The supplied project is adequate for single channel scenario. Steps for a multi-channel setup are roughly:

  • Copy and paste the grouping of “Geomap” Container, UI panel component and Window component for as many window outputs desired. (See screen shot)

  • Keep numbering consistent between Panel Controls and Window, i.e. the last number should be the same: “PanelCtls(n)” and “outWin(n)”

  • Extend the size of the panel component “SetupTools_Panel” to accommodate additional controls for each new screen. Below is an example of the UI for a four channel setup:

GOTCHAS:

  • Only one major known issue with the UI: the keystroke commands will only be captured if your mouse focus is in the same panel containing the point/mesh output. In other words if you select a point with a mouse click and then click anywhere else outside (i.e. in the Touch UI, or on another screen output) that panel will lose focus and the keyboard input will not be captured by that component.

Workaround:
Simply reselect the point, or right click again anywhere within the canvas to return focus to that panel.

SUGGESTIONS:
While I share this with mostly altruistic intentions, here’s something majorly lacking that I would love some help with:

  • The most obvious way you would want to interact with points in this scenario would be to be able to simply click and drag the point to a new location with the mouse. In an early attempt I realized this is trickier than it sounds, as it requires a translation from screen space coordinates to 3D space. I haven’t tackled this yet, so of course anyone with expertise here would be greatly welcome.

Cheers,
Dev

Thanks for the additional info - Helps me alot.

I still have a hard time putting all this together. I’ve now got my brother doing the 3d modeling and animation… now I just need to figure out how to import it into TD/your patch ( I think he’s using maya and 3d max).

There are so many questions I have but i’ll spare your thread of the agony and poke around :slight_smile:

Cheers.

Very similar to our tool actually Dev, couple of thoughts:

  1. Click and drag, the way we do this at the moment isn’t via a DAT but actually a replicator comp with SOP clones…to add the drag we actually just run a script every frame (lol framerate) to do the same as what the keyboard does but using the panels u and v values, it’s not too tricky just needs a little bit of playing with…I’m actually going to switch ours over to your DAT technique as it’s a lot cleaner, these days however a lot of our production guys just throw the click and drag to get it kind of right then switch over to the keyboard controls.

  2. You should add a reset selected point button. This is one which our production team screamed for straight away…when you’ve got a mesh with more than 100 points things can get messy.

Hello, devharlan!

Thank you for sharing this incredible tool and congratulations. It is a perfect complement to CamnShnappr, wich would not be practical in a real world enviroment without it. Awesome.

Regarding the bug encountered by cyberpatrolunit, I have found what causes it, because it also happened to me.

From time to time, the wasd controls would stop working and all points would reset, forcing me to reload the project and with no aparent cause. The thing is that sometimes only one axis would stop working and the others would be fine. Once this kicked in, no mather what you did the controls would be dead until a reload.

This is what is happening:

-In project1/geoMapSetup5 there is a table called PointSums that holds the updated point possitions and feeds the DatTo SOP.
-The first row of this table holds indications for the names of the axis: P(0), P(1), etc. Without them, the DatTo SOP cannot read the table and the positions will not change.
-For some reason, sometimes the arrowKeyHandlers DAT will not take a valid point index and will send a 0 as index, causing the names of the first row of the PointSums table to change (for example, P(1) will change to 0.3P(1) or 0.3). When this happens, all points in that axis will seem to reset (because the SOP can’t read them anymore) and further modifications in that axis will not seem to have any effect. This can happen on one or all columns at the same time.
-So, while apparently the controls have stopped working, it is misleading to say so because they are fine; the python code is executing correctly and all tables are beeing updated just fine. The problem is that the DatTo SOP cannot read the point data correctly.

While I dont know why the python reads 0 as a point index in the first place, an easy solution would be to check wether the index is 0 or not, and if it is do nothing, thus avoiding unwanted modifications to the table.

I would do the modification myself but honestly I am a little green in python and maybe someone will find a more elegant solutions to the real cause (the 0 point index). Sorry about that, I wish I could be more helpful.

If anyone encounters this problem while working, just go into the PointSums table and rename the first row to their original values. This will give you all your modifications back (the points where never really reset in the first place) and all the controls will work fine.

I have a couple of sugestions as well:

First, it would be nice to make the project more self contained for easy importing, because as it is now it is necesary to keep it in the root project or the custom variables will stop working.

Second, I didnt really find useful taking the same camera as CamShnapper for visualisation. CamShnapper generates a custom matrix for the camera. When this matrix differs a lot from the normal camera (this happened to me probably because of the lens I was using), vertpusher will not be able to select any point at all. Im guessing there is some raytracing happening under the hood and the altered projection matrix affects it.

To solve it it I had to use a different camera with the standard projection matrix. I also found myself changing this camera’s angle a lot to see occluded points and select them, sometimes having to zoom in a lot (there where a lot of points in my model).

I thing using a camera interface similar to that of CamShnapper would be more useful and make more sense. If the user needs to reproduce a similar angle to the used by CamShnapper, it would be really easy to do so.

Hope this helps, and thank you again for this amazing contribution.

All the best =)

Hey Diego,
Thanks so much for sleuthing out that bug, hadn’t had time to look into it but it was a simple fix!

Have posted a revised build of VertPusher for public use with some other minor tweaks, attached here, enjoy:
VertPusher_Public_v6.4.rar (29.8 KB)

1 Like

FYI, Just re-posted the above to include the Geo sample files. If you’ve already downloaded please grab the new one. Cheers!

D

Hi,

Thanks for this Dev
giving a quick spin to this with 31900, the op “selectedPoints” which is referenced a few times in scripts is nowhere to be found :frowning:
Since I realize this is more than one year old I’ll try an older build of Touch now.

Best
Vincent

Oh I was trying arrow keys instead of AWSD, it actually works nicely.
Still wondering what the selectedPoints is for though, I’ll dig more a bit later.

I get some errors here running this on 099. The mapping/output seems to work with the example mesh though. Did not try to import my own models yet…

Made a quick update to this so that the viewer wireframes and such work if it’s not in the /project1 path.

If I get time in the next little bit, I’ll try to clear out all the remaining errors from it.
vertpusher.tox (33 KB)

Elburz,

Thanks for updating this!

I’m getting this error

Traceback (most recent call last):
File “vertpusher/geoMapSetup5/pointDisplayPicking/chopexec1”, line 14, in offToOn
AttributeError: ‘NoneType’ object has no attribute ‘numRows’

Any ideas?

I have an errors inside project1/geoMapSetup5/pointDisplayPicking/chopexec1. Version of TD 2020.22080. Can you update please?

The points/dots do not appear on the object in the render window. What am I missing?

Thanks

TD 2020.22080

Hi,
Guys thank you so much for this Public Tool It looks amazing but for some reason I get some errors

First its in PanelCtls1 on the operator datexec2

And the second one its in pointDisplayPicking

Would guys have an idea how to fix it.
I would need to use this tool for my next video mapping project.

Thank a lot in advance