Multi-Touch Examples

[greg]

I post these examples now so you don't need to re-work out the basics of gathering multi-touch events and preparing them in a format that can be used to click gadgets on 2D or 3D panels.

This file handles events from 5 device types, some better than others, including the Windows 7 standard interface using the Multi Touch In DAT, plus others over UDP or TCP/IP.

All of them convert the events into a standard table format that is used by a "drivepanel" script to control 2D panels, and a mechanism to drive 3D scenes using the Render Pick DAT.

... .toe file deleted, new version in next post.

[greg]

Here are two multi-touch examples for Build 15820 and later. One uses Panel components, the other is based on 3D renders with the Render Pick DAT.

The tab "2D Panel" is a set of 2D panel components that can be operated with this constraint: one finger per gadget. A gadget in this example is a Slider or Button (flag and radio buttons) component.

The tab "3D Render Pick" allows for multiple simultaneous fingers in one Container component whose background is the output of a Render TOP. This is much more open. The multi-touch table in /multitouch_env/multitouch is routed to a Render Pick DAT in /multitouch_env/scene/scene. The Render Pick DAT looks at the Render TOP and outputs a table containing which object each finger is touching, and where. The result drives expressions that color objects and turn on objects for display.

And with a DAT Execute DAT, you can run a script for each object your fingers are touching, with plenty of info at your disposal, like UV 3D texture coordinate, 3D depth, etc.

This example .toe file has only one Multi Touch In DAT, which presently can only drive one panel at a time. This is why this example passes a panel's path to the multitouch component, and it outputs u, v, x and y values for each finger relative to that panel.

Another approach would be to put a Multi Touch In DAT for every panel you want to be operated by touch interfaces.

The strip on the left simply exposes the choices of input devices. "virtual" would be a stream of vclick commands coming from another computer for example. Let me know if you want to see an example of that.

You would have to customize TouchOSC and mrmr for the specific OSC data coming from those iPad/iPhone programs.

These are basic features of multi-touch in 077, where you would have to add your own logic of gestures etc, but the fundamentals are there to build upon. Eventually Derivative will add in more automated embedded support for multi-touch.

[teflonjf]

Hi Greg,

So . . . I'm not able to make this scene work. I'm using a Win7 multitouch display, I'm able to drive the scenes with a mouse and select Win7 for the mtouch input. But touching the screen has no affect. Any ideas for what I'm missing ?

Jeff

[selina]

I won't be able to check the file with a multi touch display till monday, but Greg's example is not working for you at all? are you using just a single multitouch display? if you are using more than one display, is the multitouch your primary? if you put down a MultiTouch DAT and just touch the screen, are you getting any input in the DAT?

another thing to check is the Pen & Touch settings in windows. I think you might have to disable tablet or enable something.

I'm sorry I can't be of more help right now.

Selina

[teflonjf]

Hi Selina,

I see all the correct multi-touch data in the mtouchin DAT, the touch screen works fine within the Windows 7 environment, but the touch screen input has no affect on any of the UI components in Greg's scene. Or any other scene that I create using the vclick script. We are using the mtouchin DAT to hard code some custom buttons but would really like to be able to use some of the standard UI and TUIK components.

I am using just 1 display, I'll try switching a few things on and off in the pen and tablet settings.

Thanks,
Jeff

[rodberry]

HI Jeff,

are you trying this in perform mode?

from memory, i thing you need to open the Project1 panel rather than any of its children to get the touch stuff to work in a floating panel but I forget the details and won't be near a touch screen until Monday (sunday afternoon/evening your time I guess).

Rod.

[teflonjf]

Hi Rod, Selina,

OK, got it sorted. 2 issues, first was that the panel referenced in the vlcick script needed to be same pixel dimensions as the touch screen. Kind of a "duh" there, but assumed that there was some relative UV mapping happening. Second thing is that I've found UIG gadgets seem to like being touched better than TUIK's do . . .

Thanks,
Jeff

[rodberry]

great. I plan to get working with a touchscreen a bit more in the next few weeks. I'll put up some of my sketches as I go along.

Rod.

[selina]

glad you could get things working.