how to do touchdesigner -> website -> table dat

I’m the creator of PoseNet, which does human pose estimation in the browser the browser using any rgb image:
[url]https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5[/url]

I want to be able to use this with TouchDesigner, and have Touch send images to a web browser page (in chromium), have the webpage estimate poses with the image and PoseNet, and receive human poses back from the webpage from each image it sends. These keypoint positions would be put into a table dat.

The page can accept images in utf8 format to estimate poses on.

What are some ideal ways to do this?

I’m running on a Mac, so the Web Render top is not supported.
I was thinking of starting up a flask server within touchdesigner that has a websocket.
A static website would have to be open and connect to this websocket server and receive images in utf8 format from the flask server, estimate poses, then send them back via websocket to the flask server, once the flask server receives a pose, it updates a table dat with the keypoint positions.

What are people’s thoughts on this approach?
Has someone done something like this similarly before?
What’s the right way to spin up a long running process like a flask server in touch and have it receive commands and then update a table dat after? (I’m relatively new to touch)

1 Like