RESOLVED: Floating point and non 8-bit Spout input/output

I have been writing a program which generates textures and shares them via Spout and have been testing it using TouchDesigner099 (Windows 7, 64-bit). It is mostly working but I thought I’d share some strange behavior.

When the SyphonSpoutIn node is set so the Pixel Format is “Use Input,” it seems to properly detect the input pixel format when (standard) 8-bit RGBA textures and 16-bit float textures are used. When sharing textures that are something else, it does not properly derive the input format but DOES properly work when manually setting the format.

For example, if I set my Spout interop texture format to DXGI_FORMAT_R16G16B16A16_FLOAT, TD is able to correctly determine that the pixel format should be “16-bit float RGBA.”

On the other hand, if I share a texture that is DXGI_FORMAT_R16G16B16A16_UNORM or DXGI_FORMAT_R32G32B32A32_FLOAT it does not correctly determine the texture format. It will work correctly if I manually set the Pixel Format to “16-bit fixed RGBA” and “32-bit float RGBA” respectively. I have verified proper operation in these conditions by passing in an image that has been multiplied by 1/255 (such that all pixels should not fit within 8-bit precision) and then, in Touch, multiplying the input texture by 255.

As a separate issue, TD SyphonSpoutOut only claims to like 8-bit fixed, 10-bit fixed and 16-bit float for output. This list can be expanded to support more formats including 32-bit float and 16-bit fixed using the DX format constants above. I have not tested with the 10-bit format.

In general, the list of supported texture formats for Spout in/out probably should be expanded to include all the appropriate candidates found in msdn.microsoft.com/en-us/librar … 73059.aspx

Rob

Unfortunately, DXGI cross-process texture sharing only allows for 3 formats, so it’s limited to those.

[url]Microsoft Learn: Build skills that open doors in your career

Specifically:

  • Only R10G10B10A2_UNORM, R16G16B16A16_FLOAT and R8G8B8A8_UNORM formats are allowed

Have you been able to share 32-bit float textures between other apps using Spout?

Weird, as I indicated in my OP, I have tested using the other formats I named and it works.

My verification process was to send an image multiplied by 1/255.0 and then, in Touch, multiply it by 255.0. For 8-bit texture, I get an expected bad picture… for a 16-bit float, everything works fine. For a 32-bit float or 16-bit fixed, it works but only if I explicitly TELL Touch the texture format.

The Spout receiver demo program correctly shows the DXGI format I designated.

To summarize… Documentation aside, it works.

Rob

Hi Rob,

sorry for delay in response here. I only monitor this forum from time to time.

Malcolm is right about the fundamental limitation for texture sharing that Microsoft document. Also there is the NVIDIA driver interop ([url]https://www.khronos.org/registry/OpenGL/extensions/NV/WGL_NV_DX_interop.txt[/url]) that does not document any formats at all. I was not satisfied with that so I tried every format possible by trial and error. This is a while back now but the results for 8 bit formats are documented in “SpoutSDK.h”. Final supported formats are documented in “SpoutGLDXinterop.cpp”.

However, “SpoutPanel” has a list of formats that it can report. One of these is untested, which means the others must have have been tested and this includes 32 bit float RGBA.

D3DFMT_A8R8G8B8
D3DFMT_X8R8G8B8
DXGI_FORMAT_R8G8B8A8_UNORM
DXGI_FORMAT_B8G8R8A8_UNORM
DXGI_FORMAT_B8G8R8AX_UNORM (untested)
DXGI_FORMAT_R32G32B32A32_FLOAT
DXGI_FORMAT_R16G16B16A16_FLOAT
DXGI_FORMAT_R16G16B16A16_SNORM
DXGI_FORMAT_R10G10B10A2_UNORM

So you might get lucky, but there is no guarantee that the format will work in all situations.

Lynn

Hi Lynn,

Thanks for the response.

I am guessing that the code in SyphonSpoutIn is doing something like this:

getSenderInfo(sendername, width, height, handle, dwFormat) switch (dwformat) { case DXGI_FORMAT_R10G10B10A2_UNORM: mypixelformat = 10bit_fixed_or_something_similar; break; case DXGI_FORMAT_R16G16B16A16_FLOAT: mypixelformat = 16bit_float; break; case DXGI_FORMAT_R8G8B8A8_UNORM: default: mypixelformat = 8bit_fixed; }

By observation, I have seen that other formats work and report back a valid DXGI_FORMAT but Touch hasn’t been coded to handle them. Your comments about the code in the SpoutPanel imply that it has been verified by someone else as well. My proposal is that Touch be extended to accept them. I am also proposing the SyphonSpoutOut node be extended to support at least 16-bit fixed and 32-bit float. My code uses spout.interop.SetDX11format(fmt) before creating the sender to configure my interop texture format.

Given that this report has been flagged as “resolved,” I’m assuming that the Derivative team has decided this change is risky. As a strong believer in the Robustness Principle (en.wikipedia.org/wiki/Robustness_principle) in systems design, this would seem like a reasonable change, at least for receive.

cheers,
Rob

I suppose there are limits based on whatever information there is available and without some experience it would be risky to enable anything else. I guess it is resolved because you can do what you want to do.

Nobody else but me has verified anything with SpoutPanel. Then only by trial and error. I don’t know whether these other formats would work reliably in practice but it is possible.

Have you found any other formats other than the ones I listed work OK? Apart from the testing with multiplication, does 32 bit float work in practice for your application? I imagine that other people might be interested in the possibilities.

Hey Lynn and Rob,

I’m inclined to keep following the specification. Even if other formats happen to work on some GPU/drivers, they aren’t guaranteed to work on another GPU, or even a different driver version on the same GPU it previously worked on. This doesn’t guide users towards building stable and reliable systems since a project they may can work on one system but not another, and it could be quite difficult to debug why for them.
I just did some tests on a Quardro P5000 and although I can sometimes get 32-bit float values across, other times the values are incorrect. So the spec is correct, in that other formats aren’t supported. I don’t really want to open this as a potential workflow since it’s just not stable.

In terms of the robustness principle, that seems like a double edged sword. If I follow that principle and allow other formats to come in, then the only software I’m supporting is ones that are explicitly ignoring the robustness principle. My interpretation of the robustness principle is that software should be programmed so that it can handle buggy or malicious data gracefully. It doesn’t tell me I should program to support software that is willingly ignoring specifications.

Thanks for your input on this!

Thanks, Malcom.

No point in debating this anymore but I am curious about your statement that 32-bit floating point textures on Quadro P5000 “sometimes” works and others doesn’t. Is there any guidance on what was different between the works and doesn’t work cases?

Rob

Not really, I got valid values in the receiving texture sometimes, other times the values were incorrect. All I was doing is changing the content of the source texture from one color to another.