Login | Register 
FEATURES
APPLICATIONS
DOWNLOADS
EDUCATION
BLOG
WIKI
FORUM
STORE

Floating point precision between CHOPs

Report bugs so we can squash them!

Floating point precision between CHOPs

Postby keithlostracco » Sun Mar 19, 2017 3:18 pm

It looks like channels being passed from one CHOP to another are losing precision.

I have an expression CHOP connected to a math CHOP and then another math CHOP. When the values being fed from the expression CHOP got to be less than -1.0 e^5 then the following math CHOP would take the value as 0.

Running the same formula that the 3 CHOPs were doing in a single expression CHOP works fine:

1.0 / me.inputVal * parent().panel.screenh * .5 * me.inputs[1][0]

cheers
Keith
keithlostracco
 
Posts: 828
Joined: Tue Dec 09, 2008 2:39 pm

Re: Floating point precision between CHOPs

Postby malcolm » Mon Mar 20, 2017 3:10 pm

CHOPs currently only hold 32-bit floats, which Python and parameters all hold 64-bit floats (doubles). So it would make sense that you may get more precision in a single Python expression than in CHOPs.
User avatar
malcolm
Staff
 
Posts: 3763
Joined: Tue Nov 13, 2007 1:11 am


Return to Bugs

Who is online

Users browsing this forum: No registered users and 4 guests