Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unexpected rounding for int/float division #863

Open
machinex7 opened this issue Sep 17, 2023 · 1 comment
Open

unexpected rounding for int/float division #863

machinex7 opened this issue Sep 17, 2023 · 1 comment

Comments

@machinex7
Copy link

When I evaluate the following expression, I get a result of "-0"
{ -1.0 * 5 / 1000 }

But when I instead simplify the first multiplication myself, I get the expected result of "-0.005"
{ -5.0 / 1000 }

As a workaround I figured out I can do the first multiplication and store the result in a temp variable, and then use that for the subsequent calculation, but it would be nice if it handled the math for me. Seems like it could introduce other mathematical bugs down the road. I'm using Inky version: 0.14.1

Thanks!

@JacobChrist
Copy link

Can you add parentheses?

{ (-1.0 * 5) / 1000 }

Don't know if this will work but it may be worth a try.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants