New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Floating point incrementation erratic behaviour #17
Comments
Interesting. The same occurs on OSX 10.9 too. @FichteFoll any ideas? |
Windows7, ST2220: >>> .1
0.10000000000000001 Windows7, ST3062: >>> .1
0.1 It's a precision issue with floating point numbers. Not 100% sure whether we should handle this, but if we were to, I'd probably specify some default maximum precision argument in It also seems that the exact number of max printed significant numbers is Python version- and OS-dependant since it works for me on ST3. @mattst, thanks for the great images btw, but you wouldn't have needed this many. ;) |
https://docs.python.org/3/tutorial/floatingpoint.html It seems we have 2 possible solutions:
|
I think that option 1 sounds like the best solution, without having to introduce more changes. |
I got a bit carried away. :) I was trying to show the variations in output, some work fine in my examples, as well as the format string workaround. In your solution considerations you might consider this point: The casual user (most users?) probably wants the precision only of the level they used as input. e.g. P.S. @FichteFoll - You realize that this erroneous ('oh, the shame') SublimeText issue was me as well? "In mousemap 4th mouse button is not referenced by "button4" instead its "button8". I only realized because you closed that issue 26 mins ago, so I've got back to back FichteFoll emails in my inbox about 2 different issues. |
@jbrooksuk @mattst
I generally prefer the "limit the significant digit number and round appropriately" approach but I'll have to see how well I can translate that into code. On my ToDo (but not high priority since it's a rather rare issue). PS: No, I did not. I usually memorize avatars instead of names for users because they are easier to remember (brain stuff) and only then associate a name to an avatar. However, since you're using a generated gravatar I just ignored that because they look too similar. |
Using: Sublime Text 3 Build 3059, Linux Mint 13 LTS.
Using: InsertNums v. 2.0.1 (version from: Insert Nums/package-metadata.json)
When using InsertNums to insert a number sequence with a floating point step value there are some very strange results.
I've done some screen captures in order to show a variety of examples.
Zip file with all images: sockshare.com - no waiting
Hope this helps.
The text was updated successfully, but these errors were encountered: