Ok, ok, joke’s on me. ;-) That will learn me to fire off blog posts too quickly. To clarify: Chad and I understand floating point arithmetic and its accuracy problems. What we weren’t thinking clearly on is Float#to_s.
Specifically, what was freaking us out was not the loss of precision on an FP conversion, but the fact that, when asked its value, it was rounding up. To make the point more clearly, we should have included all the output in the original ruby session:
> s = "40.87" "40.87" > f = s.to_f 40.87 > ft = f * 100 4087.0 > fti = ft.to_i 4086
I would have liked for this to be true:
> ft * 100 4086.9
but instead, what was true was:
> ft * 100 4087.0
Our confusion stemmed from the fact that we weren’t thinking about the fact that irb uses #to_s to display all its values. And, to demonstrate:
> 4086.9999999999995.to_s 4087.0
Which might be appropriate, but is still annoying. ;-) Especially given:
> 4086.999999999995.to_s "4086.99999999999"
(note: one less unit of precision)
which is what I was originally hoping to see.
To make it all weirder for us, “40.87” was the very first value we tried running through this little pain mill, and the only one with any resulting errors. C’est la vie, I guess.
Update: And, in JavaScript:
> f 40.87 > f * 100 4086.9999999999995 > (f * 100).toString() 4086.9999999999995
Reported correctly. ;-)
Comments
FWIW, in JRuby you’d get ft to be equal 4086.9999999999995. :)
Common Lisp suffers a bit less from this kind of problem, probably because it supports rationals. In SBCL:
CL-USER> (defvar s “40.87”) s “40.87” CL-USER> (defvar f (read-from-string s)) f 40.87 CL-USER> (defvar ft (* f 100)) ft 4087.0 CL-USER> (defvar fti (truncate ft)) fti 4087
Floats suck unless you’re in a hurry and don’t mind loss of precision. Use BigDecimal FTW:
BigDecimal.new(‘40.87’) * 100 = 4087.0
(BigDecimal.new(‘40.87’) * 100).to_i = 4087
A nit: irb actually calls #inspect, but Floats don’t define that method (Object#inspect then calls #to_s). I like Python’s behavior here: the Float#to_s equivalent (float.str) is somewhat similar to Ruby’s, but there’s a Float#inspect parallel (float.repr) which prints enough decimal places to ensure an accurate reproduction of the number within the type’s precision.
Spacebat: I love CL as much as the next guy, but rationals are no problem in Ruby. Just require ‘rational’.
Thanks! Really funny. I wish i could spend my time on writing articles…just have no time for it.