# 0.1 + 0.2 != 0.3November 15, 2015 10:13 AM   Subscribe

Floating point math, exemplified by Erik Wiffin. posted by metaquarry (43 comments total) 30 users marked this as a favorite

posted by fallingbadgers at 10:33 AM on November 15, 2015

I'm just going to quote pretty much the entirety of the main paragraph:

When you have a base 10 system (like ours), it can only express fractions that use a prime factor of the base. The prime factors of 10 are 2 and 5. So 1/2, 1/4, 1/5, 1/8, and 1/10 can all be expressed cleanly because the denominators all use prime factors of 10. In contrast, 1/3, 1/6, and 1/7 are all repeating decimals because their denominators use a prime factor of 3 or 7. In binary (or base 2), the only prime factor is 2. So you can only express fractions cleanly which only contain 2 as a prime factor. In binary, 1/2, 1/4, 1/8 would all be expressed cleanly as decimals. While, 1/5 or 1/10 would be repeating decimals. So 0.1 and 0.2 (1/10 and 1/5) while clean decimals in a base 10 system, are repeating decimals in the base 2 system the computer is operating in.

This just made both repeating decimals and floating point math make sense so suddenly that I swear I heard an audible click from inside my skull.
posted by 256 at 10:57 AM on November 15, 2015 [62 favorites]

256: "This just made both repeating decimals and floating point math make sense so suddenly that I swear I heard an audible click from inside my skull."

Eponysterical, in base 2.
posted by chavenet at 11:03 AM on November 15, 2015 [24 favorites]

That's not meant to denigrate the other parts of this, which are also great. But that URL is a thing of great elegance.
posted by jacquilynne at 11:17 AM on November 15, 2015 [13 favorites]

My friend Josh wrote a well-received explanation of floating point representations that really helped this stuff click for me (even though I had done numerical computing work and supposedly knew all the theory already). It has a ton of diagrams which are useful for visual/graphical thinkers.
posted by mbrubeck at 11:26 AM on November 15, 2015 [3 favorites]

It's probably more correct to say that the representation involves inaccuracy if you're not using a ratio, decimal, or currency numeric type, which create different kinds of errors but will give you an exact answer to 0.1 + 0.2 provided you're not automatically casting to float.

Otherwise, pretty cool.
posted by CBrachyrhynchos at 11:34 AM on November 15, 2015

One idea that will help you understand floating point behavior is realizing that all floating values are exact. Just like the decimal number 0.666666667 is exact. That decimal is not equal to 2/3, but it IS a precise, well-definined real number, not some fuzzy range. The only fuzziness in floating point math happens at the end of a numeric operation, when the infinite precision result is rounded to the nearest representable floating point value.
posted by ryanrs at 11:41 AM on November 15, 2015 [7 favorites]

Computer math is to math what computer music is to music.

I invented this platitude. Which is a truth statement.
posted by clvrmnky at 11:55 AM on November 15, 2015 [3 favorites]

I got stuck trying to figure out 0.2 factorial.

Where are the kerning police when you need them?
posted by JackFlash at 12:05 PM on November 15, 2015 [6 favorites]

I also read this as "0.1 + 0.2! equals 0.3".
And I just learned it is impossible to Google most punctuation symbols (like "!=").

0.1+0.2 != 0.3
0.1+0.2 <> 0.3
0.1+0.2 ≠ 0.3
One tenth plus two tenths does not equal three tenths.
posted by LEGO Damashii at 12:08 PM on November 15, 2015 [1 favorite]

This is true:

0.1 ~ 0.2 == √ ✌

For various approximations of truthiness that is.
posted by sammyo at 12:15 PM on November 15, 2015 [1 favorite]

Very timely. In my electronics class we just moved from hardware to software and I've spent the past week helping them with the conceptually easier but still maddening intro Arduino version of this problem:

int a = 0.4;
int b = 0.999;
float c = (200/6000) * 1000;
Serial.print(a * 100); // prints "0"
Serial.print(b); // zero again
Serial.print(c); // zero even though it (should) evaluate to > 1!

On quick search I'm astonished nobody sells a "C DGAF" t-shirt.
posted by range at 12:22 PM on November 15, 2015 [2 favorites]

This just made both repeating decimals and floating point math make sense so suddenly that I swear I heard an audible click from inside my skull.

Yes! I didn't know that a base N system can only represent fractions in decimal form where the fractions use prime factors of N. Very cool.

But I don't exactly understand why this is so. Is there an easy-to-grok proof for this? (The linked page is a little loosey-goosey on what it means for a fraction to "use" a number, which is making this a little hard to try to prove on my own.)
posted by painquale at 12:22 PM on November 15, 2015 [1 favorite]

painquale: does this help?
posted by metaquarry at 12:36 PM on November 15, 2015 [1 favorite]

A number can be precisely and finitely represented in a base if it can be expressed within the form of P*B^n + Q*B^(n-1) + R*B^(n-2) + ... + Z*B^0 where P,Q,R..Z are whole numbers, and B is the base raised to the positional exponent.

If the denominator of a fraction a/b has a prime factor of the base, then it can be transformed to the base by multiplication of its other factors. e.g. 1/8 multiplied above/below by 5*5*5 gives 125/1000 = 1*10^-1 + 2*10^-2 + 5 *10^-3 = 0.125
posted by Gyan at 12:45 PM on November 15, 2015 [3 favorites]

This is very topical for me. Thanks!
posted by easter queen at 12:45 PM on November 15, 2015

For you mis-parsers,
(0.2!) = Γ(0.2 + 1) ≈ 0.918169
where Γ is the gamma function.
posted by fantabulous timewaster at 1:05 PM on November 15, 2015 [7 favorites]

painquale: "decimal form" is misleading, as it implies base-10. If we generalize the "." symbol for a base-N system, then the written symbol 0.μ1μ2...μn represents the fraction (μ1·Nn-1 + μ2·Nn-2 + ... + μn·N0)/Nn for digits μ such that 0 ≤ μ < N.

This is an explanation of what the 0.____ notation means, and the point is that it's a fraction with denominator a power of N. In attempting to represent an arbitrary fraction m/d, you need an integer k such that d·k = Nn for some n. Taking both sides modulo d gives us Nn mod d = 0, which can never hold if d has any prime factor not a factor of N.
posted by 7segment at 1:20 PM on November 15, 2015 [5 favorites]

More fun with repeating decimals:
* the number of digits in the repeating cycle of n/k (such as 0.142857142857... for n=1, k=7) is always < k
* Turning this around, in base 10, you can always find a finite sequence of 9's divisible by any positive integer co-prime with 10. But the length varies. 7 multiplies with 142857 to get 999999 (length = 6), but 37 needs only 27 to get 999 (length=3 instead of 36). There's probably some number theory that correlates this?
* If your grandmother is Irish, and your great-great-grandmother, and so on, skipping each generation, and your family did this infinity times, you would be exactly one-third Irish. In binary: 1 divided by 11 is 0.0101... (also works for other ethnicities)
posted by kurumi at 1:27 PM on November 15, 2015 [2 favorites]

0.66666667 isn't just a real (real numbers are an abomination), it's a rational!

5 is far less useful than 3, which is why we should be using base-12, where 1/3 and 1/6 are non-repeating fractions.
posted by Monday, stony Monday at 2:05 PM on November 15, 2015 [2 favorites]

In 1995 I received a beta unit of a new scale indicator which would become one of the leading products for one of the largest manufacturers in the world. The manufacturer bragged that this new device used double precision floating point throughout. The first thing we did with it was hook it up to a hopper scale which was graduated to count by 0.02 lb, and within about five minutes we noticed that it occasionally threw up an odd digit in the LSD.

What many people don't know about floating point math is that the single precision libraries round off after every operation, because the precision is execrable to begin with and rounding only makes it barely tolerable, but since it takes about four times the CPU power for every operation and those libraries were written in the 1970's when you paid for your computer usage by the CPU cycle, the double precision libraries don't round off until you tell them to explicitly. So even though the precision is a lot higher the double precision libraries are much more prone to answer 1 / 10 * 10 with 0.999999999999 after truncation if you don't explicitly round off the result. They were still correcting rounding errors in the firmware of that device two years later.

There is a place for floating point math but it is a very limited place which has to be carefully monitored. For the most part I go with the dictum that if you don't know how to solve your problem with integer math, you don't know how to solve your problem. There is certainly no reason that something like a scale which is regulated by law and required to adhere to common standards of exactness should ever use floating point math.
posted by Bringer Tom at 2:32 PM on November 15, 2015 [4 favorites]

int a = 0.4;
int b = 0.999;
float c = (200/6000) * 1000;
Serial.print(a * 100); // prints "0"
Serial.print(b); // zero again
Serial.print(c); // zero even though it (should) evaluate to > 1!

I don't know much about arduinos but your problems have nothing to do with C. In C, a and b would not show as zero unless cast to an integer in a way that always rounds down. I suspect the problem is with Serial.print()

The problem with your float c is that it does the math first and makes it a float second. You are doing math with integers, where (200/6000) = 0, because 200 < 6000. If you had 200/6000.0 or 200.0/6000 or float(200)/6000 etc it would work fine.
posted by RustyBrooks at 2:35 PM on November 15, 2015 [1 favorite]

RustyBrooks, no, it's C. Float-to-integer conversion is done by truncation.
```\$ cat test.c
#include <stdio.h>
int main(){
int a = 0.4;
int b = 0.999;
float c = (200/6000) * 1000;
printf("%d\n%d\n%f\n", a,b,c);
}
\$ gcc test.c -o test
\$ ./test
0
0
0.000000
```
posted by fantabulous timewaster at 2:58 PM on November 15, 2015 [4 favorites]

OK fine but why are you trying to print floats with something meant to print ints? That is dumb. WIthout trrying I wouldn't know what %d would do to a float (round or truncate) but it doesn't matter, unless you're doing that on purpose it's irrelevant, don't do that.

The problem with the variable "c" is strictly that the right hand side is all integer math. It's doing it correctly.
posted by RustyBrooks at 3:14 PM on November 15, 2015 [1 favorite]

%d isn't doing anything to a float in that program. With any sane compiler, there will be two integer zeroes and one floating-point zero passed to printf. "0.4" and "0.999", as well as the expression "(200/6000) * 1000", exist only in the source code, and by the rules of C, are compiled to the constants 0, 0, and 0.0.
posted by mubba at 3:28 PM on November 15, 2015 [4 favorites]

I think you might have missed that a and b are typed as ints. The complaint that might be leveraged against C here is that it isn't aggressive enough at enforcing types-- it's happy to silently cast 0.4 and 0.999 to ints, and to cast the integer result of the integer expression (200/6000)*1000 [i.e., 0] into a float. Arguably the compiler should not let you assign a float literal (or expression thereof) to an int or vice-versa without an explicit cast. Ideally the compiler would emit warnings for constant expressions like 200/6000 which are almost always errors.
posted by Pyry at 3:32 PM on November 15, 2015 [1 favorite]

I didn't notice the int-ness of a and b, sure. My main point was that "c" was doing exactly what you asked it to. Anyway, none of this has to do with the oddness expressed in the OP. They're just mistakes of treating floats as ints, and doing integer math when you "think" you're doing floating-point-number math.
posted by RustyBrooks at 3:38 PM on November 15, 2015

Relevant SMBC.
posted by MikeKD at 4:09 PM on November 15, 2015 [5 favorites]

Metafilter: we noticed that it occasionally threw up an odd digit in the LSD.
posted by oceanjesse at 4:28 PM on November 15, 2015 [7 favorites]

The question is: how does, say, the calculator in Windows work? Something tells me that that's the complicated part.
posted by BiggerJ at 4:38 PM on November 15, 2015

From Raymond Chen's blog:
Today, Calc's internal computations are done with infinite precision for basic operations (addition, subtraction, multiplication, division) and 32 digits of precision for advanced operations (square root, transcendental operators)
Previously it used IEEE 754 floating point; so it would do the 0.30000000000000004 thing. Some other systems include rational numbers (represented as a ratio of two integers) or multiple precision floating point, often using the GNU MPFR.
posted by Monday, stony Monday at 5:02 PM on November 15, 2015 [1 favorite]

> real numbers are an abomination

> if you don't know how to solve your problem with integer math, you don't know how to solve your problem

I sense a whole lot of hate for Modern Analysis in this thread...
posted by 7segment at 5:17 PM on November 15, 2015

The complaint that might be leveraged against C here is that it isn't aggressive enough at enforcing types-- it's happy to silently cast 0.4 and 0.999 to ints

I'm not 100% sure, but I suspect most modern C compilers would at least raise a warning. Microsoft C++ gives a "conversion from double to int, possible loss of data" warning.

And any code reviewer who sees that should warm up their slappin' hand.
posted by Foosnark at 5:25 PM on November 15, 2015 [2 favorites]

Yeah a lot of the C typing issues I see arise (I think) from Arduino in particular being built on top of avr-gcc with all the warnings turned way way down. Didn't mean to cause a derail, though I will say that mubba gave the explanation that I believe to be true also. The commonality I noticed was the way beginning coders will use (all) variable types with all sorts of attached baggage about what they assume the machine will do. Honestly, I think there are so many levels of detail to know that it's inevitable that you get bit by this stuff somewhere somehow, whether it's C casting by truncation or 1/10 being an infinitely repeating number in base 2.

The error in my variable c is particularly evil, as the C compiler will do integer math left-to-right (and therefore 200/6000=0 by truncation) and evaluate assignment last, at which point it happily casts zero to a float).

(And at least in my own code I'm extremely sympathetic to the "integer math or gtfo" line of reasoning, though I think I came by that because floats were so expensive in Ye Olde Days, rather than having any noble thoughts about precision.)
posted by range at 6:11 PM on November 15, 2015 [1 favorite]

This one time, I worked on a graphics system that stored floating point 3D coordinates using a shared exponent. As in, 3 mantissas and 1 exponent field. This worked out well. The main difference being the sphere of uncertainty in the results was an actual sphere, instead of a weird football with more precision in the smaller dimensions (where it wasn't terribly useful).

We also used many weird fp sizes, like 20-bit floats and 44-bit floats. We were calculating this one specific formula, and each operator got its own dedicated logic, so we had wide latitude to trim away every bit that wasn't absolutely needed.

I wrote most of the software that simulated these weird size fp units, and I'm pretty sure I can still do IEEE math by hand.
posted by ryanrs at 6:14 PM on November 15, 2015 [2 favorites]

Of course, no floating point thread would be complete without a mention of Quake3's Fast InvSqrt().
posted by ryanrs at 6:18 PM on November 15, 2015 [7 favorites]

Neat article, thanks. I do some basic "what do you notice about n when 1/n terminates??" sort of mini-explorations in my math classes, this is a nice supplement to those conversations.

But perhaps most interesting to me is the huge collection of code below; one basic task accomplished over and over again in various languages. Anyone aware of similar websites that compile similar lists for other basic tasks? Printing "Hello World" or what have you? I have no misgivings of becoming a coder by looking over such tables, but I do find the examination and comparison of various syntaxes instructive.
posted by Theophrastus Johnson at 7:07 PM on November 15, 2015

painquale: does this help?

Perfect! Thank you! I get it now.
posted by painquale at 8:12 PM on November 15, 2015

Anyone aware of similar websites that compile similar lists for other basic tasks? Printing "Hello World" or what have you?

Rosetta Code
posted by Jpfed at 8:15 PM on November 15, 2015 [2 favorites]

IEEE754 lists two kinds of NaN:

sNaN = "Ghostbusters II"
qNaN = "My name is not important: the answer is Ghostbusters II!"
posted by rum-soaked space hobo at 2:20 AM on November 16, 2015

And that's why you shouldn't patent mathematics, but that's not important right now.
posted by Eleven at 4:38 AM on November 16, 2015 [4 favorites]

If you had 200/6000.0 or 200.0/6000 or float(200)/6000 etc it would work fine

Indeed. I'm enough of an old F77 type to be ranting “If they wanted reals, where are the decimal points?” at this thread. Srsly; you need to say what you mean to a computer. Everyone should use a 8 (minifloat) or 16 (half) bit float library once in their life.

Also, for the original Arduino poster: watch your sketch size balloon once you fix the code, as the avr-gcc maths library is large. Floating point and µcs don't mix well, even if ATMegas aren't exactly slow at it.
posted by scruss at 5:42 AM on November 16, 2015

#QNAN the Barbarian rampages through your operations!

I spend way to much time chasing him down.
posted by inpHilltr8r at 10:15 PM on November 16, 2015

« Older English is not normal   |   Just a bunch of blondes in a swimming pool Newer »