The Number That Isn't There — floating point, decoded

Three lines that disagree

Nothing is broken · five answers to one question

Every layer of this page is a legitimate answer to "what is this number", and the three lines in the hero are three layers contradicting each other in public.

Here they are again, run rather than remembered. The interpreter is /usr/bin/python3.14, which reports itself as CPython 3.14.6, and it is named because §1 of the house style asks that the command shown be the command that produced the output.

hero.py · python3.14 hero.py

$ python3.14 hero.py
False
0.30000000000000004
5.551115123125783e-17
0.1 0.2 0.3
True

#  the script, in full:
#    print(0.1 + 0.2 == 0.3)
#    print(0.1 + 0.2)
#    print(0.1 + 0.2 - 0.3)
#    print(0.1, 0.2, 0.3)
#    print(0.1 + 0.2 == 0.30000000000000004)

Look at the fourth line. Printed on their own, 0.1, 0.2 and 0.3 come back as themselves — so the three inputs look perfect and only the sum looks broken. That is the shape of the whole problem: the storing was already lossy, and the printing hid it. The fifth line is the tell — the sum is exactly equal to that seventeen-digit decimal, which means the digits are not noise. They are the number.

And it is not a Python problem. Here are the first two lines again in four other languages, each compiled or run on this machine.

tenths.c · tenths.js · tenths.go · tenths.rs

#  C: printf("%d\n", a + b == c); printf("%.17g\n", a + b);
$ gcc -O0 -o tenths tenths.c && ./tenths
0
0.30000000000000004

#  JavaScript: console.log(0.1 + 0.2 === 0.3); console.log(0.1 + 0.2);
$ node tenths.js
false
0.30000000000000004

#  Go: fmt.Println(a+b == c); fmt.Println(a + b)
$ go run tenths.go
false
0.30000000000000004

#  Rust: println!("{}", a + b == c); println!("{:?}", a + b);
$ rustc -O -o tenths_rs tenths.rs && ./tenths_rs
false
0.30000000000000004

Four languages, three of them compiled, one of them at -O, and not one digit differs. C prints 0 rather than false because printf has no conversion that renders a truth value as a word — true and false are keywords in the C23 this gcc defaults to, but they never come into it: C's == yields an int, 1 or 0, and %d prints the integer it was handed — and it needed %.17g to be asked for the digits at all — the other three volunteer the shortest string that round-trips, which is §09. Everything else is identical because all four are doing the same arithmetic on the same 64 bits, specified by IEEE 754 in 1985 and implemented in the silicon. Verified with gcc 16.1.1, node v26.5.1, go1.26.5-X:nodwarf5 (Arch's build of go1.26.5) and rustc 1.95.0 — those four version strings name this machine, along with the interpreter named above — the footer counts them — and the numbers beside them do not.

The reframe you have to make is this. A float is not a number. It is a 64-bit pattern, and there are five different things you can legitimately call "the number it is", depending on which layer you stand on. Those five are the colours in the legend above, and the rail tells you which one every section of this page is standing on.

The widening · five answers to "what is this number", left to right

One pattern, five true descriptions, and they get further apart as you go down. Every row here is the same double. The bits are what is in memory; the fields are the cut the format imposes on them; the encoded value is what the formula makes of the fields; the real number is where that lands on the line; the printed decimal is what a library decided to tell you. Nothing on this page is a lie — but only the top row is a fact about the machine, and only the bottom row is what you ever see.

The sentence this page argues. 0.1 is not stored. The nearest available neighbour to one tenth is stored, and print shows you the shortest decimal string that would round back to that same neighbour — which happens to be the three characters you typed. The equality operator compares the neighbours; your eyes compare the strings. They disagree because they are looking at two different layers.

Where the point cannot move

The budget argument · range and resolution are the same money

Before floating point, the obvious design: nail the binary point at a fixed position and treat the whole thing as an integer scaled by a constant. It is a real technique, it is still used, and looking at what it costs is the fastest way to see what the moving point is actually for.

Take 64 bits and put the point exactly in the middle — 32 bits of whole number, 32 bits of fraction. That is the format usually written Q32.32. Storing a value means multiplying by 2³² and keeping the integer; reading it means dividing again. Every arithmetic operation is ordinary integer arithmetic, which is why it is fast and why fixed point never went away in audio and graphics.

fixed.py · python3.14 fixed.py

$ cat fixed.py
from decimal import Decimal, getcontext
from fractions import Fraction
getcontext().prec = 40

FRAC = 32                       # 64 bits, point nailed between bit 32 and bit 31
ONE  = 1 << FRAC

def to_q(x):                    # nearest representable, ties away from zero
    return int(x * ONE + (0.5 if x >= 0 else -0.5))

def exact(q):                   # what the stored integer actually stands for
    return Decimal(q) / Decimal(ONE)

print("biggest  :", exact((1 << 63) - 1))
print("step     :", exact(1))
print()
print("0.1 stored as   :", to_q(0.1))
print("which is exactly:", exact(to_q(0.1)))
print("wanted          :", Fraction(1, 10))

$ python3.14 fixed.py
biggest  : 2147483647.999999999767169356346130371094
step     : 2.3283064365386962890625E-10

0.1 stored as   : 429496730
which is exactly: 0.1000000000931322574615478515625
wanted          : 1/10

Two numbers and one disappointment. The biggest value this format can hold is about 2.1 billion and the smallest step it can take is about two ten-billionths — and those two facts are the same fact, because the 64 bits have to pay for both. Want a bigger range? Move the point left and lose resolution. Want finer steps? Move it right and lose range. But look at the last two lines: fixed point does not fix one tenth either. It stores 0.1000000000931322574615478515625, which is not one tenth, for exactly the same reason a double will not.

That last point is the one worth carrying. The failure to store 0.1 has nothing to do with floating point. It is a fact about base two. A fraction can be written exactly in base b only when its denominator's prime factors all divide b. Base ten has the primes 2 and 5, so tenths, fifths, halves and quarters are all exact. Base two has only the prime 2, so only halves, quarters, eighths — anything whose denominator is a power of two.

One tenth is 1/(2×5). The 5 has nowhere to go. In base two, one tenth is a repeating fraction in the same way one third is a repeating fraction in base ten: 0.333… never terminates and neither does 0.0001100110011…, and no finite number of bits will ever hold either one.

Decimal	As a fraction	Denominator	Stored exactly?	What is really stored
0.5	1/2	2¹	yes	0.5
0.25	1/4	2²	yes	0.25
0.125	1/8	2³	yes	0.125
0.75	3/4	2²	yes	0.75
0.0625	1/16	2⁴	yes	0.0625
2.5	5/2	2¹	yes	2.5
0.1	1/10	2 × 5	no	0.1000000000000000055511151231257827021181583404541015625
0.2	1/5	5	no	0.200000000000000011102230246251565404236316680908203125
0.3	3/10	2 × 5	no	0.299999999999999988897769753748434595763683319091796875
0.7	7/10	2 × 5	no	0.6999999999999999555910790149937383830547332763671875

Read the last column. Those are not approximations of what is stored — they are what is stored, printed in full by decimal.Decimal(float), which converts without losing anything because every double is a terminating decimal. A number with 55 decimal digits is being kept in 64 bits, which tells you the digits are not independent: they are what 3602879701896397 / 2⁵⁵ looks like when you insist on writing it in base ten.

So what is the moving point actually for? Range. Q32.32 spans about ±2.1×10⁹. A double spans about ±1.8×10³⁰⁸ and gets down to 5×10^-324, using the same 64 bits, by spending eleven of them on where the point is rather than on more digits. It buys reach, and it pays for it in a way §07 will make very concrete: the step size is no longer constant.

Sliding the point

Scientific notation · in base two, where every number starts with 1

You already know the trick: 299,792,458 is easier written as 2.99792458 × 10⁸. Do that in base two and you have invented floating point, including the one detail that looks like a cheat and is not.

In base ten, normalised scientific notation puts exactly one non-zero digit before the point. In base two there is only one non-zero digit, so a normalised binary number always looks like 1.something × 2^E. Always. The leading digit is 1 for every number except zero, which means storing it would be spending a bit of every normal number on a value you already knew. So it is not stored. It is implied, and the format gets a 53rd bit of precision for free out of 52 bits of storage.

sci.py · python3.14 sci.py

$ cat sci.py
import math

for x in (1.0, 0.5, 0.75, 3.0, 10.0, 0.1, 1e300):
    m, e = math.frexp(x)          # x == m * 2**e, with 0.5 <= m < 1
    print("%-8g frexp %-22r 2**%-5d hex %s" % (x, m, e, float.hex(x)))

$ python3.14 sci.py
1        frexp 0.5                    2**1     hex 0x1.0000000000000p+0
0.5      frexp 0.5                    2**0     hex 0x1.0000000000000p-1
0.75     frexp 0.75                   2**0     hex 0x1.8000000000000p-1
3        frexp 0.75                   2**2     hex 0x1.8000000000000p+1
10       frexp 0.625                  2**4     hex 0x1.4000000000000p+3
0.1      frexp 0.8                    2**-3    hex 0x1.999999999999ap-4
1e+300   frexp 0.7466108948025751     2**997   hex 0x1.7e43c8800759cp+996

Two normalisations, one number. math.frexp normalises to a fraction in [0.5, 1), which is the C library's convention; float.hex normalises to [1, 2), which is IEEE 754's. That is why the exponents differ by one on every row — 0.1 is 0.8×2^-3 and also 1.6×2^-4, and both are true. The second is the one the bits actually hold, so it is the one this page uses from here on. And look at 0.1's digits in the hex column: 999999999999a, a repeating pattern that stops with something that is not a 9. That a is §06.

float.hex is worth keeping. It is the one built-in view of a double that loses nothing and hides nothing: 0x1. then the 52 mantissa bits as thirteen hex digits, then p and the exponent in decimal. Where repr negotiates with you, float.hex just reads out the fields.

Why base two and not base ten. Decimal floating point exists — IEEE 754 specifies it, and Python's decimal module implements it. It stores 0.1 exactly, which is why money belongs in it. It is also several times slower in software and is not what the floating-point hardware in your CPU is wired for. The world runs on binary64 because the hardware does, and the hardware does because binary is what the transistors already were.

Sixty-four bits, three fields

1 · 11 · 52 · and a bias of 1023

This is the whole format. One sign bit, eleven exponent bits, fifty-two mantissa bits, and one arithmetic convention to hold them together.

binary64 · one sign bit, eleven exponent bits, fifty-two mantissa bits

The eleven-bit exponent is stored biased, not signed. It holds a plain unsigned integer from 0 to 2047, and you subtract 1023 to get the real exponent — so 1023 means 2⁰, 1024 means 2¹, 1022 means 2^-1. The reason is not arithmetic convenience, it is comparison: with the bias, and with the sign bit at the top, two positive doubles compare correctly when you compare their 64-bit patterns as if they were integers. Hardware gets sorting and comparison almost for free. The two reserved patterns, all-zeros and all-ones, are §08.

Here are nine doubles with their fields split out. Nothing is interpreted yet — this is just the bytes, cut in two places.

bits.py · python3.14 bits.py

$ cat bits.py
import struct

def bits(x):
    return format(struct.unpack('>Q', struct.pack('>d', x))[0], '064b')

print("value              s exponent    mantissa")
for x in (1.0, 0.5, 0.25, 0.75, 2.0, 3.0, -1.0, 0.0, 0.1):
    b = bits(x)
    print("%-18r %s %s %s" % (x, b[0], b[1:12], b[12:]))

$ python3.14 bits.py
value              s exponent    mantissa
1.0                0 01111111111 0000000000000000000000000000000000000000000000000000
0.5                0 01111111110 0000000000000000000000000000000000000000000000000000
0.25               0 01111111101 0000000000000000000000000000000000000000000000000000
0.75               0 01111111110 1000000000000000000000000000000000000000000000000000
2.0                0 10000000000 0000000000000000000000000000000000000000000000000000
3.0                0 10000000000 1000000000000000000000000000000000000000000000000000
-1.0               1 01111111111 0000000000000000000000000000000000000000000000000000
0.0                0 00000000000 0000000000000000000000000000000000000000000000000000
0.1                0 01111111011 1001100110011001100110011001100110011001100110011010

Compare rows one and seven. 1.0 and -1.0 differ in exactly one bit, the leftmost — negation in this format is a single flip and never touches the magnitude, which is why -0.0 exists and §08 has to deal with it. Compare rows one, two and three: same mantissa, exponent walking down by one each time, and the value halving each time. And compare rows four and six: 0.75 and 3.0 have identical mantissas and differ only in the exponent, because 3 is 0.75 shifted left twice. The mantissa is the shape of a number; the exponent is its size.

Decode any double — click a bit to flip it

Type a number

Sign · 1 bit

Exponent · 11 bits

Mantissa · 52 bits

Bit · what is in memory

Field · the cut

Encoded value · the formula

Real number · exactly

Printed decimal · shortest

Every one of the five layers, at once. The exact-value row is computed with BigInt from the bits you see, not read off any table — flip a mantissa bit and watch the 50-odd digit expansion change while the printed decimal barely moves. The printed row is what JavaScript's own String(x) gives, which uses the same shortest-round-trip rule as Python's repr (§09).

The numbers that are exact

Halves, quarters, eighths · and every integer up to 2⁵³

Before working out what goes wrong with 0.1, work out what goes right with 0.5 — because a double that is exact is exact, with no error term at all, and there are a great many of them.

friendly.py · python3.14 friendly.py

$ cat friendly.py
import struct
from fractions import Fraction

print("value   exponent   significand      = the exact number stored")
for x in (1.0, 0.5, 0.25, 0.75, 2.5, 3.0):
    b = format(struct.unpack('>Q', struct.pack('>d', x))[0], '064b')
    E = int(b[1:12], 2) - 1023
    sig = 1 + Fraction(int(b[12:], 2), 1 << 52)
    print("%-7r %-10d %-6s x 2**%-3d = %s" % (x, E, sig, E, Fraction(x)))

$ python3.14 friendly.py
value   exponent   significand      = the exact number stored
1.0     0          1      x 2**0   = 1
0.5     -1         1      x 2**-1  = 1/2
0.25    -2         1      x 2**-2  = 1/4
0.75    -1         3/2    x 2**-1  = 3/4
2.5     1          5/4    x 2**1   = 5/2
3.0     1          3/2    x 2**1   = 3

The right-hand column has no decimal point in it and that is the point. Fraction(x) asks a float for the exact rational it stands for, and for these six the answer is a small tidy fraction with a power of two underneath. There is no rounding here, no epsilon, no "close enough" — 0.25 + 0.25 == 0.5 is true in the same way 1 + 1 == 2 is true. Half of all the folklore about floats not being comparable comes from not knowing this.

The same is true of whole numbers, up to a limit you can compute. A double's significand is 53 bits, so every integer that fits in 53 bits is exact; the first one that is not is 2⁵³+1, because the pattern of a double at that magnitude can only land on even numbers.

ruler.py · the integer half of the output

$ python3.14 ruler.py
...
first integer that is not exact:
  9007199254740991     -> 9007199254740991.0   exact: True
  9007199254740992     -> 9007199254740992.0   exact: True
  9007199254740993     -> 9007199254740992.0   exact: False
  9007199254740994     -> 9007199254740994.0   exact: True

Nine quadrillion and seven trillion, and then it stops counting by ones. 9007199254740992 is 2⁵³. Above it the gap between neighbouring doubles is 2, so the odd numbers are simply not there and asking for one gives you its even neighbour, silently and without an error. This is the reason JavaScript — where every numeric literal without an n on the end is a double, BigInt having arrived only in ES2020 — has Number.MAX_SAFE_INTEGER at 9007199254740991, and the reason a 64-bit database ID handed to JSON can come back changed.

The rule, in one line. A double holds a number exactly when that number is some 53-bit integer times some power of two. Every half, quarter and eighth qualifies. Every integer below 2⁵³ qualifies. One tenth does not, and never will, in any number of bits.

One tenth, bit by bit

Long division in base two · and the round that ends it

You have been told 0.1 repeats forever in binary. Here is the division that proves it, done the way you would do it on paper, with the remainder printed at every step.

Converting a fraction to binary is repeated doubling: double it, and if the result is at least 1, emit a 1 and subtract; otherwise emit a 0. Do that to 1/10 with integers and the remainders tell the whole story.

tenth.py · python3.14 tenth.py

$ cat tenth.py
n, d = 1, 10                       # long division of 1/10, in base 2
out = []
for step in range(1, 13):
    top = n * 2
    bit, n = divmod(top, d)
    out.append(str(bit))
    print("step %2d:  %2d / 10 -> bit %d, remainder %d" % (step, top, bit, n))
print()
print("0.1 in binary = 0." + "".join(out) + "...")

$ python3.14 tenth.py
step  1:   2 / 10 -> bit 0, remainder 2
step  2:   4 / 10 -> bit 0, remainder 4
step  3:   8 / 10 -> bit 0, remainder 8
step  4:  16 / 10 -> bit 1, remainder 6
step  5:  12 / 10 -> bit 1, remainder 2   ← remainder 2 again, as at step 1
step  6:   4 / 10 -> bit 0, remainder 4
step  7:   8 / 10 -> bit 0, remainder 8
step  8:  16 / 10 -> bit 1, remainder 6
step  9:  12 / 10 -> bit 1, remainder 2
step 10:   4 / 10 -> bit 0, remainder 4
step 11:   8 / 10 -> bit 0, remainder 8
step 12:  16 / 10 -> bit 1, remainder 6

0.1 in binary = 0.000110011001...

Look at the remainder column, not the bit column. Step 5 leaves remainder 2, which is what step 1 started from — and a long division that returns to a remainder it has already seen must repeat everything after it, forever. There are only ten possible remainders when dividing by ten, so this was always going to happen; the only question was where. Four bits into the cycle, so the pattern is 0011 repeating, which shifted round the other way is the 1001 you can see in the mantissa in §04.

Divide 1 by 10 in base two

The division

What has been emitted

remainder
seen before

Where the division has to stop

A double has 53 significand bits and one tenth wants infinitely many, so at some point the machine stops and rounds. The rule is round half to even: take the nearest representable value, and if you are exactly halfway between two, take the one whose last bit is 0. Here is that decision made in exact arithmetic, with nothing rounded until the last line.

round53.py · python3.14 round53.py

$ cat round53.py
from fractions import Fraction

x = Fraction(1, 10)
E = -4                                   # 1/10 = 1.6 x 2**-4, so the exponent is -4
sig = x / Fraction(2) ** E               # the significand, 1 <= sig < 2
scaled = sig * (1 << 52)                 # 52 bits of room below the leading 1

low  = scaled.numerator // scaled.denominator
left = scaled - low
print("significand        :", sig)
print("x 2**52            :", scaled)
print("round down to      :", low)
print("what is left over  :", left, "  more than 1/2, so round up")
print("round up to        :", low + 1)
print()
print("53-bit significand :", format(low + 1, '053b'))
print("the leading 1 is implied; the stored mantissa is the other 52:")
print("mantissa bits      :", format((low + 1) - (1 << 52), '052b'))
print("mantissa hex       :", format((low + 1) - (1 << 52), '013x'))

$ python3.14 round53.py
significand        : 8/5
x 2**52            : 36028797018963968/5
round down to      : 7205759403792793
what is left over  : 3/5   more than 1/2, so round up
round up to        : 7205759403792794

53-bit significand : 11001100110011001100110011001100110011001100110011010
the leading 1 is implied; the stored mantissa is the other 52:
mantissa bits      : 1001100110011001100110011001100110011001100110011010
mantissa hex       : 999999999999a

That is where the a comes from. The true significand is 8/5, which times 2⁵² is 36028797018963968/5 — not a whole number, and 3/5 of the way past 7205759403792793. Three fifths is more than a half, so the nearest available value is the one above, and rounding up turns the final repeating 1001 into 1010: hex 9 becomes hex a. Every subsequent surprise on this page is downstream of this single upward nudge — and note that the nudge is the remaining 2/5, not the 3/5 the listing prints: 3/5 is how far past the lower candidate one tenth already sat, so rounding up moves it the other 2/5, which is 2/5 of one part in 2⁵⁶. The next figure measures exactly that.

And that nudge is measurable. The stored value is above one tenth, by a specific amount:

python3.14 -c · the size of the error

$ python3.14 -c "
import math
x = 0.1
lo = math.nextafter(x, -math.inf); hi = math.nextafter(x, math.inf)
from decimal import Decimal
for label, v in (('below', lo), ('0.1  ', x), ('above', hi)):
    print(label, repr(v), Decimal(v))
print('gap  :', math.ulp(0.1))
print('0.1 is', float(Decimal(1)/Decimal(10) - Decimal(x)), 'away from one tenth')
"
below 0.09999999999999999 0.09999999999999999167332731531132594682276248931884765625
0.1   0.1 0.1000000000000000055511151231257827021181583404541015625
above 0.10000000000000002 0.10000000000000001942890293094023945741355419158935546875
gap  : 1.3877787807814457e-17
0.1 is -5.551115123125783e-18 away from one tenth

Three consecutive doubles, and one tenth is between two of them. The middle row is what 0.1 gets you. The rows above and below it are its immediate neighbours — there is nothing whatsoever between them, and the gap is 1.39×10^-17. True one tenth sits 5.55×10^-18 below the stored value, which is about 40% of one gap: closer to the middle row than to the row below, which is precisely why the middle row was chosen. Note also the printed column, and count significant digits rather than characters: the neighbour below needs sixteen of them and the neighbour above needs seventeen, while the one in the middle needs one. That is not because it is a rounder number. It is because it is the double you land on when you write 0.1, so 0.1 is already enough to identify it, and §09 is about why that is the rule.

A ruler with uneven ticks

The gap doubles every binade · and half of all positive doubles live below 1

Fixed point puts its ticks at even intervals along the whole line. Floating point does not: it puts the same number of ticks in every doubling of magnitude, so the ticks are dense near zero and enormously far apart out at the end.

Between 1 and 2 there are 2⁵² doubles. Between 2 and 4 there are also 2⁵² doubles — spread over twice the distance, so each gap is twice as wide. That interval, from one power of two to the next, is called a binade, and the gap inside it is the unit in the last place, the ulp.

ruler.py · python3.14 ruler.py

$ cat ruler.py
import math

print("binade            gap between neighbours")
for e in (-4, -1, 0, 1, 4, 10, 20, 52, 53, 60):
    x = 2.0 ** e
    print("[2**%-3d, 2**%-3d)  %r" % (e, e + 1, math.ulp(x)))
print()
print("epsilon (ulp of 1.0) :", math.ulp(1.0), "==", 2.0 ** -52)
#  ... the integer block shown in §05 follows here

$ python3.14 ruler.py
binade            gap between neighbours
[2**-4 , 2**-3 )  1.3877787807814457e-17
[2**-1 , 2**0  )  1.1102230246251565e-16
[2**0  , 2**1  )  2.220446049250313e-16
[2**1  , 2**2  )  4.440892098500626e-16
[2**4  , 2**5  )  3.552713678800501e-15
[2**10 , 2**11 )  2.2737367544323206e-13
[2**20 , 2**21 )  2.3283064365386963e-10
[2**52 , 2**53 )  1.0
[2**53 , 2**54 )  2.0
[2**60 , 2**61 )  256.0

epsilon (ulp of 1.0) : 2.220446049250313e-16 == 2.220446049250313e-16

Every row is exactly twice the row above it at the next exponent, and that is the entire mechanism. The famous "machine epsilon", sys.float_info.epsilon, is nothing more than the third row: the gap in the binade that contains 1.0, which is 2^-52. It is not a universal error bound. Out past 2⁵² the gap is 1.0, so adding 0.5 to a number that size lands exactly on a tie — and round-half-to-even then discards it for every even value there and rounds it up to a whole 1 for every odd one, which is worse than either answer on its own. In the binade above 2⁶⁰ the neighbours are 256 apart — and 2⁶⁰ itself, sitting at the bottom of it, is 256 from the double above and only 128 from the one below.

python3.14 -c · counting the ticks

$ python3.14 -c "
import struct
def u(x): return struct.unpack('>Q', struct.pack('>d', x))[0]
print('doubles in [1,2)      :', u(2.0) - u(1.0))
print('doubles in [0,1)      :', u(1.0) - u(0.0))
print('positive finite total :', u(float('inf')) - u(0.0))
print('0.1 to 0.2 is', u(0.2) - u(0.1), 'floats apart')
"
doubles in [1,2)      : 4503599627370496
doubles in [0,1)      : 4607182418800017408
positive finite total : 9218868437227405312
0.1 to 0.2 is 4503599627370496 floats apart

Just under half of every positive double there is lands between zero and one. 4607182418800017408 of 9218868437227405312 is 49.98%, which is the ruler stated as a population rather than as a spacing: the binades below 1 are countless and tiny, the ones above are few and vast. This works because of §04's bias — reading a double's bits as an unsigned integer gives you its rank in the ordered list of all doubles, so subtracting two of them counts the values in between. The last line is that trick used for something else: 0.1 and 0.2 are 2⁵² distinct doubles apart, which is exactly one whole binade, because doubling only moves the exponent.

The ruler · equal counts, unequal spacing

Twelve ticks are drawn per binade and there are 2⁵²; the counts are decorative and the ratios are exact. This is why relative error is roughly constant for floats and absolute error is not. A double gives you about sixteen significant decimal digits whether the number is 0.001 or 10³⁰⁰ — and that is a much more useful guarantee than a fixed step size, right up until you add a small number to a large one and it vanishes.

Pick a magnitude, see the gap

Exponent E

Binade

Gap between neighbours

Relative gap · the accuracy promise

Drag it to the far right. Out at 2¹⁰²³ the gap between one double and the next is larger than the number of atoms anyone has counted, and yet the relative gap — the fraction of the value it represents — is bounded by the same 2^-52 that bounds it at 2⁰, reaching that bound at the bottom of each binade and about half of it by the top. That bound is the only accuracy promise the format makes.

What that does to arithmetic

python3.14 -c · four consequences

$ python3.14 -c "
t = 0.0
for i in range(10):
    t += 0.1
print('ten tenths   :', repr(t), t == 1.0)
print('sum([0.1]*10):', repr(sum([0.1]*10)))
print('math.fsum    :', __import__('math').fsum([0.1]*10))
print('0.1*3        :', repr(0.1*3), 0.1*3 == 0.3)
print('1e16 + 1     :', repr(1e16 + 1))
print('(0.1+0.2)+0.3:', repr((0.1+0.2)+0.3))
print('0.1+(0.2+0.3):', repr(0.1+(0.2+0.3)))
"
ten tenths   : 0.9999999999999999 False
sum([0.1]*10): 1.0
math.fsum    : 1.0
0.1*3        : 0.30000000000000004 False
1e16 + 1     : 1e+16
(0.1+0.2)+0.3: 0.6000000000000001
0.1+(0.2+0.3): 0.6

The last two lines are the same three numbers added in a different order, and they are not equal. Floating-point addition is commutative but not associative, which is why a compiler is not allowed to reassociate float arithmetic without being asked and why summing an array in parallel can give a different answer each run. 1e16 + 1 returning 1e16 is the ruler again, and it is the payoff's mechanism arriving early: 1e16 is exact, the gap there is 2, so adding 1 lands exactly on the fence between two doubles. Round-half-to-even breaks the tie, 1e16's significand 5000000000000000 is the even one, and the 1 you added disappears. And the second and third lines are not luck — see the caveat below.

The honest caveat about sum(). The naive loop and the built-in sum add the same ten values in the same order and get different answers, which cannot happen unless they are doing different arithmetic. They are: on this interpreter sum() carries a compensation term for float inputs, so it recovers the error the loop discards. Check it with a case that has nothing to do with tenths — [1e16, 1.0, -1e16, 1.0] gives 1.0 from a loop and 2.0 from sum(), and math.fsum agrees with sum(). This is a CPython implementation detail rather than anything IEEE 754 requires, and it has not always been true. Everything else on this page holds in any language with binary64; this one row does not.

What the exponent buys

Two reserved patterns · zero, subnormals, infinity and NaN

Eleven bits give 2048 exponent values and the format only uses 2046 of them for ordinary numbers. The two it holds back — all zeros and all ones — are where zero, the very small, the overflowed and the undefined all live.

edges.py · python3.14 edges.py

$ python3.14 edges.py
0.0                    0 00000000000 0000000000000000000000000000000000000000000000000000
-0.0                   1 00000000000 0000000000000000000000000000000000000000000000000000
smallest normal        0 00000000001 0000000000000000000000000000000000000000000000000000
largest subnormal      0 00000000000 1111111111111111111111111111111111111111111111111111
smallest subnormal     0 00000000000 0000000000000000000000000000000000000000000000000001
largest finite         0 11111111110 1111111111111111111111111111111111111111111111111111
inf                    0 11111111111 0000000000000000000000000000000000000000000000000000
-inf                   1 11111111111 0000000000000000000000000000000000000000000000000000
nan                    0 11111111111 1000000000000000000000000000000000000000000000000000

0.0 == -0.0            : True
nan == nan             : False
inf - inf              : nan
1e308 * 10             : inf
5e-324 / 2             : 0.0
smallest subnormal     : 5e-324 == 5e-324

Read the exponent column top to bottom. All-zeros and all-ones are the two reserved patterns, and everything strange in this listing is one of them. Note the third and fourth rows especially: the smallest normal double has the smallest usable exponent and a zero mantissa, and the largest subnormal has the reserved exponent and a mantissa of all ones — they are adjacent on the number line and they are decoded by two different rules.

Exponent field	Mantissa	What it means	Decoded as
00000000000	all zero	zero, signed	±0
00000000000	non-zero	subnormal — no implied leading 1	±(m / 2⁵²) × 2^-1022
00000000001 … 11111111110	any	an ordinary number	±(1 + m / 2⁵²) × 2^e-1023
11111111111	all zero	infinity, signed	±∞
11111111111	non-zero	not a number	NaN

Subnormals: the gentle way to reach zero

Without the first two rows of that table, the smallest positive double would be 2.2250738585072014e-308 and the next value below it would be zero. Every other step on the whole number line is at most 2^-52 of the value it steps from — exactly that at the bottom of a binade, and about half of it by the top; that last one would be the value itself. At 2^-1022, which is a binade bottom, that makes the final step 2⁵² times bigger than the one before it, 4503599627370496×, right where you least want a cliff. So the all-zeros exponent turns off the implied leading 1 and fixes the exponent at 2^-1022, which lets the mantissa alone walk gradually down to nothing. That is gradual underflow, and it costs you significant bits one at a time instead of all at once.

The smallest positive double is therefore 2^-1074, printed as 5e-324 — a single mantissa bit set, with 52 leading zeros of significand that are no longer being paid for. Dividing it by 2 gives 0.0, and that is one of the two places where the format has nowhere left to go. The other is the top, and it is not quite where you would guess: a result past the largest finite double still rounds back down to it, right up to the halfway point between that double and the one that would have come next, which is where inf begins.

NaN, and the one comparison that lies

nan != nan is the single most surprising line in the listing above, and it is deliberate. NaN is the result of an operation that has no answer — inf - inf, 0/0, the square root of a negative — and the standard's position is that two things that are both "no answer" are not thereby the same thing. The consequence you will actually hit is that a NaN in a list breaks sorting, and that x != x is the idiomatic test for one.

There is not one NaN, there are about nine quadrillion of them. Any of the 2⁵² − 1 non-zero mantissas with the all-ones exponent is a NaN, times two for the sign bit — 9007199254740990 distinct bit patterns that all print as nan and all compare unequal to each other and to themselves. Some hardware and some libraries use the spare bits to record which operation went wrong; nothing portable does.

And -0.0 is the other one that catches people. It compares equal to 0.0, so you cannot find it with ==, but it is a different bit pattern — the first and second rows of that listing — and it survives arithmetic. It exists because negation is a sign-bit flip, and the format would have had to make an exception to avoid it.

python3.14 -c · finding a zero that == cannot see

$ python3.14 -c "
import math, struct
print('0.0 == -0.0       :', 0.0 == -0.0)
print('bits differ       :', struct.pack('>d',0.0).hex(), struct.pack('>d',-0.0).hex())
print('copysign(1, -0.0) :', math.copysign(1.0, -0.0))
print('atan2 sees it     :', math.atan2(0.0, -0.0), math.atan2(0.0, 0.0))
try:
    print(1 / -0.0)
except ZeroDivisionError as e:
    print('1 / -0.0          : ZeroDivisionError:', e)
"
0.0 == -0.0       : True
bits differ       : 0000000000000000 8000000000000000
copysign(1, -0.0) : -1.0
atan2 sees it     : 3.141592653589793 0.0
1 / -0.0          : ZeroDivisionError: division by zero

The last line is Python declining to implement the standard, and it is worth knowing which is which. IEEE 754 says dividing a positive number by -0.0 gives -inf and by 0.0 gives +inf; that is what C and JavaScript do, and node -e "console.log(1/-0)" prints -Infinity on this machine. Python raises instead, because the language decided a division by zero is a programming error rather than an arithmetic result. So the two zeros are still distinguishable in Python — by their bits, by math.copysign, by math.atan2 — just not by that route.

The printer lies to be kind

Shortest round-trip · the string is a choice, not a reading

Everything so far has been about storing. This section is about the other half of the deception, which is that printing a double is not reading it out — it is choosing, from many correct answers, the one you would most like to see.

A double's exact value always has an exact finite decimal expansion, because it is an integer over a power of two. Printing that expansion is possible, and it is what decimal.Decimal(x) does, and nobody wants it: 0.1 would print as fifty-five digits. So every language chooses a shorter string. The question is which.

printing.py · python3.14 printing.py

$ cat printing.py
x = 0.1 + 0.2

print("repr    :", repr(x))
print("16 digits:", "%.16g" % x)
print("17 digits:", "%.17g" % x)
print("20 digits:", "%.20g" % x)
print()
for p in range(1, 20):                     # shortest string that comes back
    s = "%.*g" % (p, x)
    if float(s) == x:
        print("shortest round-trip:", s, "at", p, "significant digits")
        break
print()
print("all of these are the same float:")
for s in ("0.1", "0.10000000000000001", "0.1000000000000000055511151231257827",
          "0.100000000000000005"):
    print("  %-38s -> %-6r  same as 0.1: %s" % (s, float(s), float(s) == 0.1))

$ python3.14 printing.py
repr    : 0.30000000000000004
16 digits: 0.3
17 digits: 0.30000000000000004
20 digits: 0.30000000000000004441

shortest round-trip: 0.30000000000000004 at 17 significant digits

all of these are the same float:
  0.1                                    -> 0.1     same as 0.1: True
  0.10000000000000001                    -> 0.1     same as 0.1: True
  0.1000000000000000055511151231257827   -> 0.1     same as 0.1: True
  0.100000000000000005                   -> 0.1     same as 0.1: True

At sixteen digits, 0.1 + 0.2 prints as 0.3. That line is the whole section. The sum is not 0.3, and a printer asked for sixteen significant digits will tell you it is, because at sixteen digits the two are indistinguishable. Ask for seventeen and the difference appears; ask for twenty and you get three more digits, and even they are not the exact ones — the true expansion continues …04440892…, so the twentieth significant digit is a 0 that %.20g has rounded up to a 1. There is no precision at which a printer stops making choices, only one at which the choices stop mattering. Meanwhile the bottom block shows the reverse direction: four different decimal strings, one float. Reading and writing are both many-to-one, in opposite directions.

What Python does now is the rule called shortest round-trip: print the shortest decimal string that, read back, gives exactly this double and no other. For the sum above that is seventeen digits, because nothing shorter is unambiguous. For 0.1 it is one digit, because "0.1" already picks out the right neighbour — every shorter or different string lands on some other double.

This is why 0.1 prints as 0.1 while being 0.1000000000000000055511151231257827021181583404541015625. The printer is not hiding the error. It is telling you the shortest thing that identifies this exact value, and for this value that thing happens to look like the number you asked for.

oldrepr.py · the two algorithms, side by side

$ cat oldrepr.py
x = 0.1
print("what str() printed before 3.1  (.12g) :", "%.12g" % x)
print("what repr() printed before 3.1 (.17g) :", "%.17g" % x)
print("what both print now            (repr) :", repr(x))
print("str is repr now                       :", str(x) == repr(x))

$ python3.14 oldrepr.py
what str() printed before 3.1  (.12g) : 0.1
what repr() printed before 3.1 (.17g) : 0.10000000000000001
what both print now            (repr) : 0.1
str is repr now                       : True

Python used to have two answers and neither was good. Until version 3.1, str was twelve significant digits — short, friendly, and it silently merged floats that were genuinely different. repr was seventeen — always unambiguous, and it made 0.1 look broken. Version 3.1 adopted the shortest-round-trip algorithm and collapsed the two into one, which is why the trailing digits in this page's hero exist at all: an older Python printed 0.1 + 0.2 as 0.3 through str, and the surprise arrived with the honesty. The two format specifiers above are still there, so you can run both algorithms yourself; the historical claim about which one shipped when is from the changelog, not from a run.

Seventeen digits, always; fifteen digits, safely. Seventeen significant decimal digits are always enough to round-trip any double — that is sys.float_info.dig + 2, and it is why %.17g is the paranoid serialisation format. In the other direction, any decimal number with fifteen or fewer significant digits survives a trip through a double and back unchanged, which is sys.float_info.dig itself, reported as 15. Between fifteen and seventeen is where the two directions disagree.

──

The payoff

All sixty-four bits of 0.1 · and where the trailing 4 comes from

Everything is now on the table. Here is one tenth, decoded from the top bit to the last decimal digit, and then the addition that produced the number in the hero.

One tenth, all sixty-four bits

decode.py · python3.14 decode.py

$ cat decode.py
import struct
from fractions import Fraction

def decode(x):
    n = struct.unpack('>Q', struct.pack('>d', x))[0]
    b = format(n, '064b')
    s, e, m = b[0], b[1:12], b[12:]
    biased = int(e, 2)
    E = biased - 1023
    frac = Fraction(int(m, 2), 1 << 52)
    value = (-1) ** int(s) * (1 + frac) * Fraction(2) ** E
    print("input        :", repr(x))
    print("64 bits      :", b)
    print("hex          :", struct.pack('>d', x).hex())
    print("sign     s   :", s, "     ->  (-1)**%s = %+d" % (s, (-1) ** int(s)))
    print("exponent e   :", e, "  ->  %d - 1023 = %d" % (biased, E))
    print("mantissa m   :", m)
    print("             :", "%d / 2**52 = %s" % (int(m, 2), frac))
    print("significand  : 1 + %s = %s" % (frac, 1 + frac))
    print("value        : %s x 2**%d = %s" % (1 + frac, E, value))
    print("             :", float(value), "  round-trips:", float(value) == x)

decode(0.1)

$ python3.14 decode.py
input        : 0.1
64 bits      : 0011111110111001100110011001100110011001100110011001100110011010
hex          : 3fb999999999999a
sign     s   : 0      ->  (-1)**0 = +1
exponent e   : 01111111011   ->  1019 - 1023 = -4
mantissa m   : 1001100110011001100110011001100110011001100110011010
             : 2702159776422298 / 2**52 = 1351079888211149/2251799813685248
significand  : 1 + 1351079888211149/2251799813685248 = 3602879701896397/2251799813685248
value        : 3602879701896397/2251799813685248 x 2**-4 = 3602879701896397/36028797018963968
             : 0.1   round-trips: True

Follow it down one line at a time and every layer of the legend goes past in order. The 64 bits are the only thing that exists. The three fields are the cut. The formula turns them into +1 × a significand × 2^-4, which is the encoded value. That resolves to one exact rational, 3602879701896397 / 36028797018963968 — and that denominator is 2⁵⁵, so the number really is a 53-bit integer over a power of two, exactly as §05 said it must be. The last line prints it, and the printer's answer is 0.1.

Written out in full, that rational is

0.1000000000000000055511151231257827021181583404541015625

— fifty-five decimal digits, all of them exact, produced by decimal.Decimal(0.1). Every double is a terminating decimal, because a denominator of 2⁵⁵ becomes 5⁵⁵ over 10⁵⁵ the moment you insist on base ten. What you type as three characters, the machine holds as this.

And now the trailing 4

Add the two stored values exactly — no rounding, using Fraction — and then work out which double the result has to become.

tie.py · python3.14 tie.py

$ cat tie.py
from fractions import Fraction

exact = Fraction(0.1) + Fraction(0.2)         # the exact sum of the two stored values
print("exact sum       :", exact)
print("denominator     : 2**55 ?", exact.denominator == 2**55)

E = -2                                        # because 0.25 <= sum < 0.5
scaled = exact / Fraction(2) ** E * (1 << 52) # its 53-bit significand, exactly
low = scaled.numerator // scaled.denominator
print()
print("significand     :", scaled)
print("two candidates  :", low, "and", low + 1)
print("exactly halfway :", scaled - low == Fraction(1, 2))
print("parity          :", low % 2, "and", (low + 1) % 2)
print()
print("round-half-to-even keeps the even one:", low + 1)
print("  which is  :", repr((low + 1) * 2.0 ** (E - 52)))
print("the odd one :", repr(low * 2.0 ** (E - 52)), "-- and that one is 0.3:",
      low * 2.0 ** (E - 52) == 0.3)

$ python3.14 tie.py
exact sum       : 10808639105689191/36028797018963968
denominator     : 2**55 ? True

significand     : 10808639105689191/2
two candidates  : 5404319552844595 and 5404319552844596
exactly halfway : True
parity          : 1 and 0

round-half-to-even keeps the even one: 5404319552844596
  which is  : 0.30000000000000004
the odd one : 0.3 -- and that one is 0.3: True

The sum lands exactly on the fence, and the tie-breaking rule pushes it up. This is the sharpest possible version of the answer. The exact sum of the two stored values is 10808639105689191 / 2⁵⁵, whose significand is a whole number and a half — it is precisely midway between two adjacent doubles. Round-half-to-even breaks the tie by taking the candidate with an even last bit. The even one is 5404319552844596, which is 0.30000000000000004. The odd one is 5404319552844595, which is 0.3. One parity bit, in a tie that occurs once, is the entire distance between what you expected and what you got.

hexes.py · the two of them, side by side

$ python3.14 hexes.py
...
0.1                    3fb999999999999a  0x1.999999999999ap-4
0.2                    3fc999999999999a  0x1.999999999999ap-3
0.3                    3fd3333333333333  0x1.3333333333333p-2
0.30000000000000004    3fd3333333333334  0x1.3333333333334p-2

Look at the last hex digit of the last two rows. Read as sixty-four-bit integers, 0.3 and 0.1 + 0.2 differ by exactly 1 — which is what "adjacent doubles" means, and which is §04's bias earning its keep, because it is only true if the encoding sorts in the same order as the numbers. As bit patterns they differ in three places, since hex 3 to hex 4 is 0011 to 0100: a carry, not a flip. Either way there is no value of any kind between them, and the entire hero paradox is that == can see the difference and your screen cannot. Note also rows one and two: 0.1 and 0.2 have identical mantissas and exponents one apart, because doubling a float only ever touches the exponent. That is why the pattern 999999999999a appears twice.

The three lines from the hero, resolved

The line	What it prints	Why
0.1 + 0.2 == 0.3	False	The sum rounds to 5404319552844596 × 2^-54; the literal `0.3` is 5404319552844595 × 2^-54. Adjacent doubles — one apart as integers, three bits apart as patterns — so `==` says no.
0.1 + 0.2	0.30000000000000004	Shortest round-trip needs seventeen digits here, because sixteen would print `0.3` and `0.3` reads back as the other double.
0.1 + 0.2 - 0.3	5.551115123125783e-17	The difference between two adjacent doubles at that magnitude — one ulp, 2^-54 — which is exact and therefore prints in full.

And the reframe the whole page was for. Nothing in those three lines is an error. Two decimal literals were stored as their nearest neighbours, an addition landed exactly between two more, a tie-breaking rule chose one, and a printer told you the shortest string that identifies it. Every step is correct, deterministic, and specified by IEEE 754 in 1985. The only thing that was ever wrong was the expectation that 0.1 was in there.

What to do about it. For money, use a decimal type — decimal.Decimal, or integer cents — because the problem is base two and no amount of care in base two fixes it. For measured quantities, compare with a tolerance rather than ==: math.isclose(a, b) exists for this and picks a relative tolerance by default, which is the right shape given §07's ruler. And for anything where the error accumulates over many terms, reach for math.fsum, which is exact. What not to do is round everything to two decimals and hope; that just moves the same problem somewhere less visible.