Why should we avoid floating points?

Floating point numbers are ubiquitous in programming, especially for calculations involving real numbers. However, despite their popularity, floating points have some issues that make them problematic for certain use cases. In this article, we will examine the drawbacks of floating point numbers and look at situations where they should be avoided.

Table of Contents

What are Floating Point Numbers?

Floating point numbers are a way to represent real numbers in computing. They allow for decimal values by using a fixed number of digits to represent the significant digits of a number (the mantissa) and an exponent to track the position of the decimal point.

For example, the number 12.34 could be represented as 1.234 x 10^1. The mantissa is 1.234 and the exponent is 1, indicating the decimal should be shifted one place to the right. This allows a wide range of values to be represented compactly in binary.

There are two common standards for floating point numbers:

Single precision (32 bits): 1 bit for the sign, 8 bits for the exponent, 23 bits for the mantissa

Double precision (64 bits): 1 bit for the sign, 11 bits for the exponent, 52 bits for the mantissa

Double precision provides more precision (15-16 decimal digits) compared to single precision (7 decimal digits). Nearly all modern CPUs have hardware support for fast floating point math using these standards.

The Imprecision of Floating Points

One of the major drawbacks of floating point numbers is that they cannot accurately represent all real numbers. Because the mantissa has a fixed number of binary digits, the values must be rounded to fit. This means many decimal fractions cannot be precisely encoded in binary floating point.

For example, the decimal value 0.1 when converted to binary is the repeating fraction 1/10 which is 0.0001100110011…. Since the mantissa can only store a fixed number of digits, the binary floating point version of 0.1 is only an approximation of the true value. This imprecision accumulates when doing successive calculations.

Many simple decimal values like 0.2 or 0.3 also cannot be perfectly represented in binary floating point. This can be surprising and lead to unintuitive situations where familiar decimal values appear to behave strangely when encoded in floating point.

Round-Off Errors

The imprecision of floating point representations means that round-off errors occur whenever values are converted to floating point. When you perform operations like addition and multiplication on floating point numbers, the result often cannot be represented exactly. So the final value is rounded to the nearest representable floating point number.

These tiny errors can accumulate over long chains of calculation and lead to significant inaccuracies. Here is a simple example in Python:

>>> sum = 0
>>> for i in range(100):
...     sum += 0.1
... 
>>> print(sum)
9.99999999999998

The final result should be 10, but the accumulated round-off errors lead to a value slightly less than 10.

Cancellation Errors

Another problem is cancellation error. This happens when you subtract two almost equal floating point numbers. The least significant digits are lost due to the finite precision of floating point numbers.

For example:

>>> x = 1.234
>>> y = 1.233
>>> print(x - y)
0.0010000000000000009

The result should be exactly 0.001, but digits have been lost in the subtraction. This has big implications for numerical stability in things like simulations and solving differential equations.

Comparison Issues

Due to their approximate nature, floating point numbers can cause unexpected behavior when checking for equality or sorting values. For example:

>>> 0.1 + 0.2 == 0.3
False

>>> 0.1 + 0.2
0.30000000000000004

Although 0.1 + 0.2 mathematically equals 0.3, the imprecise floating point representation makes the computation slightly off, so the equality comparison fails.

Likewise, sorting routines may produce counterintuitive ordering when operating on float values:

>>> nums = [0.1, 0.2, 0.3]
>>> sorted(nums)
[0.1, 0.3, 0.2]

0.2 is less than 0.3 mathematically but when converted to float, 0.2 ends up larger than 0.3 due to representation error.

When to Avoid Floating Point

Now that we’ve looked at the drawbacks, in what situations should floating point numbers be avoided? Here are some guidelines:

Financial applications – Use decimal type to avoid rounding errors
Sorting/search operations – Use integers or decimals instead
Hashing/encryption – Rounding errors alter discrete integer mappings

Equality comparison – Test approximate equality within an error margin
Geometric operations – Small errors accumulate and distort shapes
Stability-critical simulations – Tiny perturbations can grow rapidly

For most daily computation tasks, floating point works well. But for certain domains like finance, cryptography, numerical analysis etc., the imprecision becomes highly problematic.

Alternatives to Floating Point

If floating point is insufficient, what other representations can we use for real numbers?

Fixed point – Specify a fixed number of integral and fractional digits. No exponent.

Arbitrary precision – Store numbers with as many digits as needed.
Rationals – Store numerator and denominator separately.
Decimals – Base 10 representation stored in binary.

Intervals – Represent uncertainty explicitly as a range of possible values.

Most languages provide some alternatives in their standard libraries or via third-party packages. But they come with a performance cost compared to hardware-accelerated floats.

Mitigations When Using Floats

Sometimes alternatives are not feasible, and you have to live with floating point. In these cases, there are techniques to mitigate the issues:

Use epsilon comparison instead of strict equality.
Carry along calculation error estimates.
Prefer higher precision formats (double over float).

Control rounding behavior explicitly.
Use range analysis to bound accumulation of errors.

With proper analysis and testing, many applications can still use floats reliably. The key is understanding how error accumulates and propagates through the computation.

Conclusion

Floating point numbers provide an efficient way to represent real numbers in computing. However, their fixed precision leads to round-off and representation errors that make them unsuitable for some use cases. When high accuracy is critical, alternatives like decimal or arbitrary precision libraries should be considered despite their performance costs.

In most common cases, floating point works well enough. But understanding the potential pitfalls is important, especially in numerical programming. With proper numerical analysis and testing, the issues can be anticipated and controlled. Floating point may not provide perfect accuracy, but modern computers would be paralyzed without their speed and ubiquity.