Floating-Point Error Calculator

Compare exact, approximate, absolute, relative, and percentage floating-point error. This educational calculator shows the formula, result, and step-by-step interpretation.

Computer ScienceNumerical Methods

Calculator

What this calculator teaches

Floating-point error calculations help explain rounding limits in computers and numerical methods.

Use the result as a learning aid. For classwork, still show the formula and intermediate reasoning so the final answer is not just a black-box number.

Absolute, Relative, and Percentage Error

Error measures how far an approximation lies from a reference value, but different error measures answer different questions. Absolute error |xapprox − xexact| keeps the same units as the quantity. Relative error divides that distance by |xexact|, making the error dimensionless and allowing comparisons across different scales. Percentage error is the relative error multiplied by 100%.

A small absolute error can still be large relative to a tiny exact value. For example, an error of 0.0001 is negligible compared with 100 but is 5% of 0.002. Conversely, a numerically larger absolute error can be relatively small when the reference value is large. This scale dependence is why numerical analysis often reports both absolute and relative error.

MeasureFormulaInterpretation
Absolute error|xa − xe|Distance from the reference in original units
Relative errorabsolute error / |xe|Error compared with reference magnitude
Percentage error100 × relative errorRelative error expressed as a percent

When the exact value is zero, the usual relative-error formula divides by zero and is undefined. Absolute error remains meaningful in that case. Floating-point computations can also contain representation and rounding effects, so a tiny nonzero difference between mathematically equivalent decimal expressions is often expected.

Formula & Symbols

ConceptFormula or rule
Absolute error|approx - exact|
Relative errorabsolute error / |exact|

Worked example

Example: 0.1 + 0.2 in binary floating point is often slightly above 0.3.
Example 2: Exact 100 and approximate 99.5 give absolute error 0.5, relative error 0.005, and percentage error 0.5%.
Example 3: Exact 0.002 and approximate 0.0021 give absolute error 0.0001 but relative error 0.05, so the percentage error is 5%.
Example 4: Exact −20 and approximate −19.8 give absolute error 0.2. Dividing by |−20| gives relative error 0.01 and percentage error 1%.
Example 5: In binary floating-point, 0.30000000000000004 differs from 0.3 by about 5.55 × 10−17. The relative error is about 1.85 × 10−16, illustrating representation-scale rounding.

Common mistakes

⚠️
Using the wrong input format

Keep lists comma separated, matrices as rows separated by semicolons, and modular inputs as integers.

⚠️
Ignoring assumptions

Some methods require positive probabilities, valid moduli, independent trials, or small educational input sizes.

⚠️
Keeping the sign of approximate minus exact

Error magnitude is normally reported with an absolute value. A signed difference can be useful as a bias or residual, but it is not the same quantity as absolute error.

⚠️
Dividing relative error by the approximation

The standard true relative error uses the magnitude of the reference or exact value in the denominator. Using the approximation defines a different measure.

FAQ

Can I use decimals?⌄
Most numerical calculators allow decimals, but modular arithmetic and coding-theory tools usually require integers or binary strings.
Is this for homework checking?⌄
Yes. The page is designed to show both the answer and the reasoning pattern.
Why does the result sometimes say approximate?⌄
Some probability, floating-point, and numerical methods naturally produce approximations.
Why is relative error undefined when the exact value is zero?⌄
Relative error requires division by |exact|. When the exact value is zero, that denominator is zero, so the standard ratio is undefined. Report absolute error or use a problem-specific scaled error measure instead.
Can percentage error exceed 100%?⌄
Yes. If the absolute error is larger than the magnitude of the exact value, the relative error exceeds 1 and percentage error exceeds 100%. There is no general mathematical upper bound.
Is floating-point error the same as measurement error?⌄
No. Floating-point error arises from finite numerical representation and rounding during computation. Measurement error comes from observations and instruments. Both can be described with absolute or relative error, but their sources are different.
Why do decimal calculations sometimes show tiny floating-point differences?⌄
Many decimal fractions have repeating binary expansions, so they cannot be stored exactly in a finite binary significand. Operations use nearby representable values, and the final result can differ from the ideal decimal result by a very small amount.

Related calculators

🔗 Set Operations Calculator🔗 Graph Theory and Adjacency Matrix Calculator🔗 Shannon Entropy and Information Gain Calculator🔗 Numerical Error Calculator

These links will work after the calculators are registered in the final Math layout update.

Formula Explorer connections

Interpretation: This relationship converts, summarizes or checks numerical quantities using standard arithmetic and measurement rules. Assumption: Use consistent units, preserve enough significant digits, and round only the final result unless the method states otherwise.

Floating-Point ULP Calculator →Fraction Calculator – Add, Subtract, Multiply and Divide →GPA Calculator – Grade Point Average →Math Formula Explorer →