Re: Math Problem

"McKirahan" <News (AT) McKirahan (DOT) com> wrote:

How do I add floating point numbers accurately?

The following adds the 4 numbers
46.57, 45.00, 45.00, and 54.83 to give
191.39999999999998 instead of 191.40.

That's the way it is. Not all decimal fractions can be represented
exactly as binary fractions, which is how floating point numbers are
stored. Floating point numbers are necessarily somewhat imprecise. In
this case the imprecision is on the order of 10**-12, which is
extremely tiny.

--
Tim Slattery
Slattery_T (AT) bls (DOT) gov

Tim Slattery wrote:

McKirahan wrote:
How do I add floating point numbers accurately?

The following adds the 4 numbers
46.57, 45.00, 45.00, and 54.83 to give
191.39999999999998 instead of 191.40.

That's the way it is. Not all decimal fractions can be
represented exactly as binary fractions, which is how
floating point numbers are stored. Floating point numbers
are necessarily somewhat imprecise. In this case the
imprecision is on the order of 10**-12, which is extremely
tiny.

And it isn't a characteristic restricted to binary representations of
numbers. How precise is the decimal fraction representation of 1/3? And
would it be reasonable to complain that 3.33 + 3.33 + 3.33 did not
result in 1?

Richard.

Richard Cornford said the following on 10/5/2005 7:29 PM:

Tim Slattery wrote:

McKirahan wrote:

How do I add floating point numbers accurately?

The following adds the 4 numbers
46.57, 45.00, 45.00, and 54.83 to give
191.39999999999998 instead of 191.40.

That's the way it is. Not all decimal fractions can be
represented exactly as binary fractions, which is how
floating point numbers are stored. Floating point numbers
are necessarily somewhat imprecise. In this case the
imprecision is on the order of 10**-12, which is extremely
tiny.


And it isn't a characteristic restricted to binary representations of
numbers. How precise is the decimal fraction representation of 1/3? And
would it be reasonable to complain that 3.33 + 3.33 + 3.33 did not
result in 1?

Umm, yes it would since it should either result in 9.99 or 10, not 1

--
Randy
comp.lang.javascript FAQ - http://jibbering.com/faq & newsgroup weekly