iphone - Simple issue with floats and ints -
a simple question best asked 2 lines of code:
cgfloat answer = (abs(-27.460757f) - 9.0f) * (800.0f / (46.0f - 9.0f)); nslog(@"according calculator, answer should 399.15, instead is: %f", answer);
when run in xcode (specifically, in iphone simulator), get:
according calculator, answer should 399.15, instead is: 389.189209
is due lack of understanding of how floats rounded?
thanks!
stewart
the abs()
function operates on integers, abs(-27.460757f)
returns 27. since you’re using floats, use fabsf()
instead.
Comments
Post a Comment