iphone - Simple issue with floats and ints -
a simple question best asked 2 lines of code:
cgfloat answer = (abs(-27.460757f) - 9.0f) * (800.0f / (46.0f - 9.0f)); nslog(@"according calculator, answer should 399.15, instead is: %f", answer); when run in xcode (specifically, in iphone simulator), get:
according calculator, answer should 399.15, instead is: 389.189209
is due lack of understanding of how floats rounded?
thanks!
stewart
the abs() function operates on integers, abs(-27.460757f) returns 27. since you’re using floats, use fabsf() instead.
Comments
Post a Comment