iphone - Simple issue with floats and ints -


a simple question best asked 2 lines of code:

cgfloat answer = (abs(-27.460757f) - 9.0f) * (800.0f / (46.0f - 9.0f));     nslog(@"according calculator, answer should 399.15, instead is: %f", answer); 

when run in xcode (specifically, in iphone simulator), get:

according calculator, answer should 399.15, instead is: 389.189209

is due lack of understanding of how floats rounded?

thanks!

stewart

the abs() function operates on integers, abs(-27.460757f) returns 27. since you’re using floats, use fabsf() instead.


Comments

Popular posts from this blog

apache - Add omitted ? to URLs -

redirect - bbPress Forum - rewrite to wwww.mysite prohibits login -

php - How can I stop spam on my custom forum/blog? -