A Flash Developer Resource Site

Results 1 to 2 of 2

Thread: math error

  1. #1
    Member
    Join Date
    Nov 2003
    Posts
    36

    math error

    try run this basic math function after a while it starts adding a small fraction to the calculation and it goes nuts, has anyone else encountered this?



    _root.totalMoved = new Number(0)
    _root.totalMovedActual = new Number(0)
    function start(){
    _root.totalMovedActual = 0.4
    _root.totalMoved +=_root.totalMovedActual;
    trace("total Moved "+(50-_root.totalMovedActual));
    trace("total Moved Temp2 "+(50-_root.totalMoved))
    }

    setInterval(start,100)

  2. #2
    avatar free
    Join Date
    Jul 2002
    Location
    UK
    Posts
    835
    It's to do with the fact that numbers stored on any computer will have a limit to their precision. Eventually small descrepencies will add up. It should be fine with integers, up to a limit ( can't remember what it is in flash ) but with floating point numbers you are going to get these "rounding errors" when doing arithmetic on stuff. There's not a lot you can do ( as far as i know ). If you only want it to show 1 decimal place, then you could round the number to 1dp each time.
    jonmack
    flash racer blog - advanced arcade racer development blog

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  




Click Here to Expand Forum to Full Width

HTML5 Development Center