The numbers are all the same on the ESP8266. The only difference is that the 99.0 is rendered to 4 decimal places instead of 5, and therefore this bad digit falls off the bottom.

>>> print(9000160.0)
9000159.0

The code in question that would need to be looked at is:

static const float neg_pow[] = { 1e-32, 1e-16, 1e-8, 1e-4, 1e-2, 1e-1 };
static const float g_pos_pow[] = { 1e32, 1e16, 1e8, 1e4, 1e2, 1e1 };
int e, e1;
for (e = 0, e1 = 32; e1; e1 >>= 1, pos_pow++, neg_pow++) {
    if (*pos_pow <= f) {
        e += e1;
        f *= *neg_pow;
    }
}
if ((int)f >= 10) { e += 1; f *= FPCONST(0.1); }

int dec = e;
for (int i = 0; i < num_digits; ++i, --dec) {
    int32_t d = (int32_t)f;
    if (d < 0) 
        *s++ = '0';
    } else {
        *s++ = '0' + d;
    }
    if (dec == 0 && prec > 0) {
        *s++ = '.';
    }
    f -= (float)d;
    f *= (float)(10.0);
}

This is a very compact and efficient implementation. I don't immediately see any quick fix. The C99 code uses an unsigned long int to hold all the integer part to as many digits as necessary before converting them to the ascii string of decimal digits, but we don't have that. We'd need some kind of extra register that effectively cancelled the 10 and 0.1 factors before they were multiplied out and lost the accuracy.

Uh oh!

Rounding error when printing the number 460.0 #4212

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions