Skip to content

py/formatfloat: Format whole-number floats exactly. #8901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 0 commits into from

Conversation

dpwe
Copy link
Contributor

@dpwe dpwe commented Jul 12, 2022

This is to fix issue #4212

Many bugs occur when printing float values containing integer values. The successive-scaling approach in the original algorithm would quickly make these values depart from whole numbers.

This modification attempts to avoid scaling the original value for numbers in the range 1.0 to 1e10, and instead compares with other floats constructed only by multiplying whole numbers, so guaranteed to remain exact until the mantissa runs out.

I added a test that verifies consistency with CPython for a range of values.

This PR is the successor to #8878 that cleans up the commit history.

Signed-off-by: Dan Ellis dan.ellis@gmail.com

@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Jul 12, 2022
@dpgeorge
Copy link
Member

This is the increase in code size for this PR:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:  +256 +0.049% 
unix nanbox:  +304 +0.066% 
      stm32:  +180 +0.046% PYBV10
     cc3200:    +0 +0.000% 
    esp8266:  +364 +0.052% GENERIC
      esp32:   +84 +0.006% GENERIC
     mimxrt:  +192 +0.054% TEENSY40
 renesas-ra:  +176 +0.028% RA6M2_EK
        nrf:  +192 +0.103% pca10040
        rp2:  +240 +0.047% PICO
       samd:  +180 +0.128% ADAFRUIT_ITSYBITSY_M4_EXPRESS

That's quite a lot. But I'd say it's worth it because this is fixing a problem that has been reported many times.

@jimmo
Copy link
Member

jimmo commented Jul 12, 2022

That's quite a lot. But I'd say it's worth it because this is fixing a problem that has been reported many times.

+1

Specifically because although it had a reasonable explanation and it's been demonstrated many times that it's "just" a formatting issue, it creates a perception that there's something wrong more broadly with floating point handling.

py/formatfloat.c Outdated
if (u_int > f_int) {
u_int /= 10;
--e;
break;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the code can be made a tiny bit smaller by doing:

if (u_int * 10 > f_int) {
    break;
}
u_int *= 10;
++e;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I specifically didn't do this because at the limit u_int is > INT_MAX / 10, so you get overflow in the comparison

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But... it's the same code, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I limit the loop to 9 steps. So it never attempts to calculate u_int as 10^10 (which would overflow), it just ends the loop at 10^9. This is OK because of the test above, which ensures f is not bigger than 4e9.

I agree the code is a trap for the unwary as it stands.

py/formatfloat.c Outdated
f *= FPCONST(0.1);
// First block is only for numbers that could be integers
// exactly-represented in float32.
if (f < FPCONST(4294967296.0)) { // First number too big for uint32_t.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this code all based around formatting single precision floats well? Or does it also work for double precision? Maybe f_int and u_int need to be uint64_t when mp_float_t is configured to be double?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works for double precision, but still only applies for numbers up to 2^32. We could go further by extending to 64 bits, but I think the vast majority of cases of interest are covered by the existing code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current code stops "working" around 2^32. For a float32-based build:

>>> print("{:f}".format(4294967167))
4294967040.000000
>>> print("{:f}".format(4294967168))
4294967174.530029

i.e. the first number passes the test on line 268 (but then suffers precision loss since the whole value is only stored with 23 bits). The second number, after casting into a float32, does not pass the test and is formatted according to the old code path, with the resulting fractional noise.

It would be possible to make this work for any whole-valued float but I wouldn't be able to use the trick of casting into int32s. However, I'm disappointed by the code size results. Maybe my original pure-float approach was better.

@dpwe
Copy link
Contributor Author

dpwe commented Jul 12, 2022

I have an alternate implementation that does all the arithmetic in the float domain. This removes the need to special-case at 2**32, so numbers that are larger than that still don't acquire fractional noise, e.g.

>>> print("{:f}".format(4294967167))
4294967040.000000
>>> print("{:f}".format(4294967168))
4294967296.000000

(compare to the example above).

I'm not sure how this will compare in terms of code size, and I assume it's slower (but I don't think that matters). (I'm wondering if the large code size increases come from pulling in uint32_t divide that wasn't otherwise linked in?)

Would you like me to propose an alternative PR based on that?

https://github.com/dpwe/micropython/blob/pure-float/py/formatfloat.c

@dpgeorge
Copy link
Member

Would you like me to propose an alternative PR based on that?

If you could that would be great. Reducing code size is the priority in this case (speed would be second priority).

@dpwe dpwe closed this Jul 13, 2022
RetiredWizard pushed a commit to RetiredWizard/micropython that referenced this pull request Feb 12, 2024
add OV2640 support to sdkconfig for esp32s3_eye build
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
py-core Relates to py/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants