py/formatfloat: Use pow() to generate base values. #8981
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
d378b2e to
bcfc6f6
Compare
bcfc6f6 to
963e599
Compare
tannewt
added a commit
to tannewt/circuitpython
that referenced
this pull request
Feb 29, 2024
Print output muddies the serial output. Setting warnings to crash could actual raise an exception. Fixes micropython#8981
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
formatfloatneeds to calculate values of 10^X to convert numbers into decimal strings. Formerly, this was done efficiently by multiplying combinations of 10^(2^Y) read from a lookup table. However, this led to small differences from the exact values of 10^X that were calculated byparsenum, which could lead to unexpected results for expressions likeprint(1e23). These were addressed in #8905 by loosening the comparison bounds around exact values of 10^X, which led to slightly aggressive rounding up for some values.This patch instead ensures alignment between 10^X values in both parsing and formatting by using
parsenum'spow(10, e)idiom informatfloat. This eliminates the unexpected rounding-up, and also means we can drop the lookup table. It is, however, presumably slower to compute.It also introduces direct calculation of the power-of-10 exponent from the power-of-2 exponent in the floating-point representation, eliminating much of the search. It still checks a couple of values in case there are edge cases in this calculation.
Note that this patch incorporates trailing-zero mitigation for
parsenumwhich exists as a separate PR #8980; without this modification, some tests including trailing zeros were failing under this change.Signed-off-by: Dan Ellis dan.ellis@gmail.com