Skip to content

OverflowError: long int not supported in this build #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
WilliamZh99 opened this issue Apr 9, 2022 · 8 comments · Fixed by #17
Closed

OverflowError: long int not supported in this build #4

WilliamZh99 opened this issue Apr 9, 2022 · 8 comments · Fixed by #17
Assignees

Comments

@WilliamZh99
Copy link

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
code.py output:
Traceback (most recent call last):
File "code.py", line 14, in
OverflowError: long int not supported in this build

Code done running.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 7.2.4 on 2022-03-31; Adafruit QT Py M0 with samd21e18

@caternuson
Copy link
Collaborator

hmmm....may not be able to support the small board builds.

There are several large values needed internally for computing timing values as well as masking.

Adafruit CircuitPython 7.2.5 on 2022-04-06; Adafruit QT Py M0 with samd21e18
>>> macro_period_us = 0x40000000
OverflowError: long int not supported in this build
>>> mask = 0xFFFFFF00
OverflowError: long int not supported in this build
>>> 

These were taken from the STM Arduino driver and the specifics are not documented elsewhere. So would need to figure out how to re-write this code to accommodate the small builds:

macro_period_us = int(2304 * (0x40000000 / osc_freq)) >> 6
if inter_meas == 0:
# continuous mode
timing_budget_us -= 2500
else:
# autonomous mode
timing_budget_us -= 4300
timing_budget_us //= 2
# VL53L4CD_RANGE_CONFIG_A register
ms_byte = 0
timing_budget_us <<= 12
tmp = macro_period_us * 16
ls_byte = int(((timing_budget_us + ((tmp >> 6) >> 1)) / (tmp >> 6)) - 1)
while ls_byte & 0xFFFFFF00 > 0:
ls_byte >>= 1
ms_byte += 1
ms_byte = (ms_byte << 8) + (ls_byte & 0xFF)
self._write_register(_VL53L4CD_RANGE_CONFIG_A, struct.pack(">H", ms_byte))

@tekktrik
Copy link
Member

Is it worth using a guard statement when importing to check for large integer support, and raise a custom exception explaining incompatibility instead?

@caternuson
Copy link
Collaborator

Maybe. The other option here, as suggested by @dhalbert, is to use reduced precision in the math. Just haven't worked out the specifics since there's a bit going on with the computations. But that might be too much of a hack just to accommodate the small int boards?

@dhalbert
Copy link
Contributor

0x40000000 can be expressed exactly as a float. It's 1073741824, which is 2**30. So you could change

macro_period_us = int(2304 * (0x40000000 / osc_freq)) >> 6
to
macro_period_us = int(2304 * (1073741824.0 / osc_freq)) >> 6
since the division is already a float operation anyway. So this wouldn't lose any precision whether or not you have long ints.

@dhalbert
Copy link
Contributor

For the 0xFFFFFF00 mask, I think you could shift 8 bits right before you mask, unless ls_byte is initially computed to be a longint.

@tekktrik
Copy link
Member

I can try some of these changes and check it out.

@caternuson
Copy link
Collaborator

@tekktrik cool. thanks. i think it was some of the lines further down that gave me pause also. and didnt help that these were just translated from other source. i don't think i found any other documentation describing what is being done, which could have also helped to potentially just reimplement in some other way.

@caternuson
Copy link
Collaborator

neat. this got fixed. @FoamyGuy thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants