Skip to content

bytes objects lack a decode method in smaller non-Express builds #384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gpshead opened this issue Oct 30, 2017 · 8 comments
Closed

bytes objects lack a decode method in smaller non-Express builds #384

gpshead opened this issue Oct 30, 2017 · 8 comments
Assignees
Milestone

Comments

@gpshead
Copy link

gpshead commented Oct 30, 2017

The decode method for converting from bytes to str is missing in non-Express builds. This makes it awkward to take data from I/O and print it. It seems like this should exist, obviously the codec is ignored on a microcontroller but being able to switch immutable data types from one that iterates as ints to one that is printable and iterates as one character strings is desirable. Depending on the micropython gc internals both could share the same data buffer behind the scenes (I haven't looked inside that box yet...).

Workaround, a silly looking: ''.join(chr(b) for b in bytes_data)

Noticed when using the Feather M0 Basic 2.1.0 circuitpython release. I had to work around this when moving code over from where I developed it on an Express.

@tannewt tannewt added this to the 3.0 milestone Oct 31, 2017
@tannewt
Copy link
Member

tannewt commented Oct 31, 2017

This should be as easy as enabling a define in mpconfigport.h

@deshipu
Copy link

deshipu commented Oct 31, 2017

The flag for this is MICROPY_CPYTHON_COMPAT, but the comment next to it says:

// These methods are superfluous in the presence of str() and bytes()
// constructors.

@tannewt
Copy link
Member

tannewt commented Oct 31, 2017

@dhalbert I think this is a good candidate for using some of the space you freed. What do you think?

Anyone know what the code size cost of enabling this is?

@deshipu
Copy link

deshipu commented Oct 31, 2017

But from that comment, wouldn't just str(my_bytes, 'utf-8') and bytes(my_string, 'utf-8') work in place of my_bytes.decode('utf-8') and my_string.encode('utf-8')?

@dhalbert
Copy link
Collaborator

Turning on MICROPY_CPYTHON_COMPAT costs 792 bytes. Right now we have 3912 bytes free in the Gemma build (for example). But we are looking to add framebuf, and we don't know the storage budget needed for 3.0 yet.

@gpshead: is str(some_bytes, 'ascii') ok for you as a workaround?

(Note that str(b'abc') returns "b'abc'": you need the second argument to get 'abc'. However, the value of the second argument is actually ignored right now: no decoding is done.)

@deshipu
Copy link

deshipu commented Oct 31, 2017

I wonder if it would make sense to actually disable MICROPY_CPYTHON_COMPAT in the large build, so as to avoid code that only works on some platforms for no reason.

@gpshead
Copy link
Author

gpshead commented Nov 3, 2017

Workaround str(binary_data, 'ascii') works. Thanks!

As for disabling in the larger build... I'm in favor of that when reasonable workarounds exist (as in this case). The fewer behavior differences between different circuitpython builds the better; though I could just as easily argue the other way with regards to being compatible with Micropython on larger microcontrollers.

@gpshead gpshead closed this as completed Nov 3, 2017
@dhalbert
Copy link
Collaborator

dhalbert commented Jan 9, 2018

I think we should disable this for consistency across atmel-samd, esp8266, and nrf builds, as suggested by @deshipu above. The features added by MICROPY_CPYTHON_COMPAT encompass a bunch of different things, and should be broken down into separate flags: there's even a comment to that effect in py/mpconfig.h. We can wait for MicroPython to do that. If no comments I will go ahead and disable it in all CPy builds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants