Skip to content

Constants and efficiency #266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dpgeorge opened this issue Feb 9, 2014 · 8 comments
Closed

Constants and efficiency #266

dpgeorge opened this issue Feb 9, 2014 · 8 comments
Labels
rfc Request for Comment

Comments

@dpgeorge
Copy link
Member

dpgeorge commented Feb 9, 2014

Coming out of #227 was a discussion on using constants to specify ports, pins, modes, etc for configuring GPIO and peripherals. For example:

GPIO_PinA4 = 4
GPIO_OutputMode = 0x40

p = pyb.Gpio(GPIO_PinA4, GPIO_OutputMode)

This is not very efficient because the first 2 lines store into the module namespace the 2 constants, taking up RAM. The above could be optimised by the compiler remembering the value of the constants and substituting the value whenever they are used. But this would break general Python semantics (since the globals can be written to elsewhere in the code), and not all global numbers are constants. To fix this one could wrap the number in a special function which declares it as a constant:

GPIO_PinA4 = constant(4)
GPIO_OutputMode = constant(0x40)

Okay, but what about sharing such definitions over multiple Python files? They would need to be put in a separate file (say pyb.py) and imported:

import pyb

p = pyb.Gpio(pyb.GPIO_PinA4, pyb.GPIO_OutputMode)

That can't be optimised because pyb may be assigned to something else during the course of execution. Thus we require something like:

import pyb.constant

and the compiler would recognise this as saying that pyb will not change and so it can lookup constants inside this import. But that requires the compiler having in RAM a very large hash table with all these constants.

@pfalcon
Copy link
Contributor

pfalcon commented Feb 9, 2014

Good write up, close to what I had in mind ;-). So, some additions/ideas.

First of all, if we have very high-level language, then first of all we want clarity and user-friendliness, so we definitely want to have "GPIO_OutputMode" instead of "0x40" as general, user-facing API.

Granted, next thing we may want is to gain some efficiency still. Good way to get good gains is to do static optimization (like constant propagation), but statically optimizing dynamic language means giving up some dynamicity. Let's just accept that, and it will be easier to make choices.

Specific points:

To fix this one could wrap the number in a special function which declares it as a constant:

Yeah, it sucks a bit that python can't annotate statements, or at least declarations, one might hope for following syntax, but alas:

@constant
X = 1

GPIO_OutputMode = constant(0x40)

Yes, then we apparently should go for such syntax, this is similar to how MyPy annotates vars for example: http://www.mypy-lang.org/tutorial.html#expilicitypes

That can't be optimised because pyb may be assigned to something else during the course of execution.

Ok, so code which can be optimized is one which won't play such tricks on modules. The way to implement it can be:

  1. Default run mode is fully CPython-compatible dynamic mode.
  2. Warning mode, which will warn about all non-optimizable things, like: clash in names with "special" identifiers (like modules), using modules as first-class objects (vs just as a literal namespace), assigning to another module's namespace, etc. etc.
  3. Optimization mode, which will just assume that dynamic features are used conservatively, as checked by mode 2.

Also, this stuff don't have to be implemented by on-device compiler - this can be a unix version exporting optimized bytecode, or even separate tool. Though it would be both cool and IMHO possible to support simplest constant propagation by embedded compiler ;-).

@dpgeorge
Copy link
Member Author

dpgeorge commented Feb 9, 2014

Whatever the solution, the same code must work in an equivalent way without optimisation / in CPython.

@pfalcon
Copy link
Contributor

pfalcon commented Feb 9, 2014

Whatever the solution, the same code must work in an equivalent way
without optimisation / in CPython.

Sure, that should be easy.

@pfalcon
Copy link
Contributor

pfalcon commented Feb 9, 2014

Also note that 3.4 introduces Enums with constant values: http://docs.python.org/3.4/library/enum.html . But IMHO, it's a bit overengineered to use just to represent dead simple constants (but I can't be trusted here, I have the same feeling for most features introduced since 1.5.2).

@Neon22
Copy link
Contributor

Neon22 commented Feb 10, 2014

It seems that having cake and eating it too is always a problem :).
But I feel you acknowledge this already with ideas like vipercode.

So I feel its OK for a user to not get superoptimised results ALL the time.
I like approach of

  1. Its in Python - we do what we can but its a dynamic language. Sacrifice speed/size for generality (or whatever description you prefer). Your RAM may overflow.
  2. If you follow these guidelines (TBD) then your code will be optimal in python and in the embedded for RAM use. (Maybe other sets of rules for optimal program size, or optimal GC use, etc).
  3. If you run code through this preprocessor getting python as a result - then your code will be minimal but still Python (My vulture/ropetest horse - which I am in danger of beating to death.. :))
  4. Use the Viper or other options to make it as fast and dense as possible. No longer python - really you have used Python to author it into C code or assembler but there will be issues like (TBD) which this will not work well with (e.g. writing self modifying python code or classes?).
  5. It still may not fit or be too slow - you may need a more powerful platform.
  • Which I feel we can help people with by indicating helpful things like RAM usage, Benchmarks, profiling(runtime), ...
    1. YMMV.

But I certainly don't want to halt the discussion...

@Neon22
Copy link
Contributor

Neon22 commented Feb 10, 2014

@pfalcon I missed the enums but actually - even though I agree with you about 1.5.2 - this particular class is useful for allowing the definition in pure python of constants like this so it will run the same in Python, Cpython, and embedded. I wonder if we can't take advantage of these people shoving C constructs into Python (which as a LISP guy makes me physically ill) to use to our advantage.

I.e use it specifically to define pins for example. Good examples of IntEnum and the planet inheritance example on the enum page...

@pfalcon pfalcon added the rfc label Feb 10, 2014
@lurch
Copy link
Contributor

lurch commented Apr 11, 2014

Probably doesn't help, but https://docs.python.org/3/library/constants.html has a small section talking about “true” constants.

Edit: And just to keep all the cross-references in place: 57e99eb

dpgeorge added a commit that referenced this issue May 8, 2014
You can now do:

    X = const(123)
    Y = const(456 + X)

and the compiler will replace X and Y with their values.

See discussion in issue #266 and issue #573.
@dpgeorge
Copy link
Member Author

dpgeorge commented Jun 7, 2017

Constants have been supported for a long time now, via from micropython import const and X = const(123). The behaviour is compatible with CPython.

@dpgeorge dpgeorge closed this as completed Jun 7, 2017
tannewt added a commit to tannewt/circuitpython that referenced this issue Oct 9, 2017
This allows us to re-enable `os`. `random` is also enabled because
it solely depends on `os`.

Fixes micropython#266. Its also a pre-requisite for micropython#260.
tannewt added a commit to tannewt/circuitpython that referenced this issue Oct 14, 2017
* atmel-samd: Add support for internal filesystems.

This allows us to re-enable `os`. `random` is also enabled because
it solely depends on `os`.

Fixes micropython#266. Its also a pre-requisite for micropython#260.

* atmel-samd: Update SAMD51 linker script comments and MICROPY_MAX_STACK_USAGE enabling.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rfc Request for Comment
Projects
None yet
Development

No branches or pull requests

4 participants