Use case please, so I can understand the semantics you want.

Basicly, what you implemented using MICROPY_EXTRA_CONSTANTS for native modules (as far as I imagine, I didn't review closely).

Let's say (after #565) for

ITERS = 1000

for i in range(ITERS):
    print(i)

to not provide worst performance by using global var.

The basic usecase is still defining symbolic constants, but get efficiency in library code, etc.

READ = 1
WRITE = 2
...

if op == READ:
...

Okay, so do you still need module.READ to be available, or can it be completely optimised away?

Sure, it should exist, just that var should have flag "const", which would enable any reference to it be replaced with its content, and be subject to further constant folding. Again, that's how I imagine this support for native named constants works, though maybe I'm wrong.

Right.

What if you define constants within a class scope:

class A:
    VAL = const(123)

How does that behave? "As expected" I guess? And can you define consts in functions too?

Well, I so far was thinking about consts on global module scope (just like native consts?). Working with more advanced cases would be interesting after that...

Function level constants would be the easiest (and most efficient) to implement, because they don't need to be also stored as a variable (as it would at module level).

But such are least useful, realistically - constants would be defined most commonly on module level, sometimes class level. I doubt I'd ever want to define function-level constant ;-)

@dpgeorge , let's consider where we should go from here. #661 tries to set a direction and asks few questions. Please be realistic (maybe even a bit conservative ;-) ) answering them. Thanks!

@pfalcon, well, allowing cross-script constants will start to deviate from CPython compatibility. Of course, constants on there own already do (in a way). But changing the way import works seems like a big departure.

To implement cross-script constants, the compiler would have another stage (at the start) which looked only for import statements. These statements would have to be top-level, since we don't know if those within, eg, an if or function call, will ever be imported.

One other way to do it would be deferred compilation: compile a code block (eg function) only when it is first needed. Then, any imports within that block (imports at the top level of that block) are known to be executed and can be imported before compiling the rest of the block, to resolve constants. Problem is that one either needs to keep the uncompiled parse tree around, or remember the file it came from an re-parse when needed.

Whatever we do, nested compiling is going to be costly in RAM. Unless it can "save" state on disk (eg re-read the file).

Other option is as suggested in #266: have a special import form (CPython syntax compatible) that signals that the import should be performed ahead of time.

import const.module

The above has the meaning that module (not const.module) will be imported before the rest of the script is compiled.

Other option is as suggested in #266: have a special import form (CPython syntax compatible) that signals that the import should be performed ahead of time.

import const.module

But this is what will be non-compatible with CPython. What I propose will still be compatible with CPython (if const() is definedm which is easy, because it should come from "from micropython import const").

Note there's a way to make syntax like "import module.consts" be compatible with CPython is to actually have namespace named "consts" in module. That could be dict, but then it would need item, not attribute access. Well, CPy 3.4' Enums are ones again would allow to solve that. So, if uPy treat Enum as true constants with special handling, that would be fully compatible with CPython (which has bloated, not-optimized impl using metaclass on its side (well, I assume ;-) ))

Wrote a long reply, browser crashed. Will post bunch of shorts.

To implement cross-script constants, the compiler would have another stage (at the start) which looked only for import statements.

Problem is that one either needs to keep the uncompiled parse tree around, or remember the file it came from an re-parse when needed.

Yes, that's also what I say in #661 (perhaps as a thoughtstream ;-) ).

Whatever we do, nested compiling is going to be costly in RAM. Unless it can "save" state on disk (eg re-read the file).

Suddenly, this seems like viable idea how to implement it. If we do "depth first" compilation traversal for import statements (not keeping anything in RAM but stack of file names and positions), and save bytecode, then this bytecode can be easily looked into for constants (without doing complete module import).

But the main question #661 poses is: "Should MicroPython go there at all?" Answering "No, enough for bloating compiler - mind-boggling optimizations can be done on host side" would be well understood. After all, there're already enough optimizations in queue, like native and viper modes. But it would be quire ironic if viper code will call runtime to look up consts values from another module ;-).

So, I personally couldn't make a choice (and surely would chicken out from implementing cross-module constants in compiler ;-) ). So, I hope you can make a reasoned decision, and communicate. Of course, either choice will have implications, like "No" will raise priority of #222 even further.

But this is what will be non-compatible with CPython.

Right, yes, I realised just after I wrote that comment that it was stupid. Of course you must have the constants Python-compatible. The trick is to find an idiom that's optimisable.

The current constant support grew out of the necessity to support constants in the inline assembler. So maybe it's worth working on native/viper some more, and see if that gives any hints/constraints on cross-module constant support.

At the moment a const value is still stored in the global dict. Maybe if const identifiers are prefixed by the user with underscore (or double underscore) they can be made "private" to that module by not placing them in the global dict? This would save quite some RAM with drivers (eg https://github.com/micropython/micropython/blob/master/drivers/nrf24l01/nrf24l01.py) that have lots of consts.

With the example provided, it would concern me that people may make private data which really should be public... Double-underscore would seem to fit better with existing Python syntax of private class members, and would be ugly enough to hopefully deter people from using it unless really needed.

There's the question of providing such a mechanism (to have private members, to reduce RAM consumption), and using it in the right way (does constant X need to be used externally?).

Reviving this issue again to discuss how to make const properly compatible with CPython.

Currently uPy recognises <id> = const(<expr>) in the parser as a special pattern and then replaces all occurences of <id> with <expr>, as well as doing the actual assignment. But if <id> starts with an underscore then no assignment is made to the global dict.

"const" is considered a builtin in uPy which makes it incompatible with CPython (without resorting to tricks). It's also incompatibly with uPy itself for builds that don't have const optimisation enabled.

As suggested at the very top of this issue, the right way to do it is have from micropython import const at the start of the script. Then at least one can provide a dummy implementation of const in micropython.py for CPython. And for uPy builds that don't have const optimisation, the micropython module can include a const function that is the identity (returns its argument).

But note that the parser recognises "const" only, not something like "micropython.const", or any other name.

An alternative would be to implicitly make variables const if they are all upper case and start with a double underscore.

+1 for from micropython import const It's more transparent and Pythonic in my opinion.

+1 for from micropython import const though I wonder if it would be better to split the optimizations and decorators for micropython from the micropython module which can be used for mem_info() and stuff too.

@dpgeorge : So, do you propose to just add dummy "const" symbol to micropython module, which is definitely ok, or something beyond that, like a) modify parser so that if "from micropython import const" wasn't done, "const" wasn't treat specially, or maybe b) modify parser even further, so e.g. const subst happened only on RHS (re: issues reported previously)?

though I wonder if it would be better to split the optimizations and decorators for micropython from the micropython module

I think it's fair to have just one uPy-language specific module (ie should be available in all uPy ports) for language features (like const, native, viper) as well as features tightly coupled to the core (eg mem_info, heap_lock, alloc_emg_exc_buf).

So, do you propose to just add dummy "const" symbol to micropython module,

Yes, and it'll be pointing to the identity function, which already exists (mp_identity_obj).

modify parser so that if "from micropython import const" wasn't done, "const" wasn't treat specially

I wouldn't do that at this point, but we could in the future if there was a need.

modify parser even further, so e.g. const subst happened only on RHS (re: issues reported previously)

Substitutions that produce obj.42 or 42 = x are a bug and should probably be fixed anyway.

micropython.const identity function was added in 791b65f.

Const substitution bug (eg obj.42) was fixed in 6d310a5.

Main issues of this ticked have been addressed, there can be constants in Python code and there is support for having constants from external modules via MICROPY_PORT_CONSTANTS.