py/mkrules.mk: Add MICROPY_PREVIEW_VERSION_2_X. #12645
Conversation
|
Code size report: |
Codecov Report
@@ Coverage Diff @@
## master #12645 +/- ##
=======================================
Coverage 98.39% 98.39%
=======================================
Files 158 158
Lines 20965 20965
=======================================
Hits 20629 20629
Misses 336 336
|
|
This idea of having a feature flag for version 2 / breaking changes is very interesting. It definitely feels much easier to maintain than starting a new development branch, keeping 1.x and 2.0 branches in sync regarding changes that are wanted on both would be a nightmare. My only concern is it might make reviewing charges much harder: For PR's that are a new feature consisting of mostly new code, there will be no problem, feature flag or branch it'll look the same during review. For changes that involve modifying existing code significantly (which will arguably be most "breaking" charges), having a feature flag will mean the PR will basically be a duplication of existing code into #ifdef / #else blocks with the new 2.0 block having the charges - in git diff this will just look like a single large block of new code. If you were instead committing to a 2.0 branch without feature flags the git diff would be showing just changes to existing code - much easier to see the important changes being proposed. Not sure any way to get around this, probably not worth worrying about too much at this point. I for one am very excited to think about bigger / breaking new features! |
The review looks the same, but with the feature flag there's no extra work, whereas with a branch there's the additional overhead of: For PRs that are new features, I expect they should mostly apply to both branches, so I'd prefer not to have to do anything special here.
Yes. I'm definitely aware that some of these preprocessor blocks are going to get pretty gnarly. In many cases though I hope that we will be doing less of "this function has two different implementations depending on the preprocessor" and more "this function just doesn't exist here". e.g. as an example, with the 2.0 flag enabled, the port-specific machine_rtc.c implementations get replaced with a common extmod one, rather than making the relevant methods handle their arguments differently. In particular the case where a PR needs to be 1.x and 2.x aware (I hope these are rare for general PRs, and mostly only apply for a deliberate "splitting the 1.x from the 2.x behavior), but you could imagine e.g. something that is a bugfix that applies to both the 1.x and 2.x code, then the person submitting the PR needs to be aware of both versions, so doing it in the same PR makes sense. The alternative would be that they submit two distinct PRs (one to each branch), or worse, just a single PR and leave it to the maintainers to apply the fix to the other branch.
|
|
I've simplified this a bit by removing the changes to |
There was a problem hiding this comment.
This approach LGTM! I had one nitpicky kinda thought about naming, specifically "is this flag for the 2.x preview or the 2.0 preview?" but honestly that's a pretty irrelevant detail.
To confirm, is the plan for 2.0 something like this?
- Merge this PR,
MICROPY_PREVIEW_VERSION_2_xmacro is off by default. Anyone who wants shiny new preview behaviour builds withMICROPY_PREVIEW_VERSION_2_x=1. - At some point after planned 2.0 breaking changes are done then the macro will flip to on by default. If someone needs to update MicroPython to get a fix but doesn't want to change other code then they can build with
MICROPY_PREVIEW_VERSION_2_x=0instead. - Leading up to the MicroPython 2.0 release, the flag is removed entirely and everything is always 2.x behaviour from then on.
Is that right?
4c5f847 to
b2c7a5d
Compare
Yeah, I dunno. In my mind "2.x" is the series starting with 2.0 but you're right, past 2.0 this flag goes away. I've updated it to be just I considered "MicroPython 2000" but I don't want to make people think of the Python 2.x -> 3.x transition (because this is not like that!).
Exactly. |
|
How many of the changes could you make without breaking 1.x? For example, the non standard cpython functions can be made available on a new module while still existing on the cpython one. Then 1.x releases would support 2.x compatible code and the 2.0 release becomes a clean up release that finds the remainder of code to update. That code can then updated and still run on 1.x in the interim. The flash layout changes could only be triggered after the layout has been changed explicitly. This approach is one we've taken with CircuitPython. One major release series supports both the old and new way. Then you fix everything before the release with only the new way is marked stable. All that fixed code will also run on the stable release that supports both in the interim. |
48964df to
2e069d0
Compare
Yes, we plan to do that, to make most of the changes so that code can be written that runs on both 2.0 and 1.x (for large x). Eg a new But there are some other things that can only be one way or the other. Eg changing the Epoch from 2000 to 1970, and changing the order of elements in the RTC datetime tuple (maybe for the latter we could instead introduce a new method, and deprecate the old one...).
I'm not sure we can make firmware that supports both old and new flash layouts at the same time, because the layout may change the offset/size of the firmware itself. |
|
FYI, after some early feedback on our 9.0 builds we're adding a basic This will be used from C code to warn on upcoming changes. |
That looks pretty good! Note that there is already |
This provides a way to enable features and changes slated for MicroPython 2.x, by running `make MICROPY_PREVIEW_VERSION_2=1`. Also supported for the cmake ports (except Zephyr). This is an alternative to having a 2.x development branch (or equivalently, keeping a 1.x release branch). Any feature or change that needs to be "hidden" until 2.x can use this flag (either in the Makefile or the preprocessor). A good example is changing function arguments or other public API features, in particular to aid in improving consistency between ports. When `MICROPY_PREVIEW_VERSION_2` is enabled, the REPL banner is amended to say "MicroPython (with v2.0 preview) vX.Y.Z", and sys.implementation gets a new field `_v2` set to `True`. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is just scaffolding for now, but the idea is that there should be an addition to this file for every commit that uses the `MICROPY_PREVIEW_VERSION_2` macro. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This is a first step towards planning a "MicroPython 2.0" release.
It wasn't our goal that eventually we'd get tired of doing 1.x releases and at some point the next release would just be 2.0. Rather 2.0 should be an opportunity to re-visit some core things in MicroPython and take a once-a-decade opportunity to learn from previous decisions and change a few things to improve the usability, performance, and functionality.
A very simple example is something like the inconsistency of classes/methods/constants in the machine API (see e.g. #10607 for one that comes up a lot). At some point to fix the inconsistencies, we will need to break some ports to make them match. But this doesn't seem like something we can do in a 1.x release as we strive to ensure that existing code continues to work across minor releases.
Another simple example is changes around improving CPython compatibility, in particular removing/rehoming MicroPython-specific extensions.
A more complicated example is something like mapfs (#8381), which requires a bit of a re-think of how board initialisation and boot works, and for example might mean that we need to adjust flash offsets, which means that updating would require a full flash erase. This might be a good opportunity to also improve consistency across ports too.
Similarly something like the proposed
uevent(#6125) which might allow us to change a few ideas about how events and IRQs work.Anyway, the goal is not big sweeping changes that will require programs to be completely rewritten (we do not want this!), rather to identify the changes we can make that are blocking bigger improvements to the project and bundle them up into one major release. And to do this while providing an easy way to test out or work on these changes alongside the main 1.x development work.
At some point there would be a final 1.x release and the 2.x features would be considered ready, in which case this new flag gets removed and v2.0.0 gets released, along with a list of instructions for how to migrate existing code.
Branches would be simpler in (very) many ways, but also adds a lot of other complexity and overhead for the maintainers to keep them in sync. It is of course possible to raise PRs against the non-default branch. But for example, its nice to just be able to run the CI for 1.x and 2.x in the same branch (as this PR does). Also, it's not too difficult to switch to a branch model if we discover it becomes necessary.
When building with
MICROPY_PREVIEW_VERSION_2_Xenabled, the versioning (see https://github.com/orgs/micropython/discussions/12603) is unchanged, except the banner gets "(with 2.x preview)" appended before the semver. i.e. think of it as being a 1.X build, some number of commits past the tag, but with extra features enabled, rather than having a different version. This can also be access programmatically viasys.implementation._2x(set toTrue, or unset).None of this is set in stone, so feedback welcome!
This provides a way to enable features and changes slated for MicroPython 2.x, by running
make MICROPY_PREVIEW_VERSION_2_X=1. Also supported for the cmake ports (except Zephyr).This is an alternative to having a 2.x development branch (or equivalently, keeping a 1.x release branch). Any feature or change that needs to be "hidden" until 2.x can use this flag (either in the Makefile or the preprocessor).
A good example is changing function arguments or other public API features, in particular to aid in improving consistency between ports.
This work was funded through GitHub Sponsors.