Skip to content

Conversation

musteresel
Copy link
Contributor

Commits to address #9009

First commit changes py/mkrules.mk to place build outputs ($(PROG) and $(LIBMICROPYTHON)) in the build directory $(BUILD).

Second commit attempts to fix all paths that assumed the build outputs to be in the working directory of the corresponding build, changing these paths to use the binaries from the respective build directory. It also changes the build rule for the mpy-cross dependency! See below for information why.

Third commit was originally made to work around an issue regarding failing builds for teensy and samd; the mpy-cross was no longer automatically generated in those ci builds. It is now not strictly necessary ... but IMO would be clearer to force this build step (as is done in the ci rules for other ports).

Reason for the issues around the mpy-cross build was that I (orignally, not in these commits!) changed the MICROPY_MPYCROSS_DEPENDENCY to point to $(TOP)/mpy-cross/ .. however this directory (ofc) exists, and thus (since MICROPY_MPYCROSS_DEPENDENCY only shows up as an order-only dependency) the rule to build the dependency (using a sub make call) was not run, and thus we had no mpy-cross.
I then set MICROPY_MPYCROSS_DEPENDENCY to $(TOP)/mpy-cross/-non-exist-dummy- .. which "worked" but caused the rule to be run everytime. This caused the windows and qemu builds to fail, because there the additional run of the build for mpy-cross tried to use the cross-compiler ... which complained about wrong file format / target architecture.
So now I changed the build rule to be explicitly about building the mpy-cross from the repository, which seems like the most sane solution here to me.

The rules for lib (static library with name $(LIBMICROPYTHON)) and the
default rule to build a binary ($(PROG)) produced outputs in the
current working directory. Change this to build these files in the
build directory.

Note: An empty BUILD variable can cause issues (references to the root
directory); this is not addressed by this commit due to multiple other
places having the same issue.
Binaries built using the Make build system now no longer appear in the
working directory of the build, but rather in the build directory. Thus
some paths had to be adjusted. Adjust build rule for mpy-cross
dependency.
@musteresel
Copy link
Contributor Author

One additonal thing: This most likely breaks code downstream, which expects the micropython binary (or mpy-cross binary, or libmicropython.a) to show up in the working directory rather than the build directory.

@musteresel
Copy link
Contributor Author

musteresel commented Aug 3, 2022

Added a commit which might fixes the appveyor run .. if not I need help debugging this (where's appveyor.yml?)

@dpgeorge
Copy link
Member

dpgeorge commented Aug 6, 2022

One additonal thing: This most likely breaks code downstream, which expects the micropython binary (or mpy-cross binary, or libmicropython.a) to show up in the working directory rather than the build directory.

Yes this will break my (and probably other's) daily workflow, where I have grown accustomed to the unix (and mpy-cross) executables to be where there currently are.

Although I do agree it's neater (and follows the bare-metal ports output) to put these excutables in the build/ directory... just need to think how big of a breaking change this is.

@jimmo
Copy link
Member

jimmo commented Aug 6, 2022

It also breaks my workflow, but not in a way that I really feel very strongly about and can easy adapt (or symlink around).

Overall I'm in favour of merging this.

@musteresel
Copy link
Contributor Author

musteresel commented Aug 6, 2022

Hmm ... I could add a rule which moves the build output to where it was, an option to turn that rule off, and make the rule enabled by default?
That way someone running into problems with the current setup (for example someone with a readonly source directory) can just use that option?

And one could (step by step) first deprecate the current behaviour, and then some time in the future (when everybody had time to adjust) remove it?

edit, clarification:

  • add a rule which move build output so that after merging this branch, everything still looks the same from the outside.
  • allow this rule to be deactivated. By default the rule however is activated.
  • Add a warning that the future behaviour will change.
  • In future, by default deactivate the rule (but leave the option to activate it)
  • Perhaps, in far future, remove the rule.

@jimmo
Copy link
Member

jimmo commented Aug 6, 2022

I nearly suggested exactly this, but you'd still want to make the other changes to reference the mpy-cross and micropython binaries from the build directories, so I think having two copies of the binary is just a recipe for confusion.

@dpgeorge
Copy link
Member

dpgeorge commented Aug 8, 2022

I think if we are going to merge this PR, then just do it without the backwards compatible copy/link.

Would it make sense to remove the PROG ?= micropython-coverage etc lines from mpconfigvariant.mk files, now that these executables live in build directories with the variant name, eg build-coverage/micropython?

@jimmo
Copy link
Member

jimmo commented Aug 8, 2022

Yes, +1 to that.

@dpgeorge
Copy link
Member

Merged in b2e8240 through c7aa6a2, and -coverage etc suffix removed in d53c3b6

I also reverted some of the changes to the mpy-cross dependency. It should be that if you define MICROPY_MPYCROSS_DEPENDENCY then that disables the automatic building of mpy-cross.

Thank you @musteresel for this contribution, it does clean up things nicely!

@dpgeorge dpgeorge closed this Aug 11, 2022
jimmo added a commit to jimmo/micropython that referenced this pull request Aug 11, 2022
PR micropython#9012 (b2e8240) changed the output to
$(BUILD)/$(PROG) but the tests are still looking for $(PROG).

Also remove the now-unnecessary override of $(PROG) in the standard
variant.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
karfas pushed a commit to karfas/micropython that referenced this pull request Apr 23, 2023
PR micropython#9012 (b2e8240) changed the output to
$(BUILD)/$(PROG) but the tests are still looking for $(PROG).

Also remove the now-unnecessary override of $(PROG) in the standard
variant.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
alphonse82 pushed a commit to alphonse82/micropython-wch-ch32v307 that referenced this pull request May 8, 2023
PR micropython#9012 (b2e8240) changed the output to
$(BUILD)/$(PROG) but the tests are still looking for $(PROG).

Also remove the now-unnecessary override of $(PROG) in the standard
variant.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
RetiredWizard pushed a commit to RetiredWizard/micropython that referenced this pull request Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants