Skip to content

tools/mpremote: Add mpremote mip install to install packages. #9467

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Oct 2, 2022
Merged

tools/mpremote: Add mpremote mip install to install packages. #9467

merged 4 commits into from
Oct 2, 2022

Conversation

jimmo
Copy link
Member

@jimmo jimmo commented Sep 29, 2022

This supports the same package sources as the new mip tool. (micropython/micropython-lib#542)

  • micropython-lib (by name)
  • http(s) & github packages with json description
  • directly downloading a .py/.mpy file

The version is specified with an optional @version on the end of the package name.

Also adds support for optional --args to multi-arg commands like "edit" and the filesystem commands (and now "install"). This allows --mpy/--no-mpy and --target to be specified to the install command.

This work was funded through GitHub Sponsors.

@jimmo
Copy link
Member Author

jimmo commented Sep 29, 2022

This depends on #9463 for the reference :ref:``packages`` to make sense.

@jimmo jimmo force-pushed the mpremote-install branch 3 times, most recently from f5b5cfc to 040f369 Compare September 30, 2022 04:56
@jimmo jimmo mentioned this pull request Sep 30, 2022
Merged
@jimmo jimmo changed the title tools/mpremote: Add mpremote install to install packages. tools/mpremote: Add mpremote mip install to install packages. Sep 30, 2022
@jimmo
Copy link
Member Author

jimmo commented Sep 30, 2022

Updated to make install a subcommand of mip. We will likely add more subcommands of mip in the future (e.g. search, download, etc).

@jimmo jimmo force-pushed the mpremote-install branch 4 times, most recently from 81f118d to 965beb4 Compare September 30, 2022 13:43
@dpgeorge dpgeorge added the tools Relates to tools/ directory in source, or other tooling label Oct 1, 2022
dpgeorge
dpgeorge reviewed Oct 1, 2022
View reviewed changes
tools/mpremote/mpremote/main.py Outdated

# If no commands were "actions" then implicitly finish with the REPL.
if state.run_repl_on_completion():
do_repl(state)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this call fails because it's defined as do_repl(state, args)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accidental cleanup in one of the later updates. Fixed.

@dpgeorge
Copy link
Member

dpgeorge commented Oct 1, 2022

A few things I noticed while testing that were "unexpected":

  • no progress display when mip is copying files
  • mip will redownload and recopy files that already exist on the target

Maybe these are things that will be improved in a future PR?

jimmo added 4 commits October 1, 2022 22:44
@jimmo
tools/mpremote: Simplify dispatch of commands.
413a69b 
No functional change.

This makes each built-in command defined by just a handler method and
simplifies a lot of the logic around tracking the board state.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
@jimmo
tools/mpremote: Use argparse for command line parsing.
68d0943 
No functional change other than to allow slightly more flexibility in how
--foo arguments are specified.

This removes all custom handling for --foo args in all commands and
replaces it with per-command argparse configs.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
@jimmo
tools/mpremote: Add mpremote mip install to install packages.
12ca918 
This supports the same package sources as the new `mip` tool.
 - micropython-lib (by name)
 - http(s) & github packages with json description
 - directly downloading a .py/.mpy file

The version is specified with an optional `@version` on the end of the
package name. The target dir, index, and mpy/no-mpy can be set through
command line args.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
@jimmo
tools/pyboard.py: Handle unsupported fs command.
7705b9b 
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
@jimmo jimmo force-pushed the mpremote-install branch from 965beb4 to 7705b9b Compare October 1, 2022 12:44
@jimmo
Copy link
Member Author

jimmo commented Oct 1, 2022

  • no progress display when mip is copying files

Done. There's now a progress for downloading then another one for copying.

  • mip will redownload and recopy files that already exist on the target

I'm not sure this is important for the PC version of mip. But it would be easy to add in a future PR.

@dpgeorge dpgeorge merged commit 7705b9b into micropython:master Oct 2, 2022
@dpgeorge
Copy link
Member

dpgeorge commented Oct 2, 2022

A great feature, thank you!

@peterhinch
Copy link
Contributor

mip will redownload and recopy files that already exist on the target

In the case where the file(s) have been updated this would be welcome behaviour.

@jimmo
Copy link
Member Author

jimmo commented Oct 3, 2022

mip will redownload and recopy files that already exist on the target

In the case where the file(s) have been updated this would be welcome behaviour.

@peterhinch

Yes, definitely.

What Damien is referring to here though is that mip when running on the device is "smarter" and will only re-download if the file on the device is different to the one in the index. (The index sends a single manifest with the required file hashes for the package and all dependencies, and it computes the hashes and decides what to fetch -- which is to say, if you mip.install("foo") and only a single file in a dependency changed, then there are only two total HTTP requests: the initial manifest, and that one file)

When running in mpremote, mip always just unconditionally downloads everything, and never computes and checks the local on-device hashes. We can definitely add the smarter behavior though if it's useful.

( A note on the list of required hashes for anyone who is curious: the list of hashes is computed at the time the package is published and includes all dependencies. i.e. version x.y.z "freezes" all it's dependencies too at that point in time, so mpremote mip install package@version (or mip.install(package, version=)) will always give you the exact same files, even if a dependency gets updated without bumping its version number. )

tannewt pushed a commit to tannewt/circuitpython that referenced this pull request Aug 8, 2024
@jepler
Merge pull request micropython#9467 from jepler/merge-91x-main
184d8ef 
Merge 9.1.x to main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dpgeorge dpgeorge dpgeorge left review comments

Assignees
No one assigned
Labels
tools Relates to tools/ directory in source, or other tooling
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jimmo @dpgeorge @peterhinch