diff --git a/.git-blame-ignore-revs b/.git-blame-ignore-revs index 7b63f7549d46d..195e1e06eea46 100644 --- a/.git-blame-ignore-revs +++ b/.git-blame-ignore-revs @@ -1,3 +1,21 @@ +# all: Prune trailing whitespace. +dda9b9c6da5d3c31fa8769e581a753e95a270803 + +# all: Remove the "STATIC" macro and just use "static" instead. +decf8e6a8bb940d5829ca3296790631fcece7b21 + +# renesas-ra: Fix spelling mistakes found by codespell. +b3f2f18f927fa2fad10daf63d8c391331f5edf58 + +# all: Update Python formatting to ruff-format. +bbd8760bd9a2302e5abee29db279102bb11d7732 + +# all: Fix various spelling mistakes found by codespell 2.2.6. +cf490a70917a1b2d38ba9b58e763e0837d0f7ca7 + +# all: Fix spelling mistakes based on codespell check. +b1229efbd1509654dec6053865ab828d769e29db + # top: Update Python formatting to black "2023 stable style". 8b2748269244304854b3462cb8902952b4dcb892 diff --git a/.gitattributes b/.gitattributes index e6d31d6aa31fc..2d8496db50488 100644 --- a/.gitattributes +++ b/.gitattributes @@ -8,10 +8,12 @@ # These are binary so should never be modified by git. *.a binary +*.ico binary *.png binary *.jpg binary *.dxf binary *.mpy binary +*.der binary # These should also not be modified by git. tests/basics/string_cr_conversion.py -text diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md deleted file mode 100644 index 7bad9562964af..0000000000000 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -name: Bug report -about: Report an issue -title: '' -labels: bug -assignees: '' - ---- - -* Please search existing issues before raising a new issue. For questions about MicroPython or for help using MicroPython, or any sort of "how do I?" requests, please use the Discussions tab or raise a documentation request instead. - -* In your issue, please include a clear and concise description of what the bug is, the expected output, and how to replicate it. - -* If this issue involves external hardware, please include links to relevant datasheets and schematics. - -* If you are seeing code being executed incorrectly, please provide a minimal example and expected output (e.g. comparison to CPython). - -* For build issues, please include full details of your environment, compiler versions, command lines, and build output. - -* Please provide as much information as possible about the version of MicroPython you're running, such as: - - firmware file name - - git commit hash and port/board - - version information shown in the REPL (hit Ctrl-B to see the startup message) - -* Remove all placeholder text above before submitting. diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml new file mode 100644 index 0000000000000..1ec6c7067f9cd --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -0,0 +1,109 @@ +name: Bug report +description: Report a bug or unexpected behaviour +labels: ["bug"] +body: + - type: markdown + attributes: + value: | + Please provide as much detail as you can, it really helps us find and fix bugs faster. + + #### Not a bug report? + + * If you have a question \"How Do I ...?\", please post it on [GitHub Discussions](https://github.com/orgs/micropython/discussions/) or [Discord](https://discord.gg/RB8HZSAExQ) instead of here. + * For missing or incorrect documentation, or feature requests, then please [choose a different issue type](https://github.com/micropython/micropython/issues/new/choose). + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) matching this bug before reporting. + - type: input + id: port-board-hw + attributes: + label: Port, board and/or hardware + description: | + Which MicroPython port(s) and board(s) are you using? + placeholder: | + esp32 port, ESP32-Fantastic board. + validations: + required: true + - type: textarea + id: version + attributes: + label: MicroPython version + description: | + To find the version: + + 1. Open a serial REPL. + 2. Type Ctrl-B to see the startup message. + 3. Copy-paste that output here. + + If the issue is about building MicroPython, please provide output of `git describe --dirty` and as much information as possible about the build environment. + + If the version or configuration is modified from the official MicroPython releases or the master branch, please tell us the details of this as well. + placeholder: | + MicroPython v6.28.3 on 2029-01-23; PyBoard 9 with STM32F9 + validations: + required: true + - type: textarea + id: steps-reproduce + attributes: + label: Reproduction + description: | + What steps will reproduce the problem? Please include all details that could be relevant about the environment, configuration, etc. + + If there is Python code to reproduce this issue then please either: + a. Type it into a code block below ([code block guide](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks)), or + b. Post longer code to a [GitHub gist](https://gist.github.com/), or + c. Create a sample project on GitHub. + + For build issues, please provide the exact build commands that you ran. + placeholder: | + 1. Copy paste the code provided below into a new file + 2. Use `mpremote run` to execute it on the board. + validations: + required: true + - type: textarea + id: expected + attributes: + label: Expected behaviour + description: | + What did you expect MicroPython to do? If comparing output with CPython or a different MicroPython port/version then please provide that output here. + placeholder: | + Expected to print "Hello World". + + Here is the correct output, seen with previous MicroPython version v3.14.159: + + > [...] + - type: textarea + id: what-happened + attributes: + label: Observed behaviour + description: | + What actually happened? Where possible please paste exact output, or the complete build log, etc. Very long output can be linked in a [GitHub gist](https://gist.github.com/). + placeholder: | + This unexpected exception appears: + + > [...] + validations: + required: true + - type: textarea + id: additional + attributes: + label: Additional Information + description: | + Is there anything else that might help to resolve this issue? + value: No, I've provided everything above. + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true + - type: markdown + attributes: + value: | + Thanks for taking the time to help improve MicroPython. diff --git a/.github/ISSUE_TEMPLATE/documentation.md b/.github/ISSUE_TEMPLATE/documentation.md deleted file mode 100644 index e36fa62ac29a4..0000000000000 --- a/.github/ISSUE_TEMPLATE/documentation.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -name: Documentation issue -about: Report areas of the documentation or examples that need improvement -title: 'docs: ' -labels: documentation -assignees: '' - ---- - -* Please search existing issues before raising a new issue. For questions about MicroPython or for help using MicroPython, or any sort of "how do I?" requests, please use the Discussions tab instead. - -* Describe what was missing from the documentation and/or what was incorrect/incomplete. - -* If possible, please link to the relevant page on https://docs.micropython.org/ - -* Remove all placeholder text above before submitting. diff --git a/.github/ISSUE_TEMPLATE/documentation.yml b/.github/ISSUE_TEMPLATE/documentation.yml new file mode 100644 index 0000000000000..93051e51c80c7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/documentation.yml @@ -0,0 +1,46 @@ +name: Documentation issue +description: Report areas of the documentation or examples that need improvement +title: "docs: " +labels: ["documentation"] +body: + - type: markdown + attributes: + value: | + This form is for reporting issues with the documentation or examples provided with MicroPython. + + If you have a general question \"How Do I ...?\", please post it on [GitHub Discussions](https://github.com/orgs/micropython/discussions/) or [Discord](https://discord.gg/RB8HZSAExQ) instead of here. + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) before reporting a new one. + - type: input + id: page + attributes: + label: Documentation URL + description: | + Does this issue relate to a particular page in the [online documentation](https://docs.micropython.org/en/latest/)? If yes, please paste the URL of the page: + placeholder: | + https://docs.micropython.org/en/latest/ + - type: textarea + id: version + attributes: + label: Description + description: | + Please describe what was missing from the documentation and/or what was incorrect/incomplete. + validations: + required: true + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true + - type: markdown + attributes: + value: | + Thanks for taking the time to help improve MicroPython. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md deleted file mode 100644 index 81b55d98e0da8..0000000000000 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -name: Feature request -about: Request a feature or improvement -title: '' -labels: enhancement -assignees: '' - ---- - -* Please search existing issues before raising a new issue. For questions about MicroPython or for help using MicroPython, or any sort of "how do I?" requests, please use the Discussions tab or raise a documentation request instead. - -* Describe the feature you'd like to see added to MicroPython. In particular, what does this feature enable and why is it useful. MicroPython aims to strike a balance between functionality and code size, so please consider whether this feature can be optionally enabled and whether it can be provided in other ways (e.g. pure-Python library). - -* For core Python features, where possible please include a link to the relevant PEP. - -* For new architectures / ports / boards, please provide links to relevant documentation, specifications, and toolchains. Any information about the popularity and unique features about this hardware would also be useful. - -* For features for existing ports (e.g. new peripherals or microcontroller features), please describe which port(s) it applies too, and whether this is could be an extension to the machine API or a port-specific module? - -* For drivers (e.g. for external hardware), please link to datasheets and/or existing drivers from other sources. - -* Who do you expect will implement the feature you are requesting? Would you be willing to sponsor this work? - -* Remove all placeholder text above before submitting. diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml new file mode 100644 index 0000000000000..7d5162a32a4af --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -0,0 +1,74 @@ +name: Feature request +description: Request a feature or improvement +labels: ['enhancement'] +body: + - type: markdown + attributes: + value: | + This form is for requesting features or improvements in MicroPython. + + #### Get feedback first + + Before submitting a new feature idea here, suggest starting a discussion on [Discord](https://discord.gg/RB8HZSAExQ) or [GitHub Discussions](https://github.com/orgs/micropython/discussions/) to get early feedback from the community and maintainers. + + #### Not a MicroPython core feature? + + * If you have a question \"How Do I ...?\", please post it on GitHub Discussions or Discord instead of here. + * Could this feature be implemented as a pure Python library? If so, please open the request on the [micropython-lib repository](https://github.com/micropython/micropython-lib/issues) instead. + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) before opening a new one. + - type: textarea + id: feature + attributes: + label: Description + description: | + Describe the feature you'd like to see added to MicroPython. What does this feature enable and why is it useful? + + * For core Python features, where possible please include a link to the relevant PEP or CPython documentation. + * For new architectures / ports / boards, please provide links to relevant documentation, specifications, and toolchains. Any information about the popularity and unique features about this hardware would also be useful. + * For features for existing ports (e.g. new peripherals or microcontroller features), please describe which port(s) it applies to, and whether this is could be an extension to the machine API or a port-specific module? + * For drivers (e.g. for external hardware), please link to datasheets and/or existing drivers from other sources. + + If there is an existing discussion somewhere about this feature, please add a link to it as well. + validations: + required: true + - type: textarea + id: size + attributes: + label: Code Size + description: | + MicroPython aims to strike a balance between functionality and code size. Can this feature be optionally enabled? + + If you believe the usefulness of this feature would outweigh the additional code size, please explain. (It's OK to say you're unsure here, we're happy to discuss this with you.) + - type: dropdown + id: implementation + attributes: + label: Implementation + description: | + What is your suggestion for implementing this feature? + + (See also: [How to sponsor](https://github.com/sponsors/micropython#sponsors), [How to submit a Pull Request](https://github.com/micropython/micropython/wiki/ContributorGuidelines).) + options: + - I hope the MicroPython maintainers or community will implement this feature + - I intend to implement this feature and would submit a Pull Request if desirable + - I would like to sponsor development of this feature + multiple: true + validations: + required: true + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true + - type: markdown + attributes: + value: | + Thanks for taking the time to suggest improvements for MicroPython. diff --git a/.github/ISSUE_TEMPLATE/security.md b/.github/ISSUE_TEMPLATE/security.md deleted file mode 100644 index 2bbfede6ce069..0000000000000 --- a/.github/ISSUE_TEMPLATE/security.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -name: Security report -about: Report a security issue or vunerability in MicroPython -title: '' -labels: security -assignees: '' - ---- - -* If you need to raise this issue privately with the MicroPython team, please email contact@micropython.org instead. - -* Include a clear and concise description of what the security issue is. - -* What does this issue allow an attacker to do? - -* Remove all placeholder text above before submitting. diff --git a/.github/ISSUE_TEMPLATE/security.yml b/.github/ISSUE_TEMPLATE/security.yml new file mode 100644 index 0000000000000..57c2a5885ed85 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/security.yml @@ -0,0 +1,60 @@ +name: Security report +description: Report a security issue or vulnerability in MicroPython +labels: ["security"] +body: + - type: markdown + attributes: + value: | + This form is for reporting security issues in MicroPython that are not readily exploitable. + + 1. For issues that are readily exploitable or have high impact, please email contact@micropython.org instead. + 1. If this is a question about security, please ask it in [Discussions](https://github.com/orgs/micropython/discussions/) or [Discord](https://discord.gg/RB8HZSAExQ) instead. + + #### Existing issue? + + * Please search for [existing issues](https://github.com/micropython/micropython/issues) before reporting a new one. + + - type: input + id: port-board-hw + attributes: + label: Port, board and/or hardware + description: | + Which MicroPython port(s) and board(s) are you using? + placeholder: | + esp32 port, ESP32-Duper board. + - type: textarea + id: version + attributes: + label: MicroPython version + description: | + To find the version: + + 1. Open a serial REPL. + 2. Type Ctrl-B to see the startup message. + 3. Copy-paste that output here. + + If the version or configuration is modified from the official MicroPython releases or the master branch, please tell us the details of this as well. + placeholder: | + MicroPython v6.28.3 on 2029-01-23; PyBoard 9 with STM32F9 + - type: textarea + id: report + attributes: + label: Issue Report + description: | + Please provide a clear and concise description of the security issue. + + * What does this issue allow an attacker to do? + * How does the attacker exploit this issue? + validations: + required: true + - type: dropdown + id: code-of-conduct + attributes: + label: Code of Conduct + description: | + Do you agree to follow the MicroPython [Code of Conduct](https://github.com/micropython/micropython/blob/master/CODEOFCONDUCT.md) to ensure a safe and respectful space for everyone? + options: + - "Yes, I agree" + multiple: true + validations: + required: true diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000000000..e11cebddb3710 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,33 @@ + + +### Summary + + + + +### Testing + + + + +### Trade-offs and Alternatives + + + diff --git a/.github/workflows/biome.yml b/.github/workflows/biome.yml new file mode 100644 index 0000000000000..88744f16ca7d6 --- /dev/null +++ b/.github/workflows/biome.yml @@ -0,0 +1,16 @@ +name: JavaScript code lint and formatting with Biome + +on: [push, pull_request] + +jobs: + eslint: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + - name: Setup Biome + uses: biomejs/setup-biome@v2 + with: + version: 1.5.3 + - name: Run Biome + run: biome ci --indent-style=space --indent-width=4 tests/ ports/webassembly diff --git a/.github/workflows/code_formatting.yml b/.github/workflows/code_formatting.yml index 542edd46cfa20..9f30f048cfdbe 100644 --- a/.github/workflows/code_formatting.yml +++ b/.github/workflows/code_formatting.yml @@ -7,14 +7,14 @@ concurrency: cancel-in-progress: true jobs: - build: + code-formatting: runs-on: ubuntu-22.04 steps: - - uses: actions/checkout@v3 - - uses: actions/setup-python@v4 + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 - name: Install packages - run: source tools/ci.sh && ci_code_formatting_setup + run: source tools/ci.sh && ci_c_code_formatting_setup - name: Run code formatting - run: source tools/ci.sh && ci_code_formatting_run + run: source tools/ci.sh && ci_c_code_formatting_run - name: Check code formatting run: git diff --exit-code diff --git a/.github/workflows/code_size.yml b/.github/workflows/code_size.yml index 5d955703b666a..67261933798cb 100644 --- a/.github/workflows/code_size.yml +++ b/.github/workflows/code_size.yml @@ -1,16 +1,21 @@ name: Check code size on: - push: pull_request: paths: - '.github/workflows/*.yml' - 'tools/**' - 'py/**' - 'extmod/**' + - 'shared/**' - 'lib/**' - 'ports/bare-arm/**' + - 'ports/mimxrt/**' - 'ports/minimal/**' + - 'ports/rp2/**' + - 'ports/samd/**' + - 'ports/stm32/**' + - 'ports/unix/**' concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -18,9 +23,9 @@ concurrency: jobs: build: - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 with: fetch-depth: 100 - name: Install packages @@ -36,7 +41,7 @@ jobs: run: echo $PR_NUMBER > pr_number - name: Upload diff if: github.event_name == 'pull_request' - uses: actions/upload-artifact@v3 + uses: actions/upload-artifact@v4 with: name: code-size-report path: | diff --git a/.github/workflows/code_size_comment.yml b/.github/workflows/code_size_comment.yml index 8baf76a47a7f6..521afad709d17 100644 --- a/.github/workflows/code_size_comment.yml +++ b/.github/workflows/code_size_comment.yml @@ -11,11 +11,11 @@ concurrency: jobs: comment: - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 steps: - name: 'Download artifact' id: download-artifact - uses: actions/github-script@v6 + uses: actions/github-script@v7 with: result-encoding: string script: | @@ -56,7 +56,7 @@ jobs: run: unzip code-size-report.zip - name: Post comment to pull request if: steps.download-artifact.outputs.result == 'ok' - uses: actions/github-script@v6 + uses: actions/github-script@v7 with: github-token: ${{secrets.GITHUB_TOKEN}} script: | diff --git a/.github/workflows/codespell.yml b/.github/workflows/codespell.yml new file mode 100644 index 0000000000000..1d6b1dc9d2b27 --- /dev/null +++ b/.github/workflows/codespell.yml @@ -0,0 +1,13 @@ +name: Check spelling with codespell + +on: [push, pull_request] + +jobs: + codespell: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + # codespell version should be kept in sync with .pre-commit-config.yml + - run: pip install --user codespell==2.4.1 tomli + - run: codespell + diff --git a/.github/workflows/commit_formatting.yml b/.github/workflows/commit_formatting.yml index 0b27038f2d264..fcbcaa7092ee2 100644 --- a/.github/workflows/commit_formatting.yml +++ b/.github/workflows/commit_formatting.yml @@ -1,6 +1,6 @@ name: Check commit message formatting -on: [push, pull_request] +on: [pull_request] concurrency: group: ${{ github.workflow }}-${{ github.ref }} @@ -10,9 +10,9 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 with: - fetch-depth: '100' - - uses: actions/setup-python@v4 + fetch-depth: 100 + - uses: actions/setup-python@v5 - name: Check commit message formatting run: source tools/ci.sh && ci_commit_formatting_run diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index e9b17007476c1..d01a4b50c9810 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -1,6 +1,7 @@ name: Build docs on: + push: pull_request: paths: - docs/** @@ -14,9 +15,9 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 - - uses: actions/setup-python@v4 + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 - name: Install Python packages - run: pip install Sphinx + run: pip install -r docs/requirements.txt - name: Build docs run: make -C docs/ html diff --git a/.github/workflows/examples.yml b/.github/workflows/examples.yml index 450805a6bde41..6613f106625a2 100644 --- a/.github/workflows/examples.yml +++ b/.github/workflows/examples.yml @@ -18,7 +18,7 @@ jobs: embedding: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Build run: make -C examples/embedding -f micropython_embed.mk && make -C examples/embedding - name: Run diff --git a/.github/workflows/mpremote.yml b/.github/workflows/mpremote.yml index 3cfb9d47d39f2..ee91b6360b9b4 100644 --- a/.github/workflows/mpremote.yml +++ b/.github/workflows/mpremote.yml @@ -11,20 +11,18 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 with: - # Version is determined from git, - # should be deep enough to get to latest tag - fetch-depth: '1000' - - run: | - git fetch --prune --unshallow --tags - - uses: actions/setup-python@v4 + # Setting this to zero means fetch all history and tags, + # which hatch-vcs can use to discover the version tag. + fetch-depth: 0 + - uses: actions/setup-python@v5 - name: Install build tools run: pip install build - name: Build mpremote wheel run: cd tools/mpremote && python -m build --wheel - name: Archive mpremote wheel - uses: actions/upload-artifact@v3 + uses: actions/upload-artifact@v4 with: name: mpremote path: | diff --git a/.github/workflows/mpy_format.yml b/.github/workflows/mpy_format.yml index 66abb19b81a46..b6768a46c3cdc 100644 --- a/.github/workflows/mpy_format.yml +++ b/.github/workflows/mpy_format.yml @@ -15,9 +15,9 @@ concurrency: jobs: test: - runs-on: ubuntu-20.04 # use 20.04 to get python2 + runs-on: ubuntu-22.04 # use 22.04 to get python2 steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_mpy_format_setup - name: Test mpy-tool.py diff --git a/.github/workflows/ports.yml b/.github/workflows/ports.yml index fb574ad981908..1f262b0ba4bee 100644 --- a/.github/workflows/ports.yml +++ b/.github/workflows/ports.yml @@ -17,6 +17,6 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Build ports download metadata run: mkdir boards && ./tools/autobuild/build-downloads.py . ./boards diff --git a/.github/workflows/ports_alif.yml b/.github/workflows/ports_alif.yml new file mode 100644 index 0000000000000..0e96e7d816e50 --- /dev/null +++ b/.github/workflows/ports_alif.yml @@ -0,0 +1,33 @@ +name: alif port + +on: + push: + pull_request: + paths: + - '.github/workflows/*.yml' + - 'tools/**' + - 'py/**' + - 'extmod/**' + - 'shared/**' + - 'lib/**' + - 'drivers/**' + - 'ports/alif/**' + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + build_alif: + strategy: + fail-fast: false + matrix: + ci_func: # names are functions in ci.sh + - alif_ae3_build + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Install packages + run: source tools/ci.sh && ci_alif_setup + - name: Build ci_${{matrix.ci_func }} + run: source tools/ci.sh && ci_${{ matrix.ci_func }} diff --git a/.github/workflows/ports_cc3200.yml b/.github/workflows/ports_cc3200.yml index b58bc24b58b0a..f178a140587db 100644 --- a/.github/workflows/ports_cc3200.yml +++ b/.github/workflows/ports_cc3200.yml @@ -21,7 +21,7 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_cc3200_setup - name: Build diff --git a/.github/workflows/ports_esp32.yml b/.github/workflows/ports_esp32.yml index 6fc009d4fee83..4c07f89437757 100644 --- a/.github/workflows/ports_esp32.yml +++ b/.github/workflows/ports_esp32.yml @@ -18,20 +18,40 @@ concurrency: cancel-in-progress: true jobs: - build_idf402: - runs-on: ubuntu-20.04 + build_idf: + strategy: + fail-fast: false + matrix: + ci_func: # names are functions in ci.sh + - esp32_build_cmod_spiram_s2 + - esp32_build_s3_c3 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 - - name: Install packages - run: source tools/ci.sh && ci_esp32_idf402_setup - - name: Build - run: source tools/ci.sh && ci_esp32_build - - build_idf44: - runs-on: ubuntu-20.04 - steps: - - uses: actions/checkout@v3 - - name: Install packages - run: source tools/ci.sh && ci_esp32_idf44_setup - - name: Build - run: source tools/ci.sh && ci_esp32_build + - uses: actions/checkout@v4 + + - id: idf_ver + name: Read the ESP-IDF version (including Python version) + run: source tools/ci.sh && echo "IDF_VER=${IDF_VER}-py${PYTHON_VER}" | tee "$GITHUB_OUTPUT" + + - name: Cached ESP-IDF install + id: cache_esp_idf + uses: actions/cache@v4 + with: + path: | + ./esp-idf/ + ~/.espressif/ + !~/.espressif/dist/ + ~/.cache/pip/ + key: esp-idf-${{ steps.idf_ver.outputs.IDF_VER }} + + - name: Install ESP-IDF packages + if: steps.cache_esp_idf.outputs.cache-hit != 'true' + run: source tools/ci.sh && ci_esp32_idf_setup + + - name: ccache + uses: hendrikmuhs/ccache-action@v1.2 + with: + key: esp32-${{ matrix.ci_func }} + + - name: Build ci_${{matrix.ci_func }} + run: source tools/ci.sh && ci_${{ matrix.ci_func }} diff --git a/.github/workflows/ports_esp8266.yml b/.github/workflows/ports_esp8266.yml index ba89d5e9529eb..5236edf40b959 100644 --- a/.github/workflows/ports_esp8266.yml +++ b/.github/workflows/ports_esp8266.yml @@ -21,7 +21,7 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_esp8266_setup && ci_esp8266_path >> $GITHUB_PATH - name: Build diff --git a/.github/workflows/ports_mimxrt.yml b/.github/workflows/ports_mimxrt.yml index d91562534188a..7743e036ab377 100644 --- a/.github/workflows/ports_mimxrt.yml +++ b/.github/workflows/ports_mimxrt.yml @@ -19,9 +19,14 @@ concurrency: jobs: build: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest + defaults: + run: + working-directory: 'micropython repo' # test build with space in path steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 + with: + path: 'micropython repo' - name: Install packages run: source tools/ci.sh && ci_mimxrt_setup - name: Build diff --git a/.github/workflows/ports_nrf.yml b/.github/workflows/ports_nrf.yml index 89211217800b0..76727c9d1f6bd 100644 --- a/.github/workflows/ports_nrf.yml +++ b/.github/workflows/ports_nrf.yml @@ -19,9 +19,9 @@ concurrency: jobs: build: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_nrf_setup - name: Build diff --git a/.github/workflows/ports_powerpc.yml b/.github/workflows/ports_powerpc.yml index a15c4da97fa6a..c41b13e5ddffe 100644 --- a/.github/workflows/ports_powerpc.yml +++ b/.github/workflows/ports_powerpc.yml @@ -21,7 +21,7 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_powerpc_setup - name: Build diff --git a/.github/workflows/ports_qemu-arm.yml b/.github/workflows/ports_qemu-arm.yml deleted file mode 100644 index 93ec4da76700e..0000000000000 --- a/.github/workflows/ports_qemu-arm.yml +++ /dev/null @@ -1,32 +0,0 @@ -name: qemu-arm port - -on: - push: - pull_request: - paths: - - '.github/workflows/*.yml' - - 'tools/**' - - 'py/**' - - 'extmod/**' - - 'shared/**' - - 'lib/**' - - 'drivers/**' - - 'ports/qemu-arm/**' - - 'tests/**' - -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -jobs: - build_and_test: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v3 - - name: Install packages - run: source tools/ci.sh && ci_qemu_arm_setup - - name: Build and run test suite - run: source tools/ci.sh && ci_qemu_arm_build - - name: Print failures - if: failure() - run: grep --before-context=100 --text "FAIL" ports/qemu-arm/build/console.out diff --git a/.github/workflows/ports_qemu.yml b/.github/workflows/ports_qemu.yml new file mode 100644 index 0000000000000..ac09dde86408b --- /dev/null +++ b/.github/workflows/ports_qemu.yml @@ -0,0 +1,51 @@ +name: qemu port + +on: + push: + pull_request: + paths: + - '.github/workflows/*.yml' + - 'tools/**' + - 'py/**' + - 'extmod/**' + - 'shared/**' + - 'lib/**' + - 'drivers/**' + - 'ports/qemu/**' + - 'tests/**' + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + build_and_test_arm: + strategy: + fail-fast: false + matrix: + ci_func: # names are functions in ci.sh + - bigendian + - sabrelite + - thumb + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Install packages + run: source tools/ci.sh && ci_qemu_setup_arm + - name: Build and run test suite ci_qemu_build_arm_${{ matrix.ci_func }} + run: source tools/ci.sh && ci_qemu_build_arm_${{ matrix.ci_func }} + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + build_and_test_rv32: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Install packages + run: source tools/ci.sh && ci_qemu_setup_rv32 + - name: Build and run test suite + run: source tools/ci.sh && ci_qemu_build_rv32 + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ports_renesas-ra.yml b/.github/workflows/ports_renesas-ra.yml index 33e17a385a1e3..b9fa74331dc0b 100644 --- a/.github/workflows/ports_renesas-ra.yml +++ b/.github/workflows/ports_renesas-ra.yml @@ -19,9 +19,9 @@ concurrency: jobs: build_renesas_ra_board: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_renesas_ra_setup - name: Build diff --git a/.github/workflows/ports_rp2.yml b/.github/workflows/ports_rp2.yml index f042ff1151ab1..748f38e143893 100644 --- a/.github/workflows/ports_rp2.yml +++ b/.github/workflows/ports_rp2.yml @@ -20,8 +20,13 @@ concurrency: jobs: build: runs-on: ubuntu-latest + defaults: + run: + working-directory: 'micropython repo' # test build with space in path steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 + with: + path: 'micropython repo' - name: Install packages run: source tools/ci.sh && ci_rp2_setup - name: Build diff --git a/.github/workflows/ports_samd.yml b/.github/workflows/ports_samd.yml index 9833a2fae2ef9..5bf1826cd17bb 100644 --- a/.github/workflows/ports_samd.yml +++ b/.github/workflows/ports_samd.yml @@ -21,7 +21,7 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_samd_setup - name: Build diff --git a/.github/workflows/ports_stm32.yml b/.github/workflows/ports_stm32.yml index b278ea862ce61..8800f145189b8 100644 --- a/.github/workflows/ports_stm32.yml +++ b/.github/workflows/ports_stm32.yml @@ -18,20 +18,19 @@ concurrency: cancel-in-progress: true jobs: - build_pyb: - runs-on: ubuntu-20.04 + build_stm32: + strategy: + fail-fast: false + matrix: + ci_func: # names are functions in ci.sh + - stm32_pyb_build + - stm32_nucleo_build + - stm32_misc_build + runs-on: ubuntu-22.04 steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_stm32_setup - - name: Build - run: source tools/ci.sh && ci_stm32_pyb_build + - name: Build ci_${{matrix.ci_func }} + run: source tools/ci.sh && ci_${{ matrix.ci_func }} - build_nucleo: - runs-on: ubuntu-20.04 - steps: - - uses: actions/checkout@v3 - - name: Install packages - run: source tools/ci.sh && ci_stm32_setup - - name: Build - run: source tools/ci.sh && ci_stm32_nucleo_build diff --git a/.github/workflows/ports_teensy.yml b/.github/workflows/ports_teensy.yml deleted file mode 100644 index f1299603259a6..0000000000000 --- a/.github/workflows/ports_teensy.yml +++ /dev/null @@ -1,28 +0,0 @@ -name: teensy port - -on: - push: - pull_request: - paths: - - '.github/workflows/*.yml' - - 'tools/**' - - 'py/**' - - 'extmod/**' - - 'shared/**' - - 'lib/**' - - 'drivers/**' - - 'ports/teensy/**' - -concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -jobs: - build: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v3 - - name: Install packages - run: source tools/ci.sh && ci_teensy_setup - - name: Build - run: source tools/ci.sh && ci_teensy_build diff --git a/.github/workflows/ports_unix.yml b/.github/workflows/ports_unix.yml index fe25deb4ad51d..4b22926eaf8e5 100644 --- a/.github/workflows/ports_unix.yml +++ b/.github/workflows/ports_unix.yml @@ -23,7 +23,7 @@ jobs: minimal: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Build run: source tools/ci.sh && ci_unix_minimal_build - name: Run main test suite @@ -35,7 +35,7 @@ jobs: reproducible: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Build with reproducible date run: source tools/ci.sh && ci_unix_minimal_build env: @@ -46,7 +46,7 @@ jobs: standard: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Build run: source tools/ci.sh && ci_unix_standard_build - name: Run main test suite @@ -55,10 +55,27 @@ jobs: if: failure() run: tests/run-tests.py --print-failures + standard_v2: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Build + run: source tools/ci.sh && ci_unix_standard_v2_build + - name: Run main test suite + run: source tools/ci.sh && ci_unix_standard_v2_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + coverage: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' - name: Install packages run: source tools/ci.sh && ci_unix_coverage_setup - name: Build @@ -76,18 +93,19 @@ jobs: (cd ports/unix && gcov -o build-coverage/py ../../py/*.c || true) (cd ports/unix && gcov -o build-coverage/extmod ../../extmod/*.c || true) - name: Upload coverage to Codecov - uses: codecov/codecov-action@v3 + uses: codecov/codecov-action@v5 with: fail_ci_if_error: true verbose: true + token: ${{ secrets.CODECOV_TOKEN }} - name: Print failures if: failure() run: tests/run-tests.py --print-failures coverage_32bit: - runs-on: ubuntu-latest + runs-on: ubuntu-22.04 # use 22.04 to get libffi-dev:i386 steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_unix_32bit_setup - name: Build @@ -103,9 +121,9 @@ jobs: run: tests/run-tests.py --print-failures nanbox: - runs-on: ubuntu-20.04 # use 20.04 to get python2 + runs-on: ubuntu-22.04 # use 22.04 to get python2, and libffi-dev:i386 steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_unix_32bit_setup - name: Build @@ -119,7 +137,7 @@ jobs: float: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Build run: source tools/ci.sh && ci_unix_float_build - name: Run main test suite @@ -129,9 +147,9 @@ jobs: run: tests/run-tests.py --print-failures stackless_clang: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_unix_clang_setup - name: Build @@ -143,9 +161,9 @@ jobs: run: tests/run-tests.py --print-failures float_clang: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_unix_clang_setup - name: Build @@ -156,22 +174,15 @@ jobs: if: failure() run: tests/run-tests.py --print-failures - settrace: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v3 - - name: Build - run: source tools/ci.sh && ci_unix_settrace_build - - name: Run main test suite - run: source tools/ci.sh && ci_unix_settrace_run_tests - - name: Print failures - if: failure() - run: tests/run-tests.py --print-failures - settrace_stackless: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' - name: Build run: source tools/ci.sh && ci_unix_settrace_stackless_build - name: Run main test suite @@ -181,10 +192,10 @@ jobs: run: tests/run-tests.py --print-failures macos: - runs-on: macos-11.0 + runs-on: macos-latest steps: - - uses: actions/checkout@v3 - - uses: actions/setup-python@v4 + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 with: python-version: '3.8' - name: Build @@ -196,9 +207,10 @@ jobs: run: tests/run-tests.py --print-failures qemu_mips: - runs-on: ubuntu-latest + # ubuntu-22.04 is needed for older libffi. + runs-on: ubuntu-22.04 steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_unix_qemu_mips_setup - name: Build @@ -210,9 +222,10 @@ jobs: run: tests/run-tests.py --print-failures qemu_arm: - runs-on: ubuntu-latest + # ubuntu-22.04 is needed for older libffi. + runs-on: ubuntu-22.04 steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_unix_qemu_arm_setup - name: Build @@ -222,3 +235,68 @@ jobs: - name: Print failures if: failure() run: tests/run-tests.py --print-failures + + qemu_riscv64: + # ubuntu-22.04 is needed for older libffi. + runs-on: ubuntu-22.04 + steps: + - uses: actions/checkout@v4 + - name: Install packages + run: source tools/ci.sh && ci_unix_qemu_riscv64_setup + - name: Build + run: source tools/ci.sh && ci_unix_qemu_riscv64_build + - name: Run main test suite + run: source tools/ci.sh && ci_unix_qemu_riscv64_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + sanitize_address: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' + - name: Install packages + run: source tools/ci.sh && ci_unix_coverage_setup + - name: Build + run: source tools/ci.sh && ci_unix_sanitize_address_build + - name: Run main test suite + run: source tools/ci.sh && ci_unix_sanitize_address_run_tests + - name: Test merging .mpy files + run: source tools/ci.sh && ci_unix_coverage_run_mpy_merge_tests + - name: Build native mpy modules + run: source tools/ci.sh && ci_native_mpy_modules_build + - name: Test importing .mpy generated by mpy_ld.py + run: source tools/ci.sh && ci_unix_coverage_run_native_mpy_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures + + sanitize_undefined: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + # Python 3.12 is the default for ubuntu-24.04, but that has compatibility issues with settrace tests. + # Can remove this step when ubuntu-latest uses a more recent Python 3.x as the default. + with: + python-version: '3.11' + - name: Install packages + run: source tools/ci.sh && ci_unix_coverage_setup + - name: Build + run: source tools/ci.sh && ci_unix_sanitize_undefined_build + - name: Run main test suite + run: source tools/ci.sh && ci_unix_sanitize_undefined_run_tests + - name: Test merging .mpy files + run: source tools/ci.sh && ci_unix_coverage_run_mpy_merge_tests + - name: Build native mpy modules + run: source tools/ci.sh && ci_native_mpy_modules_build + - name: Test importing .mpy generated by mpy_ld.py + run: source tools/ci.sh && ci_unix_coverage_run_native_mpy_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ports_webassembly.yml b/.github/workflows/ports_webassembly.yml index 2e0865662f061..880f15ab34469 100644 --- a/.github/workflows/ports_webassembly.yml +++ b/.github/workflows/ports_webassembly.yml @@ -20,7 +20,7 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_webassembly_setup - name: Build diff --git a/.github/workflows/ports_windows.yml b/.github/workflows/ports_windows.yml index b31718c591402..f33277d471d3c 100644 --- a/.github/workflows/ports_windows.yml +++ b/.github/workflows/ports_windows.yml @@ -18,10 +18,132 @@ concurrency: cancel-in-progress: true jobs: - build: + build-vs: + strategy: + fail-fast: false + matrix: + platform: [x86, x64] + configuration: [Debug, Release] + variant: [dev, standard] + visualstudio: ['2017', '2019', '2022'] + include: + - visualstudio: '2017' + vs_version: '[15, 16)' + - visualstudio: '2019' + vs_version: '[16, 17)' + - visualstudio: '2022' + vs_version: '[17, 18)' + # trim down the number of jobs in the matrix + exclude: + - variant: standard + configuration: Debug + - visualstudio: '2019' + configuration: Debug + runs-on: windows-latest + env: + CI_BUILD_CONFIGURATION: ${{ matrix.configuration }} + steps: + - name: Install Visual Studio 2017 + if: matrix.visualstudio == '2017' + run: | + choco install visualstudio2017buildtools + choco install visualstudio2017-workload-vctools + choco install windows-sdk-8.1 + - name: Install Visual Studio 2019 + if: matrix.visualstudio == '2019' + run: | + choco install visualstudio2019buildtools + choco install visualstudio2019-workload-vctools + choco install windows-sdk-8.1 + - uses: microsoft/setup-msbuild@v2 + with: + vs-version: ${{ matrix.vs_version }} + - uses: actions/checkout@v4 + - name: Build mpy-cross.exe + run: msbuild mpy-cross\mpy-cross.vcxproj -maxcpucount -property:Configuration=${{ matrix.configuration }} -property:Platform=${{ matrix.platform }} + - name: Update submodules + run: git submodule update --init lib/micropython-lib + - name: Build micropython.exe + run: msbuild ports\windows\micropython.vcxproj -maxcpucount -property:Configuration=${{ matrix.configuration }} -property:Platform=${{ matrix.platform }} -property:PyVariant=${{ matrix.variant }} + - name: Get micropython.exe path + id: get_path + run: | + $exePath="$(msbuild ports\windows\micropython.vcxproj -nologo -v:m -t:ShowTargetPath -property:Configuration=${{ matrix.configuration }} -property:Platform=${{ matrix.platform }} -property:PyVariant=${{ matrix.variant }})" + echo ("micropython=" + $exePath.Trim()) >> $env:GITHUB_OUTPUT + - name: Run tests + id: test + env: + MICROPY_MICROPYTHON: ${{ steps.get_path.outputs.micropython }} + working-directory: tests + run: python run-tests.py + - name: Print failures + if: failure() && steps.test.conclusion == 'failure' + working-directory: tests + run: python run-tests.py --print-failures + - name: Run mpy tests + id: test_mpy + env: + MICROPY_MICROPYTHON: ${{ steps.get_path.outputs.micropython }} + working-directory: tests + run: python run-tests.py --via-mpy -d basics float micropython + - name: Print mpy failures + if: failure() && steps.test_mpy.conclusion == 'failure' + working-directory: tests + run: python run-tests.py --print-failures + + build-mingw: + strategy: + fail-fast: false + matrix: + variant: [dev, standard] + sys: [mingw32, mingw64] + include: + - sys: mingw32 + env: i686 + - sys: mingw64 + env: x86_64 + runs-on: windows-latest + env: + CHERE_INVOKING: enabled_from_arguments + defaults: + run: + shell: msys2 {0} + steps: + - uses: actions/setup-python@v5 + # note: can go back to installing mingw-w64-${{ matrix.env }}-python after + # MSYS2 updates to Python >3.12 (due to settrace compatibility issue) + with: + python-version: '3.11' + - uses: msys2/setup-msys2@v2 + with: + msystem: ${{ matrix.sys }} + update: true + install: >- + make + mingw-w64-${{ matrix.env }}-gcc + pkg-config + git + diffutils + path-type: inherit # Remove when setup-python is removed + - uses: actions/checkout@v4 + - name: Build mpy-cross.exe + run: make -C mpy-cross -j2 + - name: Update submodules + run: make -C ports/windows VARIANT=${{ matrix.variant }} submodules + - name: Build micropython.exe + run: make -C ports/windows -j2 VARIANT=${{ matrix.variant }} + - name: Run tests + id: test + run: make -C ports/windows test_full VARIANT=${{ matrix.variant }} + - name: Print failures + if: failure() && steps.test.conclusion == 'failure' + working-directory: tests + run: python run-tests.py --print-failures + + cross-build-on-linux: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Install packages run: source tools/ci.sh && ci_windows_setup - name: Build diff --git a/.github/workflows/ports_zephyr.yml b/.github/workflows/ports_zephyr.yml index f64401b316a5b..eb85af6a36154 100644 --- a/.github/workflows/ports_zephyr.yml +++ b/.github/workflows/ports_zephyr.yml @@ -20,10 +20,41 @@ jobs: build: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v3 + - uses: jlumbroso/free-disk-space@main + with: + # Only free up a few things so this step runs quickly. + android: false + dotnet: true + haskell: true + large-packages: false + docker-images: false + swap-storage: false + - uses: actions/checkout@v4 + - id: versions + name: Read Zephyr version + run: source tools/ci.sh && echo "ZEPHYR=$ZEPHYR_VERSION" | tee "$GITHUB_OUTPUT" + - name: Cached Zephyr Workspace + id: cache_workspace + uses: actions/cache@v4 + with: + # note that the Zephyr CI docker image is 15GB. At time of writing + # GitHub caches are limited to 10GB total for a project. So we only + # cache the "workspace" + path: ./zephyrproject + key: zephyr-workspace-${{ steps.versions.outputs.ZEPHYR }} + - name: ccache + uses: hendrikmuhs/ccache-action@v1.2 + with: + key: zephyr - name: Install packages run: source tools/ci.sh && ci_zephyr_setup - name: Install Zephyr + if: steps.cache_workspace.outputs.cache-hit != 'true' run: source tools/ci.sh && ci_zephyr_install - name: Build run: source tools/ci.sh && ci_zephyr_build + - name: Run main test suite + run: source tools/ci.sh && ci_zephyr_run_tests + - name: Print failures + if: failure() + run: tests/run-tests.py --print-failures diff --git a/.github/workflows/ruff.yml b/.github/workflows/ruff.yml new file mode 100644 index 0000000000000..4c4a2a3162ed6 --- /dev/null +++ b/.github/workflows/ruff.yml @@ -0,0 +1,13 @@ +name: Python code lint and formatting with ruff + +on: [push, pull_request] + +jobs: + ruff: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + # ruff version should be kept in sync with .pre-commit-config.yaml & also micropython-lib + - run: pipx install ruff==0.11.6 + - run: ruff check --output-format=github . + - run: ruff format --diff . diff --git a/.gitignore b/.gitignore index 2d20cb18970e8..b5010dfd14db6 100644 --- a/.gitignore +++ b/.gitignore @@ -11,8 +11,9 @@ build/ build-*/ docs/genrst/ -# Test failure outputs +# Test failure outputs and intermediate artefacts tests/results/* +tests/ports/unix/ffi_lib.so # Python cache files __pycache__/ diff --git a/.gitmodules b/.gitmodules index 992fec3d1836f..02849ec9bdd11 100644 --- a/.gitmodules +++ b/.gitmodules @@ -3,13 +3,13 @@ url = https://github.com/micropython/axtls.git [submodule "lib/libffi"] path = lib/libffi - url = https://github.com/atgreen/libffi + url = https://github.com/libffi/libffi [submodule "lib/lwip"] path = lib/lwip url = https://github.com/lwip-tcpip/lwip.git [submodule "lib/berkeley-db-1.xx"] path = lib/berkeley-db-1.xx - url = https://github.com/pfalcon/berkeley-db-1.xx + url = https://github.com/micropython/berkeley-db-1.xx [submodule "lib/stm32lib"] path = lib/stm32lib url = https://github.com/micropython/stm32lib @@ -56,3 +56,21 @@ [submodule "lib/micropython-lib"] path = lib/micropython-lib url = https://github.com/micropython/micropython-lib.git +[submodule "lib/protobuf-c"] + path = lib/protobuf-c + url = https://github.com/protobuf-c/protobuf-c.git +[submodule "lib/open-amp"] + path = lib/open-amp + url = https://github.com/OpenAMP/open-amp.git +[submodule "lib/libmetal"] + path = lib/libmetal + url = https://github.com/OpenAMP/libmetal.git +[submodule "lib/arduino-lib"] + path = lib/arduino-lib + url = https://github.com/arduino/arduino-lib-mpy.git +[submodule "lib/alif_ensemble-cmsis-dfp"] + path = lib/alif_ensemble-cmsis-dfp + url = https://github.com/alifsemi/alif_ensemble-cmsis-dfp.git +[submodule "lib/alif-security-toolkit"] + path = lib/alif-security-toolkit + url = https://github.com/micropython/alif-security-toolkit.git diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 12f3d79c93533..ac9785bb59232 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -2,8 +2,8 @@ repos: - repo: local hooks: - id: codeformat - name: MicroPython codeformat.py for changed files - entry: tools/codeformat.py -v -f + name: MicroPython codeformat.py for changed C files + entry: tools/codeformat.py -v -c -f language: python - id: verifygitlog name: MicroPython git commit message format checker @@ -11,3 +11,17 @@ repos: language: python verbose: true stages: [commit-msg] + - repo: https://github.com/charliermarsh/ruff-pre-commit + # Version should be kept in sync with .github/workflows/ruff.yml & also micropython-lib + rev: v0.11.6 + hooks: + - id: ruff + - id: ruff-format + - repo: https://github.com/codespell-project/codespell + # Version should be kept in sync with .github/workflows/codespell.yml + rev: v2.4.1 + hooks: + - id: codespell + name: Spellcheck for changed files (codespell) + additional_dependencies: + - tomli diff --git a/CODECONVENTIONS.md b/CODECONVENTIONS.md index 3b678bfeb800b..d3f71cb083deb 100644 --- a/CODECONVENTIONS.md +++ b/CODECONVENTIONS.md @@ -11,7 +11,7 @@ It's also ok to drop file extensions. Besides prefix, first line of a commit message should describe a change clearly and to the point, and be a grammatical sentence with -final full stop. First line should fit within 72 characters. Examples +final full stop. First line must fit within 72 characters. Examples of good first line of commit messages: py/objstr: Add splitlines() method. @@ -27,12 +27,9 @@ change beyond 5 lines would likely require such detailed description. To get good practical examples of good commits and their messages, browse the `git log` of the project. -When committing you are encouraged to sign-off your commit by adding -"Signed-off-by" lines and similar, eg using "git commit -s". If you don't -explicitly sign-off in this way then the commit message, which includes your -name and email address in the "Author" line, implies your sign-off. In either -case, of explicit or implicit sign-off, you are certifying and signing off -against the following: +When committing you must sign-off your commit by adding "Signed-off-by:" +line(s) at the end of the commit message, e.g. using `git commit -s`. You +are then certifying and signing off against the following: * That you wrote the change yourself, or took it from a project with a compatible license (in the latter case the commit message, and possibly @@ -49,21 +46,23 @@ against the following: * Your contribution including commit message will be publicly and indefinitely available for anyone to access, including redistribution under the terms of the project's license. -* Your signature for all of the above, which is the "Signed-off-by" line - or the "Author" line in the commit message, includes your full real name and - a valid and active email address by which you can be contacted in the - foreseeable future. +* Your signature for all of the above, which is the "Signed-off-by" line, + includes your full real name and a valid and active email address by + which you can be contacted in the foreseeable future. Code auto-formatting ==================== -Both C and Python code are auto-formatted using the `tools/codeformat.py` -script. This uses [uncrustify](https://github.com/uncrustify/uncrustify) to -format C code and [black](https://github.com/psf/black) to format Python code. -After making changes, and before committing, run this tool to reformat your -changes to the correct style. Without arguments this tool will reformat all -source code (and may take some time to run). Otherwise pass as arguments to -the tool the files that changed and it will only reformat those. +Both C and Python code formatting are controlled for consistency across the +MicroPython codebase. C code is formatted using the `tools/codeformat.py` +script which uses [uncrustify](https://github.com/uncrustify/uncrustify). +Python code is linted and formatted using +[ruff & ruff format](https://github.com/astral-sh/ruff). +After making changes, and before committing, run `tools/codeformat.py` to +reformat your C code and `ruff format` for any Python code. Without +arguments this tool will reformat all source code (and may take some time +to run). Otherwise pass as arguments to the tool the files that changed, +and it will only reformat those. uncrustify ========== @@ -105,6 +104,22 @@ This command may work, please raise a new Issue if it doesn't: curl -L https://github.com/Homebrew/homebrew-core/raw/2b07d8192623365078a8b855a164ebcdf81494a6/Formula/uncrustify.rb > uncrustify.rb && brew install uncrustify.rb && rm uncrustify.rb ``` +Code spell checking +=================== + +Code spell checking is done using [codespell](https://github.com/codespell-project/codespell#codespell) +and runs in a GitHub action in CI. Codespell is configured via `pyproject.toml` +to avoid false positives. It is recommended run codespell before submitting a +PR. To simplify this, codespell is configured as a pre-commit hook and will be +installed if you run `pre-commit install` (see below). + +If you want to install and run codespell manually, you can do so by running: + +``` +$ pip install codespell tomli +$ codespell +``` + Automatic Pre-Commit Hooks ========================== @@ -155,12 +170,22 @@ Tips: * To ignore the pre-commit message format check temporarily, start the commit message subject line with "WIP" (for "Work In Progress"). +Running pre-commit manually +=========================== + +Once pre-commit is installed as per the previous section it can be manually +run against the MicroPython python codebase to update file formatting on +demand, with either: +* `pre-commit run --all-files` to fix all files in the MicroPython codebase +* `pre-commit run --file ./path/to/my/file` to fix just one file +* `pre-commit run --file ./path/to/my/folder/*` to fix just one folder + Python code conventions ======================= Python code follows [PEP 8](https://legacy.python.org/dev/peps/pep-0008/) and -is auto-formatted using [black](https://github.com/psf/black) with a line-length -of 99 characters. +is auto-formatted using [ruff format](https://docs.astral.sh/ruff/formatter) +with a line-length of 99 characters. Naming conventions: - Module names are short and all lowercase; eg pyb, stm. @@ -181,14 +206,21 @@ adhere to the existing style and use `tools/codeformat.py` to check any changes. The main conventions, and things not enforceable via the auto-formatter, are described below. -White space: +As the MicroPython code base is over ten years old, not every source file +conforms fully to these conventions. If making small changes to existing code, +then it's usually acceptable to follow the existing code's style. New code or +major changes should follow the conventions described here. + +## White space + - Expand tabs to 4 spaces. - Don't leave trailing whitespace at the end of a line. - For control blocks (if, for, while), put 1 space between the keyword and the opening parenthesis. - Put 1 space after a comma, and 1 space around operators. -Braces: +## Braces + - Use braces for all blocks, even no-line and single-line pieces of code. - Put opening braces on the end of the line it belongs to, not on @@ -196,18 +228,43 @@ Braces: - For else-statements, put the else on the same line as the previous closing brace. -Header files: +## Header files + - Header files should be protected from multiple inclusion with #if directives. See an existing header for naming convention. -Names: +## Names + - Use underscore_case, not camelCase for all names. - Use CAPS_WITH_UNDERSCORE for enums and macros. - When defining a type use underscore_case and put '_t' after it. -Integer types: MicroPython runs on 16, 32, and 64 bit machines, so it's -important to use the correctly-sized (and signed) integer types. The -general guidelines are: +### Public names (declared in headers) + +- MicroPython-specific names (especially any declared in `py/` and `extmod/` + directories) should generally start with `mp_` or `MP_`. +- Functions and variables declared in a header should generally share a longer + common prefix. Usually the prefix matches the file name (i.e. items defined in + `py/obj.c` are declared in `py/obj.h` and should be prefixed `mp_obj_`). There + are exceptions, for example where one header file contains declarations + implemented in multiple source files for expediency. + +### Private names (specific to a single .c file) + +- For static functions and variables exposed to Python (i.e. a static C function + that is wrapped in `MP_DEFINE_CONST_FUN_...` and attached to a module), use + the file-level shared common prefix, i.e. name them as if the function or + variable was not static. +- Other static definitions in source files (i.e. functions or variables defined + in a .c file that are only used within that .c file) don't need any prefix + (specifically: no `s_` or `_` prefix, and generally avoid adding the + file-level common prefix). + +## Integer types + +MicroPython runs on 16, 32, and 64 bit machines, so it's important to use the +correctly-sized (and signed) integer types. The general guidelines are: + - For most cases use mp_int_t for signed and mp_uint_t for unsigned integer values. These are guaranteed to be machine-word sized and therefore big enough to hold the value from a MicroPython small-int @@ -216,11 +273,13 @@ general guidelines are: - You can use int/uint, but remember that they may be 16-bits wide. - If in doubt, use mp_int_t/mp_uint_t. -Comments: +## Comments + - Be concise and only write comments for things that are not obvious. - Use `// ` prefix, NOT `/* ... */`. No extra fluff. -Memory allocation: +## Memory allocation + - Use m_new, m_renew, m_del (and friends) to allocate and free heap memory. These macros are defined in py/misc.h. @@ -255,7 +314,7 @@ Documentation conventions ========================= MicroPython generally follows CPython in documentation process and -conventions. reStructuredText syntax is used for the documention. +conventions. reStructuredText syntax is used for the documentation. Specific conventions/suggestions: diff --git a/LICENSE b/LICENSE index 3a2f15d1f41be..929a2e97de7bf 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ The MIT License (MIT) -Copyright (c) 2013-2023 Damien P. George +Copyright (c) 2013-2025 Damien P. George Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal @@ -36,7 +36,6 @@ used during the build process and is not part of the compiled source code. / (MIT) /drivers /cc3100 (BSD-3-clause) - /wiznet5k (BSD-3-clause) /lib /asf4 (Apache-2.0) /axtls (BSD-3-clause) @@ -49,24 +48,31 @@ used during the build process and is not part of the compiled source code. /cmsis (BSD-3-clause) /crypto-algorithms (NONE) /libhydrogen (ISC) + /libmetal (BSD-3-clause) /littlefs (BSD-3-clause) /lwip (BSD-3-clause) /mynewt-nimble (Apache-2.0) /nrfx (BSD-3-clause) /nxp_driver (BSD-3-Clause) /oofatfs (BSD-1-clause) + /open-amp (BSD-3-clause) /pico-sdk (BSD-3-clause) /re15 (BSD-3-clause) /stm32lib (BSD-3-clause) - /tinytest (BSD-3-clause) /tinyusb (MIT) /uzlib (Zlib) + /wiznet5k (MIT) /logo (uses OFL-1.1) /ports /cc3200 /hal (BSD-3-clause) /simplelink (BSD-3-clause) /FreeRTOS (GPL-2.0 with FreeRTOS exception) + /esp32 + /ppp_set_auth.* (Apache-2.0) + /rp2 + /mutex_extra.c (BSD-3-clause) + /clocks_extra.c (BSD-3-clause) /stm32 /usbd*.c (MCD-ST Liberty SW License Agreement V2) /stm32_it.* (MIT + BSD-3-clause) @@ -76,8 +82,6 @@ used during the build process and is not part of the compiled source code. /*/stm32*.h (BSD-3-clause) /usbdev (MCD-ST Liberty SW License Agreement V2) /usbhost (MCD-ST Liberty SW License Agreement V2) - /teensy - /core (PJRC.COM) /zephyr /src (Apache-2.0) /tools diff --git a/README.md b/README.md index 6482899b251d9..c78a2384604fe 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -[![CI badge](https://github.com/micropython/micropython/workflows/unix%20port/badge.svg)](https://github.com/micropython/micropython/actions?query=branch%3Amaster+event%3Apush) [![codecov](https://codecov.io/gh/micropython/micropython/branch/master/graph/badge.svg?token=I92PfD05sD)](https://codecov.io/gh/micropython/micropython) +[![Unix CI badge](https://github.com/micropython/micropython/actions/workflows/ports_unix.yml/badge.svg)](https://github.com/micropython/micropython/actions?query=branch%3Amaster+event%3Apush) [![STM32 CI badge](https://github.com/micropython/micropython/actions/workflows/ports_stm32.yml/badge.svg)](https://github.com/micropython/micropython/actions?query=branch%3Amaster+event%3Apush) [![Docs CI badge](https://github.com/micropython/micropython/actions/workflows/docs.yml/badge.svg)](https://docs.micropython.org/) [![codecov](https://codecov.io/gh/micropython/micropython/branch/master/graph/badge.svg?token=I92PfD05sD)](https://codecov.io/gh/micropython/micropython) The MicroPython project ======================= @@ -19,7 +19,7 @@ Python 3.5 and some select features from later versions). The following core datatypes are provided: `str`(including basic Unicode support), `bytes`, `bytearray`, `tuple`, `list`, `dict`, `set`, `frozenset`, `array.array`, `collections.namedtuple`, classes and instances. Builtin modules include -`os`, `sys`, `time`, `re`, and `struct`, etc. Select ports have support for +`os`, `sys`, `time`, `re`, and `struct`, etc. Some ports have support for `_thread` module (multithreading), `socket` and `ssl` for networking, and `asyncio`. Note that only a subset of Python 3 functionality is implemented for the data types and modules. @@ -35,8 +35,8 @@ DAC, PWM, SPI, I2C, CAN, Bluetooth, and USB. Getting started --------------- -See the [online documentation](https://docs.micropython.org/) for API -references and information about using MicroPython and information about how +See the [online documentation](https://docs.micropython.org/) for the API +reference and information about using MicroPython and information about how it is implemented. We use [GitHub Discussions](https://github.com/micropython/micropython/discussions) @@ -108,18 +108,17 @@ track of the code size of the core runtime and VM. In addition, the following ports are provided in this repository: - [cc3200](ports/cc3200) -- Texas Instruments CC3200 (including PyCom WiPy). - - [esp32](ports/esp32) -- Espressif ESP32 SoC (including ESP32S2, ESP32S3, ESP32C3). + - [esp32](ports/esp32) -- Espressif ESP32 SoC (including ESP32S2, ESP32S3, ESP32C3, ESP32C6). - [esp8266](ports/esp8266) -- Espressif ESP8266 SoC. - [mimxrt](ports/mimxrt) -- NXP m.iMX RT (including Teensy 4.x). - [nrf](ports/nrf) -- Nordic Semiconductor nRF51 and nRF52. - [pic16bit](ports/pic16bit) -- Microchip PIC 16-bit. - [powerpc](ports/powerpc) -- IBM PowerPC (including Microwatt) - - [qemu-arm](ports/qemu-arm) -- QEMU-based emulated target, for testing) + - [qemu](ports/qemu) -- QEMU-based emulated target (for testing) - [renesas-ra](ports/renesas-ra) -- Renesas RA family. - [rp2](ports/rp2) -- Raspberry Pi RP2040 (including Pico and Pico W). - [samd](ports/samd) -- Microchip (formerly Atmel) SAMD21 and SAMD51. - [stm32](ports/stm32) -- STMicroelectronics STM32 family (including F0, F4, F7, G0, G4, H7, L0, L4, WB) - - [teensy](ports/teensy) -- Teensy 3.x. - [webassembly](ports/webassembly) -- Emscripten port targeting browsers and NodeJS. - [zephyr](ports/zephyr) -- Zephyr RTOS. diff --git a/docs/conf.py b/docs/conf.py index 933e7b34b2741..eb61487582024 100755 --- a/docs/conf.py +++ b/docs/conf.py @@ -19,55 +19,59 @@ # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. -sys.path.insert(0, os.path.abspath('.')) +sys.path.insert(0, os.path.abspath(".")) + +# The MICROPY_VERSION env var should be "vX.Y.Z" (or unset). +micropy_version = os.getenv("MICROPY_VERSION") or "latest" +micropy_all_versions = (os.getenv("MICROPY_ALL_VERSIONS") or "latest").split(",") +url_pattern = "%s/en/%%s" % (os.getenv("MICROPY_URL_PREFIX") or "/",) # The members of the html_context dict are available inside topindex.html -micropy_version = os.getenv('MICROPY_VERSION') or 'latest' -micropy_all_versions = (os.getenv('MICROPY_ALL_VERSIONS') or 'latest').split(',') -url_pattern = '%s/en/%%s' % (os.getenv('MICROPY_URL_PREFIX') or '/',) html_context = { - 'cur_version':micropy_version, - 'all_versions':[ - (ver, url_pattern % ver) for ver in micropy_all_versions - ], - 'downloads':[ - ('PDF', url_pattern % micropy_version + '/micropython-docs.pdf'), + "cur_version": micropy_version, + "all_versions": [(ver, url_pattern % ver) for ver in micropy_all_versions], + "downloads": [ + ("PDF", url_pattern % micropy_version + "/micropython-docs.pdf"), ], - 'is_release': micropy_version != 'latest', + "is_release": micropy_version != "latest", } +# Authors used in various parts of the documentation. +micropy_authors = "MicroPython authors and contributors" + # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. -#needs_sphinx = '1.0' +# needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ - 'sphinx.ext.autodoc', - 'sphinx.ext.doctest', - 'sphinx.ext.intersphinx', - 'sphinx.ext.todo', - 'sphinx.ext.coverage', + "sphinx.ext.autodoc", + "sphinx.ext.doctest", + "sphinx.ext.intersphinx", + "sphinx.ext.todo", + "sphinx.ext.coverage", + "sphinxcontrib.jquery", ] # Add any paths that contain templates here, relative to this directory. -templates_path = ['templates'] +templates_path = ["templates"] # The suffix of source filenames. -source_suffix = '.rst' +source_suffix = ".rst" # The encoding of source files. -#source_encoding = 'utf-8-sig' +# source_encoding = 'utf-8-sig' # The master toctree document. -master_doc = 'index' +master_doc = "index" # General information about the project. -project = 'MicroPython' -copyright = '- The MicroPython Documentation is Copyright © 2014-2023, Damien P. George, Paul Sokolovsky, and contributors' +project = "MicroPython" +copyright = "- The MicroPython Documentation is Copyright © 2014-2025, " + micropy_authors # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -79,41 +83,41 @@ # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. -#language = None +# language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: -#today = '' +# today = '' # Else, today_fmt is used as the format for a strftime call. -#today_fmt = '%B %d, %Y' +# today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. -exclude_patterns = ['build', '.venv'] +exclude_patterns = ["build", ".venv"] # The reST default role (used for this markup: `text`) to use for all # documents. -default_role = 'any' +default_role = "any" # If true, '()' will be appended to :func: etc. cross-reference text. -#add_function_parentheses = True +# add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). -#add_module_names = True +# add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. -#show_authors = False +# show_authors = False # The name of the Pygments (syntax highlighting) style to use. -pygments_style = 'sphinx' +pygments_style = "sphinx" # A list of ignored prefixes for module index sorting. -#modindex_common_prefix = [] +# modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. -#keep_warnings = False +# keep_warnings = False # Global include files. Sphinx docs suggest using rst_epilog in preference # of rst_prolog, so we follow. Absolute paths below mean "from the base @@ -125,144 +129,148 @@ # -- Options for HTML output ---------------------------------------------- # on_rtd is whether we are on readthedocs.org -on_rtd = os.environ.get('READTHEDOCS', None) == 'True' +on_rtd = os.environ.get("READTHEDOCS", None) == "True" if not on_rtd: # only import and set the theme if we're building docs locally try: import sphinx_rtd_theme - html_theme = 'sphinx_rtd_theme' - html_theme_path = [sphinx_rtd_theme.get_html_theme_path(), '.'] + + html_theme = "sphinx_rtd_theme" + html_theme_path = [sphinx_rtd_theme.get_html_theme_path(), "."] except: - html_theme = 'default' - html_theme_path = ['.'] + html_theme = "default" + html_theme_path = ["."] else: - html_theme_path = ['.'] + html_theme_path = ["."] # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. -#html_theme_options = {} +# html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. # html_theme_path = ['.'] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". -#html_title = None +# html_title = None # A shorter title for the navigation bar. Default is the same as html_title. -#html_short_title = None +# html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. -#html_logo = '../../logo/trans-logo.png' +# html_logo = '../../logo/trans-logo.png' # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. -html_favicon = 'static/favicon.ico' +html_favicon = "static/favicon.ico" # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ['static'] +html_static_path = ["static"] # Add a custom CSS file for HTML generation html_css_files = [ - 'custom.css', + "custom.css", ] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied # directly to the root of the documentation. -#html_extra_path = [] +# html_extra_path = [] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. -html_last_updated_fmt = '%d %b %Y' +html_last_updated_fmt = "%d %b %Y" # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. -#html_use_smartypants = True +# html_use_smartypants = True # Custom sidebar templates, maps document names to template names. -#html_sidebars = {} +# html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. html_additional_pages = {"index": "topindex.html"} # If false, no module index is generated. -#html_domain_indices = True +# html_domain_indices = True # If false, no index is generated. -#html_use_index = True +# html_use_index = True # If true, the index is split into individual pages for each letter. -#html_split_index = False +# html_split_index = False # If true, links to the reST sources are added to the pages. -#html_show_sourcelink = True +# html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. -#html_show_sphinx = True +# html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. -#html_show_copyright = True +# html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. -#html_use_opensearch = '' +# html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). -#html_file_suffix = None +# html_file_suffix = None # Output file base name for HTML help builder. -htmlhelp_basename = 'MicroPythondoc' +htmlhelp_basename = "MicroPythondoc" # -- Options for LaTeX output --------------------------------------------- latex_elements = { -# The paper size ('letterpaper' or 'a4paper'). -#'papersize': 'letterpaper', - -# The font size ('10pt', '11pt' or '12pt'). -#'pointsize': '10pt', - -# Additional stuff for the LaTeX preamble. -#'preamble': '', -# Include 3 levels of headers in PDF ToC -'preamble': '\setcounter{tocdepth}{2}', + # The paper size ('letterpaper' or 'a4paper'). + #'papersize': 'letterpaper', + # The font size ('10pt', '11pt' or '12pt'). + #'pointsize': '10pt', + # Additional stuff for the LaTeX preamble. + #'preamble': '', + # Include 3 levels of headers in PDF ToC + "preamble": r"\setcounter{tocdepth}{2}", } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ - (master_doc, 'MicroPython.tex', 'MicroPython Documentation', - 'Damien P. George, Paul Sokolovsky, and contributors', 'manual'), + ( + master_doc, + "MicroPython.tex", + "MicroPython Documentation", + micropy_authors, + "manual", + ), ] # The name of an image file (relative to this directory) to place at the top of # the title page. -#latex_logo = None +# latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. -#latex_use_parts = False +# latex_use_parts = False # If true, show page references after internal links. -#latex_show_pagerefs = False +# latex_show_pagerefs = False # If true, show URL addresses after external links. -#latex_show_urls = False +# latex_show_urls = False # Documents to append as an appendix to all manuals. -#latex_appendices = [] +# latex_appendices = [] # If false, no module index is generated. -#latex_domain_indices = True +# latex_domain_indices = True # Enable better Unicode support so that `make latexpdf` doesn't fail latex_engine = "xelatex" @@ -272,12 +280,17 @@ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ - ('index', 'micropython', 'MicroPython Documentation', - ['Damien P. George, Paul Sokolovsky, and contributors'], 1), + ( + "index", + "micropython", + "MicroPython Documentation", + [micropy_authors], + 1, + ), ] # If true, show URL addresses after external links. -#man_show_urls = False +# man_show_urls = False # -- Options for Texinfo output ------------------------------------------- @@ -286,23 +299,29 @@ # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ - (master_doc, 'MicroPython', 'MicroPython Documentation', - 'Damien P. George, Paul Sokolovsky, and contributors', 'MicroPython', 'One line description of project.', - 'Miscellaneous'), + ( + master_doc, + "MicroPython", + "MicroPython Documentation", + micropy_authors, + "MicroPython", + "One line description of project.", + "Miscellaneous", + ), ] # Documents to append as an appendix to all manuals. -#texinfo_appendices = [] +# texinfo_appendices = [] # If false, no module index is generated. -#texinfo_domain_indices = True +# texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. -#texinfo_show_urls = 'footnote' +# texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. -#texinfo_no_detailmenu = False +# texinfo_no_detailmenu = False # Example configuration for intersphinx: refer to the Python standard library. -intersphinx_mapping = {'python': ('https://docs.python.org/3.5', None)} +intersphinx_mapping = {"python": ("https://docs.python.org/3.5", None)} diff --git a/docs/develop/cmodules.rst b/docs/develop/cmodules.rst index 6bc2f62ab4d5a..c5aa919b7645a 100644 --- a/docs/develop/cmodules.rst +++ b/docs/develop/cmodules.rst @@ -53,13 +53,13 @@ A MicroPython user C module is a directory with the following files: ``SRC_USERMOD_C`` or ``SRC_USERMOD_LIB_C`` variables. The former will be processed for ``MP_QSTR_`` and ``MP_REGISTER_MODULE`` definitions, the latter will not (e.g. helpers and library code that isn't MicroPython-specific). - These paths should include your expaned copy of ``$(USERMOD_DIR)``, e.g.:: + These paths should include your expanded copy of ``$(USERMOD_DIR)``, e.g.:: SRC_USERMOD_C += $(EXAMPLE_MOD_DIR)/modexample.c SRC_USERMOD_LIB_C += $(EXAMPLE_MOD_DIR)/utils/algorithm.c Similarly, use ``SRC_USERMOD_CXX`` and ``SRC_USERMOD_LIB_CXX`` for C++ - source files. + source files. If you want to include assembly files use ``SRC_USERMOD_LIB_ASM``. If you have custom compiler options (like ``-I`` to add directories to search for header files), these should be added to ``CFLAGS_USERMOD`` for C code diff --git a/docs/develop/compiler.rst b/docs/develop/compiler.rst index cac92585ff4ee..0c25ad3a01ef1 100644 --- a/docs/develop/compiler.rst +++ b/docs/develop/compiler.rst @@ -98,7 +98,7 @@ Then also edit ``py/lexer.c`` to add the new keyword literal text: .. code-block:: c :emphasize-lines: 12 - STATIC const char *const tok_kw[] = { + static const char *const tok_kw[] = { ... "or", "pass", @@ -157,7 +157,7 @@ The most relevant method you should know about is this: mp_compile_to_raw_code(parse_tree, source_file, is_repl, &cm); // Create and return a function object that executes the outer module. - return mp_make_function_from_raw_code(cm.rc, cm.context, NULL); + return mp_make_function_from_proto_fun(cm.rc, cm.context, NULL); } The compiler compiles the code in four passes: scope, stack size, code size and emit. @@ -301,7 +301,7 @@ code statement: .. code-block:: c - STATIC void emit_native_unary_op(emit_t *emit, mp_unary_op_t op) { + static void emit_native_unary_op(emit_t *emit, mp_unary_op_t op) { vtype_kind_t vtype; emit_pre_pop_reg(emit, &vtype, REG_ARG_2); if (vtype == VTYPE_PYOBJ) { diff --git a/docs/develop/gettingstarted.rst b/docs/develop/gettingstarted.rst index c2d3816d425a4..fed632ea1ac15 100644 --- a/docs/develop/gettingstarted.rst +++ b/docs/develop/gettingstarted.rst @@ -100,7 +100,7 @@ For the stm32 port, the ARM cross-compiler is required: .. code-block:: bash - $ sudo apt-get install arm-none-eabi-gcc arm-none-eabi-binutils arm-none-eabi-newlib + $ sudo apt-get install gcc-arm-none-eabi libnewlib-arm-none-eabi See the `ARM GCC toolchain `_ @@ -112,7 +112,7 @@ Check that you have Python available on your system: .. code-block:: bash $ python3 - Python 3.5.0 (default, Jul 17 2020, 14:04:10) + Python 3.5.0 (default, Jul 17 2020, 14:04:10) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> @@ -228,7 +228,7 @@ You can also specify which board to use: .. code-block:: bash $ cd ports/stm32 - $ make submodules + $ make BOARD= submodules $ make BOARD= See `ports/stm32/boards `_ @@ -245,7 +245,7 @@ that you use a virtual environment: $ python3 -m venv env $ source env/bin/activate - $ pip install sphinx + $ pip install -r docs/requirements.txt Navigate to the ``docs`` directory: @@ -278,7 +278,7 @@ To run a selection of tests on a board/device connected over USB use: .. code-block:: bash $ cd tests - $ ./run-tests.py --target minimal --device /dev/ttyACM0 + $ ./run-tests.py -t /dev/ttyACM0 See also :ref:`writingtests`. diff --git a/docs/develop/library.rst b/docs/develop/library.rst index c2a86ea1699bc..830211d81c862 100644 --- a/docs/develop/library.rst +++ b/docs/develop/library.rst @@ -48,16 +48,16 @@ hypothetical new module ``subsystem`` in the file ``modsubsystem.c``: #if MICROPY_PY_SUBSYSTEM // info() - STATIC mp_obj_t py_subsystem_info(void) { + static mp_obj_t py_subsystem_info(void) { return MP_OBJ_NEW_SMALL_INT(42); } MP_DEFINE_CONST_FUN_OBJ_0(subsystem_info_obj, py_subsystem_info); - STATIC const mp_rom_map_elem_t mp_module_subsystem_globals_table[] = { + static const mp_rom_map_elem_t mp_module_subsystem_globals_table[] = { { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_subsystem) }, { MP_ROM_QSTR(MP_QSTR_info), MP_ROM_PTR(&subsystem_info_obj) }, }; - STATIC MP_DEFINE_CONST_DICT(mp_module_subsystem_globals, mp_module_subsystem_globals_table); + static MP_DEFINE_CONST_DICT(mp_module_subsystem_globals, mp_module_subsystem_globals_table); const mp_obj_module_t mp_module_subsystem = { .base = { &mp_type_module }, diff --git a/docs/develop/natmod.rst b/docs/develop/natmod.rst index 6d15f867bcfe8..2ccd83288565c 100644 --- a/docs/develop/natmod.rst +++ b/docs/develop/natmod.rst @@ -39,7 +39,8 @@ options for the ``ARCH`` variable, see below): * ``armv7emsp`` (ARM Thumb 2, single precision float, eg Cortex-M4F, Cortex-M7) * ``armv7emdp`` (ARM Thumb 2, double precision float, eg Cortex-M7) * ``xtensa`` (non-windowed, eg ESP8266) -* ``xtensawin`` (windowed with window size 8, eg ESP32) +* ``xtensawin`` (windowed with window size 8, eg ESP32, ESP32S3) +* ``rv32imc`` (RISC-V 32 bits with compressed instructions, eg ESP32C3, ESP32C6) When compiling and linking the native .mpy file the architecture must be chosen and the corresponding file can only be imported on that architecture. For more @@ -69,11 +70,25 @@ The known limitations are: So, if your C code has writable data, make sure the data is defined globally, without an initialiser, and only written to within functions. +The native module is not automatically linked against the standard static libraries +like ``libm.a`` and ``libgcc.a``, which can lead to ``undefined symbol`` errors. +You can link the runtime libraries by setting ``LINK_RUNTIME = 1`` +in your Makefile. Custom static libraries can also be linked by adding +``MPY_LD_FLAGS += -l path/to/library.a``. Note that these are linked into +the native module and will not be shared with other modules or the system. + Linker limitation: the native module is not linked against the symbol table of the full MicroPython firmware. Rather, it is linked against an explicit table of exported symbols found in ``mp_fun_table`` (in ``py/nativeglue.h``), that is fixed at firmware build time. It is thus not possible to simply call some arbitrary HAL/OS/RTOS/system -function, for example. +function, for example, unless that resides at a fixed address. In that case, the path +of a linkerscript containing a series of symbol names and their fixed address can be +passed to ``mpy_ld.py`` via the ``--externs`` command line argument. That way symbols +appearing in the linkerscript will take precedence over what is provided from object +files, but at the moment the object files' implementation will still reside in the +final MPY file. The linkerscript parser is limited in its capabilities, and is +currently used only for parsing the ESP8266 port ROM symbols list (see +``ports/esp8266/boards/eagle.rom.addr.v6.ld``). New symbols can be added to the end of the table and the firmware rebuilt. The symbols also need to be added to ``tools/mpy_ld.py``'s ``fun_table`` dict in the @@ -128,7 +143,7 @@ The file ``factorial.c`` contains: #include "py/dynruntime.h" // Helper function to compute factorial - STATIC mp_int_t factorial_helper(mp_int_t x) { + static mp_int_t factorial_helper(mp_int_t x) { if (x == 0) { return 1; } @@ -136,7 +151,7 @@ The file ``factorial.c`` contains: } // This is the function which will be called from Python, as factorial(x) - STATIC mp_obj_t factorial(mp_obj_t x_obj) { + static mp_obj_t factorial(mp_obj_t x_obj) { // Extract the integer from the MicroPython input object mp_int_t x = mp_obj_get_int(x_obj); // Calculate the factorial @@ -145,7 +160,7 @@ The file ``factorial.c`` contains: return mp_obj_new_int(result); } // Define a Python reference to the function above - STATIC MP_DEFINE_CONST_FUN_OBJ_1(factorial_obj, factorial); + static MP_DEFINE_CONST_FUN_OBJ_1(factorial_obj, factorial); // This is the entry point and is called when the module is imported mp_obj_t mpy_init(mp_obj_fun_bc_t *self, size_t n_args, size_t n_kw, mp_obj_t *args) { @@ -172,7 +187,7 @@ The file ``Makefile`` contains: # Source files (.c or .py) SRC = factorial.c - # Architecture to build for (x86, x64, armv6m, armv7m, xtensa, xtensawin) + # Architecture to build for (x86, x64, armv6m, armv7m, xtensa, xtensawin, rv32imc) ARCH = x64 # Include to get the rules for compiling and linking the module diff --git a/docs/develop/optimizations.rst b/docs/develop/optimizations.rst index 7f2c8cbe7282c..533dd64116f03 100644 --- a/docs/develop/optimizations.rst +++ b/docs/develop/optimizations.rst @@ -33,7 +33,7 @@ Variables MicroPython processes local and global variables differently. Global variables are stored and looked up from a global dictionary that is allocated on the heap (note that each module has its own separate dict, so separate namespace). -Local variables on the other hand are are stored on the Python value stack, which may +Local variables on the other hand are stored on the Python value stack, which may live on the C stack or on the heap. They are accessed directly by their offset within the Python stack, which is more efficient than a global lookup in a dict. diff --git a/docs/develop/porting.rst b/docs/develop/porting.rst index 74974a39e1b4d..99725e1000bef 100644 --- a/docs/develop/porting.rst +++ b/docs/develop/porting.rst @@ -38,6 +38,7 @@ The basic MicroPython firmware is implemented in the main port file, e.g ``main. .. code-block:: c + #include "py/builtin.h" #include "py/compile.h" #include "py/gc.h" #include "py/mperrno.h" @@ -82,7 +83,7 @@ The basic MicroPython firmware is implemented in the main port file, e.g ``main. } // There is no filesystem so opening a file raises an exception. - mp_lexer_t *mp_lexer_new_from_file(const char *filename) { + mp_lexer_t *mp_lexer_new_from_file(qstr filename) { mp_raise_OSError(MP_ENOENT); } @@ -110,6 +111,9 @@ We also need a Makefile at this point for the port: shared/runtime/pyexec.c \ shared/runtime/stdout_helpers.c \ + # Define source files containung qstrs. + SRC_QSTR += shared/readline/readline.c shared/runtime/pyexec.c + # Define the required object files. OBJ = $(PY_CORE_O) $(addprefix $(BUILD)/, $(SRC_C:.c=.o)) @@ -147,9 +151,6 @@ The following is an example of an ``mpconfigport.h`` file: #define MICROPY_ERROR_REPORTING (MICROPY_ERROR_REPORTING_TERSE) #define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_FLOAT) - // Enable u-modules to be imported with their standard name, like sys. - #define MICROPY_MODULE_WEAK_LINKS (1) - // Fine control over Python builtins, classes, modules, etc. #define MICROPY_PY_ASYNC_AWAIT (0) #define MICROPY_PY_BUILTINS_SET (0) @@ -261,17 +262,17 @@ To add a custom module like ``myport``, first add the module definition in a fil #include "py/runtime.h" - STATIC mp_obj_t myport_info(void) { + static mp_obj_t myport_info(void) { mp_printf(&mp_plat_print, "info about my port\n"); return mp_const_none; } - STATIC MP_DEFINE_CONST_FUN_OBJ_0(myport_info_obj, myport_info); + static MP_DEFINE_CONST_FUN_OBJ_0(myport_info_obj, myport_info); - STATIC const mp_rom_map_elem_t myport_module_globals_table[] = { + static const mp_rom_map_elem_t myport_module_globals_table[] = { { MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(MP_QSTR_myport) }, { MP_ROM_QSTR(MP_QSTR_info), MP_ROM_PTR(&myport_info_obj) }, }; - STATIC MP_DEFINE_CONST_DICT(myport_module_globals, myport_module_globals_table); + static MP_DEFINE_CONST_DICT(myport_module_globals, myport_module_globals_table); const mp_obj_module_t myport_module = { .base = { &mp_type_module }, diff --git a/docs/develop/writingtests.rst b/docs/develop/writingtests.rst index 9bb5178f55f22..a7d033f17e9b9 100644 --- a/docs/develop/writingtests.rst +++ b/docs/develop/writingtests.rst @@ -60,7 +60,7 @@ Then to run on a board: .. code-block:: bash - $ ./run-tests.py --target minimal --device /dev/ttyACM0 + $ ./run-tests.py -t /dev/ttyACM0 And to run only a certain set of tests (eg a directory): diff --git a/docs/differences/modules_preamble.txt b/docs/differences/modules_preamble.txt new file mode 100644 index 0000000000000..1958f0084d2db --- /dev/null +++ b/docs/differences/modules_preamble.txt @@ -0,0 +1,33 @@ +.. Preamble section inserted into generated output + +Positional-only Parameters +-------------------------- + +To save code size, many functions that accept keyword arguments in CPython only accept positional arguments in MicroPython. + +MicroPython marks positional-only parameters in the same way as CPython, by inserting a ``/`` to mark the end of the positional parameters. Any function whose signature ends in ``/`` takes *only* positional arguments. For more details, see `PEP 570 `_. + +Example +~~~~~~~ + +For example, in CPython 3.4 this is the signature of the constructor ``socket.socket``:: + + socket.socket(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None) + +However, the signature documented in :func:`MicroPython` is:: + + socket(af=AF_INET, type=SOCK_STREAM, proto=IPPROTO_TCP, /) + +The ``/`` at the end of the parameters indicates that they are all positional-only in MicroPython. The following code works in CPython but not in most MicroPython ports:: + + import socket + s = socket.socket(type=socket.SOCK_DGRAM) + +MicroPython will raise an exception:: + + TypeError: function doesn't take keyword arguments + +The following code will work in both CPython and MicroPython:: + + import socket + s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) diff --git a/docs/differences/python_37.rst b/docs/differences/python_37.rst index 86d1b6e81f96d..2b9a089217f6d 100644 --- a/docs/differences/python_37.rst +++ b/docs/differences/python_37.rst @@ -105,4 +105,4 @@ Changes to built-in modules: .. rubric:: Notes -.. [#ftimenanosec] Only :func:`time.time_ns` is implemented. +.. [#ftimenanosec] Only :func:`time.time_ns` is implemented. diff --git a/docs/esp32/quickref.rst b/docs/esp32/quickref.rst index f74926c374602..c394414a76f51 100644 --- a/docs/esp32/quickref.rst +++ b/docs/esp32/quickref.rst @@ -18,6 +18,9 @@ working with this board it may be useful to get an overview of the microcontroll general.rst tutorial/index.rst +Note that there are several varieties of ESP32 -- ESP32, ESP32C3, ESP32C6, ESP32S2, ESP32S3 -- +supported by MicroPython, with some differences in functionality between them. + Installing MicroPython ---------------------- @@ -57,52 +60,57 @@ The :mod:`esp32` module:: import esp32 - esp32.hall_sensor() # read the internal hall sensor esp32.raw_temperature() # read the internal temperature of the MCU, in Fahrenheit - esp32.ULP() # access to the Ultra-Low-Power Co-processor + esp32.ULP() # access to the Ultra-Low-Power Co-processor, not on ESP32C3/C6 Note that the temperature sensor in the ESP32 will typically read higher than ambient due to the IC getting warm while it runs. This effect can be minimised by reading the temperature sensor immediately after waking up from sleep. +ESP32C3, ESP32C6, ESP32S2, and ESP32S3 also have an internal temperature sensor available. +It is implemented a bit differently to the ESP32 and returns the temperature in +Celsius:: + + esp32.mcu_temperature() # read the internal temperature of the MCU, in Celsius + Networking ---------- WLAN ^^^^ -The :mod:`network` module:: +The :class:`network.WLAN` class in the :mod:`network` module:: import network - wlan = network.WLAN(network.STA_IF) # create station interface - wlan.active(True) # activate the interface - wlan.scan() # scan for access points - wlan.isconnected() # check if the station is connected to an AP + wlan = network.WLAN() # create station interface (the default, see below for an access point interface) + wlan.active(True) # activate the interface + wlan.scan() # scan for access points + wlan.isconnected() # check if the station is connected to an AP wlan.connect('ssid', 'key') # connect to an AP - wlan.config('mac') # get the interface's MAC address - wlan.ifconfig() # get the interface's IP/netmask/gw/DNS addresses + wlan.config('mac') # get the interface's MAC address + wlan.ipconfig('addr4') # get the interface's IPv4 addresses - ap = network.WLAN(network.AP_IF) # create access-point interface - ap.config(ssid='ESP-AP') # set the SSID of the access point - ap.config(max_clients=10) # set how many clients can connect to the network - ap.active(True) # activate the interface + ap = network.WLAN(network.WLAN.IF_AP) # create access-point interface + ap.config(ssid='ESP-AP') # set the SSID of the access point + ap.config(max_clients=10) # set how many clients can connect to the network + ap.active(True) # activate the interface A useful function for connecting to your local WiFi network is:: def do_connect(): - import network - wlan = network.WLAN(network.STA_IF) + import machine, network + wlan = network.WLAN() wlan.active(True) if not wlan.isconnected(): print('connecting to network...') wlan.connect('ssid', 'key') while not wlan.isconnected(): - pass - print('network config:', wlan.ifconfig()) + machine.idle() + print('network config:', wlan.ipconfig('addr4')) Once the network is established the :mod:`socket ` module can be used -to create and use TCP/UDP sockets as usual, and the ``urequests`` module for +to create and use TCP/UDP sockets as usual, and the ``requests`` module for convenient HTTP requests. After a call to ``wlan.connect()``, the device will by default retry to connect @@ -113,44 +121,61 @@ calling ``wlan.config(reconnects=n)``, where n are the number of desired reconne attempts (0 means it won't retry, -1 will restore the default behaviour of trying to reconnect forever). +.. _esp32_network_lan: + LAN ^^^ -To use the wired interfaces one has to specify the pins and mode :: +Built-in MAC (original ESP32) +""""""""""""""""""""""""""""" + +The original ESP32 SoC has a built-in Ethernet MAC. Using this MAC requires an +external Ethernet PHY to be wired to the chip's EMAC pins. Most of the EMAC pin +assignments are fixed, consult the ESP32 datasheet for details. + +If the PHY is connected, the internal Ethernet MAC can be configured via +the :class:`network.LAN` constructor:: import network lan = network.LAN(mdc=PIN_MDC, ...) # Set the pin and mode configuration lan.active(True) # activate the interface - lan.ifconfig() # get the interface's IP/netmask/gw/DNS addresses + lan.ipconfig('addr4') # get the interface's IPv4 addresses -The keyword arguments for the constructor defining the PHY type and interface are: +Required keyword arguments for the constructor: -- mdc=pin-object # set the mdc and mdio pins. -- mdio=pin-object -- power=pin-object # set the pin which switches the power of the PHY device. -- phy_type= # Select the PHY device type. Supported devices are PHY_LAN8710, - PHY_LAN8720, PH_IP101, PHY_RTL8201, PHY_DP83848 and PHY_KSZ8041 -- phy_addr=number # The address number of the PHY device. -- ref_clk_mode=mode # Defines, whether the ref_clk at the ESP32 is an input - or output. Suitable values are Pin.IN and Pin.OUT. -- ref_clk=pin-object # defines the Pin used for ref_clk. +- ``mdc`` and ``mdio`` - :class:`machine.Pin` objects (or integers) specifying + the MDC and MDIO pins. +- ``phy_type`` - Select the PHY device type. Supported devices are + ``PHY_GENERIC``, + ``PHY_LAN8710``, ``PHY_LAN8720``, ``PHY_IP101``, ``PHY_RTL8201``, + ``PHY_DP83848``, ``PHY_KSZ8041`` and ``PHY_KSZ8081``. These values are all + constants defined in the ``network`` module. +- ``phy_addr`` - The address number of the PHY device. Must be an integer in the + range 0x00 to 0x1f, inclusive. Common values are ``0`` and ``1``. -The options ref_clk_mode and ref_clk require at least esp-idf version 4.4. For -earlier esp-idf versions, these parameters must be defined by kconfig board options. +All of the above keyword arguments must be present to configure the interface. -These are working configurations for LAN interfaces of popular boards:: +Optional keyword arguments: + +- ``reset`` - :class:`machine.Pin` object (or integer) specifying the PHY reset pin. +- ``power`` - :class:`machine.Pin` object (or integer) specifying a pin which + switches the power of the PHY device. +- ``ref_clk`` - :class:`machine.Pin` object (or integer) specifying the pin used + for the EMAC ``ref_clk`` signal. If not specified, the board default is used + (typically GPIO 0, but may be different if a particular board has Ethernet.) +- ``ref_clk_mode`` - Defines whether the EMAC ``ref_clk`` pin of the ESP32 + should be an input or an output. Suitable values are ``machine.Pin.IN`` and + ``machine.Pin.OUT``. If not specified, the board default is used + (typically input, but may be different if a particular board has Ethernet.) + +These are working configurations for LAN interfaces of some popular ESP32 boards:: # Olimex ESP32-GATEWAY: power controlled by Pin(5) # Olimex ESP32 PoE and ESP32-PoE ISO: power controlled by Pin(12) lan = network.LAN(mdc=machine.Pin(23), mdio=machine.Pin(18), power=machine.Pin(5), - phy_type=network.PHY_LAN8720, phy_addr=0) - - # or with dynamic ref_clk pin configuration - - lan = network.LAN(mdc=machine.Pin(23), mdio=machine.Pin(18), power=machine.Pin(5), phy_type=network.PHY_LAN8720, phy_addr=0, ref_clk=machine.Pin(17), ref_clk_mode=machine.Pin.OUT) @@ -159,20 +184,76 @@ These are working configurations for LAN interfaces of popular boards:: lan = network.LAN(mdc=machine.Pin(23), mdio=machine.Pin(18), phy_type=network.PHY_LAN8720, phy_addr=1, power=None) + # Wireless-Tag's WT32-ETH01 v1.4 + + lan = network.LAN(mdc=machine.Pin(23), mdio=machine.Pin(18), + phy_type=network.PHY_LAN8720, phy_addr=1, + power=machine.Pin(16)) + # Espressif ESP32-Ethernet-Kit_A_V1.2 lan = network.LAN(id=0, mdc=Pin(23), mdio=Pin(18), power=Pin(5), phy_type=network.PHY_IP101, phy_addr=1) -A suitable definition of the PHY interface in a sdkconfig.board file is:: - CONFIG_ETH_PHY_INTERFACE_RMII=y - CONFIG_ETH_RMII_CLK_OUTPUT=y - CONFIG_ETH_RMII_CLK_OUT_GPIO=17 - CONFIG_LWIP_LOCAL_HOSTNAME="ESP32_POE" +.. _esp32_spi_ethernet: + +SPI Ethernet Interface +"""""""""""""""""""""" + +All ESP32 SoCs support external SPI Ethernet interface chips. These are Ethernet +interfaces that connect via a SPI bus, rather than an Ethernet RMII interface. + +.. note:: The only exception is the ESP32 ``d2wd`` variant, where this feature is disabled + to save code size. + +SPI Ethernet uses the same :class:`network.LAN` constructor, with a different +set of keyword arguments:: + + import machine, network + + spi = machine.SPI(1, sck=SCK_PIN, mosi=MOSI_PIN, miso=MISO_PIN) + lan = network.LAN(spi=spi, cs=CS_PIN, ...) # Set the pin and mode configuration + lan.active(True) # activate the interface + lan.ipconfig('addr4') # get the interface's IPv4 addresses + +Required keyword arguments for the constructor: + +- ``spi`` - Should be a :class:`machine.SPI` object configured for this + connection. Note that any clock speed configured on the SPI object is ignored, + the SPI Ethernet clock speed is configured at compile time. +- ``cs`` - :class:`machine.Pin` object (or integer) specifying the CS pin + connected to the interface. +- ``int`` - :class:`machine.Pin` object (or integer) specifying the INT pin + connected to the interface. +- ``phy_type`` - Select the SPI Ethernet interface type. Supported devices are + ``PHY_KSZ8851SNL``, ``PHY_DM9051``, ``PHY_W5500``. These values are all + constants defined in the ``network`` module. +- ``phy_addr`` - The address number of the PHY device. Must be an integer in the + range 0x00 to 0x1f, inclusive. This is usually ``0`` for SPI Ethernet devices. + +All of the above keyword arguments must be present to configure the interface. -The value assigned to CONFIG_ETH_RMII_CLK_OUT_GPIO may vary depending on the -board's wiring. +Optional keyword arguments for the constructor: + +- ``reset`` - :class:`machine.Pin` object (or integer) specifying the SPI Ethernet + interface reset pin. +- ``power`` - :class:`machine.Pin` object (or integer) specifying a pin which + switches the power of the SPI Ethernet interface. + +Here is a sample configuration for a WIZNet W5500 chip connected to pins on +an ESP32-S3 development board:: + + import machine, network + from machine import Pin, SPI + + spi = SPI(1, sck=Pin(12), mosi=Pin(13), miso=Pin(14)) + lan = network.LAN(spi=spi, phy_type=network.PHY_W5500, phy_addr=0, + cs=Pin(10), int=Pin(11)) + +.. note:: WIZnet W5500 Ethernet is also supported on some other MicroPython + ports, but using a :ref:`different software interface + `. Delay and timing ---------------- @@ -190,8 +271,10 @@ Use the :mod:`time