Skip to content

extmod/mod_network: Define network hostname globally for all nic boards. #9086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 137 commits into from
Closed

extmod/mod_network: Define network hostname globally for all nic boards. #9086

wants to merge 137 commits into from

Conversation

omogenot
Copy link
Contributor

@omogenot omogenot commented Aug 23, 2022

This PR proposes to globally move hostname definition to the network module, so that it will be defined for all nic boards by using the network.hostname() method. The default host name is set to mp-host and the hostname length is controlled by the MICROPY_PY_HOSTNAME_LENGTH constant which is defaulted to 32 if not defined.

>>> import network
>>> network.hostname("my-host")
>>> network.hostname()
'my-host'
>>>

Note: This PR would replace some of the modifications done specifically for the cyw43 board only (PR #8918), but keeps the ioctl functions up and running using this global hostname definition.

If the nic board uses the LWIP library, we take advantage of the integrated mDNS server (if the LWIP_MDNS_RESPONDER constant is defined) to declare hostname for that nic board.

>>> import network
>>> network.hostname("my-host")
>>> nic = network.WIZNET5K()
>>> nic.active(True)
>>> nic.ifconfig("dhcp")  
>>> # nic.ifconfig(("192.168.1.100", "255.255.255.0", "0.0.0.0", "0.0.0.0"))  # for fixed IP

From now on, on the network, you can perform a ping my-host.local to get a response from your micropython board.
I could not test these modifications for all nic boards, I have only WIZNET5K boards available.

omogenot and others added 6 commits August 23, 2022 12:29
extmod/modnetwork.c:
extmod/modnetwork.h: Added hostname variable and
get/set functions. If the NIC board uses LWIP
library, the hostame is publish through mDNS.
Make mod_network_nic_type_t definition globally used
by all NIC boards.

drivers/cyw43/cyw43.h
drivers/cyw43/cyw43_ctrl.c
drivers/cyw43/cyw43_lwip.c
extmod/network_cyw43.c
extmod/network_cyw43.h: Remove previous hostname definition
to use the global one. Use mod_network_nic_type_t object for
nic board access.

extmod/network_wiznet5k.c: Use mod_network_nic_type_t object.

ports/stm32/eth.c:
ports/stm32/eth.h:
ports/stm32/mpconfigport.h:
ports/stm32/network_lan.c:
ports/mimxrt/eth.c:
ports/mimxrt/eth.h:
ports/mimxrt/mpconfigport.h:
ports/mimxrt/network_lan.c: Move struct eth definition to
eth.h to be able to use mod_network_nic_type_t object for
the nic board access.

ports/rp2/mpconfigport.h: Use mod_network_nic_type_t object.
extmod/modnetwork.c: Add #if LWIP_NETIF_HOSTNAME
to check if netif struct has a hostname member.
Following the usual style of instantiation and init().
So that SDRAM can be used as the heap on ARDUINO_PORTENTA_H7, for example.

Fixes issue #9087.
@dpgeorge dpgeorge added the extmod Relates to extmod/ directory in source label Aug 25, 2022
pi-anl and others added 23 commits August 25, 2022 17:02
If mboot is built with support for packing (signing/encryption) it needs up
to 32KiB.  So for simplicity increase the mboot region to 32KiB
unconditionally for WB55 boards (custom WB55 board configurations can still
provide their own linker scripts to override this).
Rather than having the autobuild know about the particular variants, have
the mpconfigboard.mk describe them and make autobuild discover them
automatically.

Adds a "query-variants" target to stm32/Makefile to allow the set of
possible variants to be queried.

Removes pybv3 from the autobuild as this isn't use by the downloads page.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Not safe to use mp_hal_delay_ms before boot if threading is enabled,
because threading will not have been initialised, and
MICROPY_EVENT_POLL_HOOK assumes threading is initialised.

HAL_Delay doesn't call MICROPY_EVENT_POLL_HOOK, but is still
power-efficient like mp_hal_delay_ms (unlike mp_hal_delay_us).

Fixes #7816.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
It has been about 8 years since support for this chip was added.  Reasons
to remove it are:
- It is no longer easy to obtain this part.
- There are now many other options for WiFi.
- It's not a good use of developer time to maintain it.

Signed-off-by: Damien George <damien@micropython.org>
It's no longer used by any port.

Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
Signed-off-by: Damien George <damien@micropython.org>
Also document support for 'cp :a :b'.

Signed-off-by: Damien George <damien@micropython.org>
Updated some of the CPython feature differences:
- Updated status of some features.
- Added CSS to fix table widths to 100% and word wrap.
- Specified explicit table column ratios to improve layout appearance.
- Added missing references to anchors.
- Better consistency with use of formatting and case.
Signed-off-by: Tomasz 'CeDeROM' CEDRO <tomek@cedro.info>
* `init()` can be called multiple times to reconfigure UART.
* After `deinit()` it is impossible to call `init()` again.

Signed-off-by: Tomasz 'CeDeROM' CEDRO <tomek@cedro.info>
These can't be nested, so apply styling separately.

Signed-off-by: Laurens Valk <laurens@pybricks.com>
Signed-off-by: Laurens Valk <laurens@pybricks.com>
The existing non-Docker instructions are basically impossible to follow
because the esp-open-sdk does not compile.  Update these instructions to
use the exact toolchain that our CI uses.

Also split the Docker from non-Docker instructions, to avoid confusion
about which commands need to be prefixed.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Previously the desired output type was specified.  Now make the type part
of the function name.  Because this function is used in a few places this
saves code size due to smaller call-site.

This makes `mp_obj_new_str_type_from_vstr` a private function of objstr.c
(which is almost the only place where the output type isn't a compile-time
constant).

This saves ~140 bytes on PYBV11.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Now that we have `mp_obj_new_str_type_from_vstr` (private helper used by
objstr.c) split from the public API (`mp_obj_new_str_from_vstr`), we can
enforce a unicode check at the public API without incurring a performance
cost on the various objstr.c methods (which are already working on known
unicode-safe strings).

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The new `mp_obj_new_str_from_utf8_vstr` can be used when you know you
already have a unicode-safe string.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
All uses of this are either tiny strings or not-known-to-be-safe.

Update comments for mp_obj_new_str_copy and mp_obj_new_str_of_type.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
There don't seem to be many changes going from v2.1.2 to v2.1.3 of lwIP.
Mostly they are:
- IPv6 fixes and improvements
- changes to apps and other code that MicroPython doesn't use
- comments and tests
- minor bug fixes

In particular there doesn't look to be any change to the API of any
function used by MicroPython.

Network multi tests pass on PYBD_SF2 and PYBD_SF6.  PYBD_SF2, PYBD_SF6 and
PICO_W have unchanged iperf3 performance.  Similar results for networking
on the mimxrt port.

Signed-off-by: Damien George <damien@micropython.org>
Having two separate manifests is confusing.  It's simpler to have the daily
builds use the same configuration as the stable, release builds.

Signed-off-by: Damien George <damien@micropython.org>
jimmo and others added 14 commits September 19, 2022 19:06
The check for make_new (i.e. used to determine something's type) is now
more complicated due to the slot access.  This commit changes the inlining
of a few frequently-used helpers to overall improve code size and
performance.
The intent is to allow us to make breaking changes to the native ABI (e.g.
changes to dynruntime.h) without needing the bytecode version to increment.

With this commit the two bits previously used for the feature flags (but
now unused as of .mpy version 6) encode a sub-version.  A bytecode-only
.mpy file can be loaded as long as MPY_VERSION matches, but a native .mpy
(i.e. one with an arch set) must also match MPY_SUB_VERSION.  This allows 3
additional updates to the native ABI per bytecode revision.

The sub-version is set to 1 because the previous commits that changed the
layout of mp_obj_type_t have changed the native ABI.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
This matches class `__dict__`, and is similarly gated on
MICROPY_CPYTHON_COMPAT. Unlike class though, because modules's globals are
actually dict instances, the result is a mutable dictionary.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Useful when more detail is needed for an OSError associated with a file.

Signed-off-by: Damien George <damien@micropython.org>
This improves error messages in mpy-cross:
- When loading a .py file that doesn't exist (or can't be opened) it now
  includes the filename in the OSError.
- When saving a .mpy file that can't be opened it now raises an exception
  (prior, it would silently fail), and includes the filename in the
  OSError.

Signed-off-by: Damien George <damien@micropython.org>
Prevent handle leaks when file objects aren't closed explicitly and
fix some MICROPY_CPYTHON_COMPAT issues: this wasn't properly adhered
to because #ifdef was used so it was always on, and closing files
multiple times should be avoided unconditionally.
frozentest.mpy was previously duplicated in ports/minimal and
ports/powerpc.

This needs to be re-generated on every .mpy version increase, so might as
well just have a single copy of it.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This uses the frozentest.mpy that is also used by ports/minimal.

Also fixes two bugs that these new tests picked up:
 - File extension matching in manifestfile.py.
 - Handling of freeze_mpy results in makemanifest.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Allows running from a different directory, etc.

This work was funded by Planet Innovation.
extmod/modnetwork.c:
extmod/modnetwork.h: Added hostname variable and
get/set functions. If the NIC board uses LWIP
library, the hostame is publish through mDNS.
Make mod_network_nic_type_t definition globally used
by all NIC boards.

drivers/cyw43/cyw43.h
drivers/cyw43/cyw43_ctrl.c
drivers/cyw43/cyw43_lwip.c
extmod/network_cyw43.c
extmod/network_cyw43.h: Remove previous hostname definition
to use the global one. Use mod_network_nic_type_t object for
nic board access.

extmod/network_wiznet5k.c: Use mod_network_nic_type_t object.

ports/stm32/eth.c:
ports/stm32/eth.h:
ports/stm32/mpconfigport.h:
ports/stm32/network_lan.c:
ports/mimxrt/eth.c:
ports/mimxrt/eth.h:
ports/mimxrt/mpconfigport.h:
ports/mimxrt/network_lan.c: Move struct eth definition to
eth.h to be able to use mod_network_nic_type_t object for
the nic board access.

ports/rp2/mpconfigport.h: Use mod_network_nic_type_t object.
extmod/modnetwork.c: Add #if LWIP_NETIF_HOSTNAME
to check if netif struct has a hostname member.
@robert-hh
Copy link
Contributor

This PR looks like a git fault.

@omogenot
Copy link
Contributor Author

@robert-hh

This PR looks like a git fault.

This git stuff is really a pain... All this to resolve the conflicts introduced by the new MP_DEFINE_CONST_OBJ_FULL_TYPE macro usage in master.

@omogenot
Copy link
Contributor Author

Commit message formatting error is not mine (don't know what to do...):

error: commit fe55984: Unwanted email address: peterhinch@users.noreply.github.com

@robert-hh
Copy link
Contributor

How did you attempt to update? With git merge master or with git rebase master. I usually use git rebase to keep my PRs up-to-date. And I do it frequently. There were quiet a few git bombs recently which required a major rework of the PRs.

@omogenot
Copy link
Contributor Author

@robert-hh
I did a git rebase master, after having re-synced my master via the GitHub web page Sync fork button.
But since, there were modifications in the same files as the one I modified, I had to resolve conflicts. I solved them, etc.
Then I ran a multiple build bash command file to check that everything was OK, before pushing the branch.
And for any strange reason, the >>> Git conflict sign was added in the mod network.h file afterwards (since the multiple build worked before). This git stuff is really driving me nuts.
The funny thing is that my master is in sync with the micropython:master: This branch is up to date with micropython/micropython:master., but my global-hostname brach is still behind the master This branch is [9 commits ahead], [14 commits behind] micropython:master..
That's totally illogical, but I don't care as long as my RP now includes the master modified files with my own mods.
But as you can see, it still states: This branch cannot be rebased due to conflicts without showing any file being in conflict.
This git does whatever it wants...

extmod/modnetwork.c:
extmod/modnetwork.h: Added hostname variable and
get/set functions. If the NIC board uses LWIP
library, the hostame is publish through mDNS.
Make mod_network_nic_type_t definition globally used
by all NIC boards.

drivers/cyw43/cyw43.h
drivers/cyw43/cyw43_ctrl.c
drivers/cyw43/cyw43_lwip.c
extmod/network_cyw43.c
extmod/network_cyw43.h: Remove previous hostname definition
to use the global one. Use mod_network_nic_type_t object for
nic board access.

extmod/network_wiznet5k.c: Use mod_network_nic_type_t object.

ports/stm32/eth.c:
ports/stm32/eth.h:
ports/stm32/mpconfigport.h:
ports/stm32/network_lan.c:
ports/mimxrt/eth.c:
ports/mimxrt/eth.h:
ports/mimxrt/mpconfigport.h:
ports/mimxrt/network_lan.c: Move struct eth definition to
eth.h to be able to use mod_network_nic_type_t object for
the nic board access.

ports/rp2/mpconfigport.h: Use mod_network_nic_type_t object.
extmod/modnetwork.c: Add #if LWIP_NETIF_HOSTNAME
to check if netif struct has a hostname member.
extmod/modnetwork.c:
extmod/modnetwork.h: Added hostname variable and
get/set functions. If the NIC board uses LWIP
library, the hostame is publish through mDNS.
Make mod_network_nic_type_t definition globally used
by all NIC boards.

drivers/cyw43/cyw43.h
drivers/cyw43/cyw43_ctrl.c
drivers/cyw43/cyw43_lwip.c
extmod/network_cyw43.c
extmod/network_cyw43.h: Remove previous hostname definition
to use the global one. Use mod_network_nic_type_t object for
nic board access.

extmod/network_wiznet5k.c: Use mod_network_nic_type_t object.

ports/stm32/eth.c:
ports/stm32/eth.h:
ports/stm32/mpconfigport.h:
ports/stm32/network_lan.c:
ports/mimxrt/eth.c:
ports/mimxrt/eth.h:
ports/mimxrt/mpconfigport.h:
ports/mimxrt/network_lan.c: Move struct eth definition to
eth.h to be able to use mod_network_nic_type_t object for
the nic board access.

ports/rp2/mpconfigport.h: Use mod_network_nic_type_t object.
@robert-hh
Copy link
Contributor

@omogenot You cannot do it like that. There are 137 commits and 520 changed files in this PR, which aims to change a single feature. So something went fundamentally wrong, and you have to start over.

The structures I use are:
I have called the repository at micropython.org the upstream repository, and my fork of it is the origin repository. You may name them different, it's just names.
When I want to sync to a changed master, I change locally in to the master branch and run git pull upstream master --tags.
When I want to resync a branch (let's call it new_feature), I checkout into that branch with git checkout new_feature and rebase it with git rebase master. The rebase might bring up conflicts or even worse silent merge errors, which have to be resolved. If finally you run e.f. git log --online, you should see the master branch and on top of that your commit. Example below.
Then I might push it to my fork with git push origin my_feature -f. The -f option is required after the rebase.

Example of a rebased branch. Note the line with the upstream/master tag:

336696883 (mimxrt_qecnt) mimxrt: Adapt for PR8813 - Optimise mp_obj_type_t storage.
fea4ffb8f mimxrt: Fixes to the documentation of the Encoder/Counter class.
6f2d90414 mimxrt: Add the documentation for the Encoder/Counter class.
9fb21ebde mimxrt: Add settings for MIMXRT1015.
20fe65df7 mimxrt: Add missing definition for the OLIMEX_RT1010.
ee6ca5339 mimxrt: Implement a Quadrature Encoder and Counter class.
bdbc44474 (upstream/master, origin/master, origin/HEAD, master) rp2/boards/WEACTSTUDIO: Add WEACTSTUDIO with multiple variants.
0bc1d1055 rp2/Makefile: Add support for BOARD_VARIANTS.
9d6f474ea py/objstr: Don't treat bytes as unicode in str.count.
dd9dcb594 esp32/machine_pwm: Don't use LEDC_USE_REF_TICK on ESP32-C3 variants.
3abcfb9ae esp32/modsocket: Use mp_obj_is_integer to test port type.
e6d351318 stm32/boards/NUCLEO_L152RE: Add NUCLEO-L152RE board support.
427d72667 stm32: Add support for STM32L1 MCUs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
extmod Relates to extmod/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.