micropython · dpgeorge · Jun 1, 2023 · May 10, 2023 · May 10, 2023 · May 10, 2023
diff --git a/docs/library/index.rst b/docs/library/index.rst
@@ -8,15 +8,17 @@ MicroPython libraries
    Important summary of this section
 
    * MicroPython provides built-in modules that mirror the functionality of the
-     Python standard library (e.g. :mod:`os`, :mod:`time`), as well as
-     MicroPython-specific modules (e.g. :mod:`bluetooth`, :mod:`machine`).
-   * Most standard library modules implement a subset of the functionality of
-     the equivalent Python module, and in a few cases provide some
-     MicroPython-specific extensions (e.g. :mod:`array`, :mod:`os`)
+     :ref:`Python standard library <micropython_lib_python>` (e.g. :mod:`os`,
+     :mod:`time`), as well as :ref:`MicroPython-specific modules <micropython_lib_micropython>`
+     (e.g. :mod:`bluetooth`, :mod:`machine`).
+   * Most Python standard library modules implement a subset of the
+     functionality of the equivalent Python module, and in a few cases provide
+     some MicroPython-specific extensions (e.g. :mod:`array`, :mod:`os`)
    * Due to resource constraints or other limitations, some ports or firmware
      versions may not include all the functionality documented here.
-   * To allow for extensibility, the built-in modules can be extended from
-     Python code loaded onto the device.
+   * To allow for extensibility, some built-in modules can be
+     :ref:`extended from Python code <micropython_lib_extending>` loaded onto
+     the device filesystem.
 
 This chapter describes modules (function and class libraries) which are built
 into MicroPython. This documentation in general aspires to describe all modules
@@ -41,6 +43,8 @@ Beyond the built-in libraries described in this documentation, many more
 modules from the Python standard library, as well as further MicroPython
 extensions to it, can be found in :term:`micropython-lib`.
 
+.. _micropython_lib_python:
+
 Python standard libraries and micro-libraries
 ---------------------------------------------
 
@@ -77,6 +81,7 @@ library.
    zlib.rst
    _thread.rst
 
+.. _micropython_lib_micropython:
 
 MicroPython-specific libraries
 ------------------------------
@@ -181,23 +186,48 @@ The following libraries are specific to the Zephyr port.
 
   zephyr.rst
 
+.. _micropython_lib_extending:
+
 Extending built-in libraries from Python
 ----------------------------------------
 
-In most cases, the above modules are actually named ``umodule`` rather than
-``module``, but MicroPython will alias any module prefixed with a ``u`` to the
-non-``u`` version. However a file (or :term:`frozen module`) named
-``module.py`` will take precedence over this alias.
+Many built-in modules are actually named ``umodule`` rather than ``module``, but
+MicroPython will alias any module prefixed with a ``u`` to the non-``u``
+version. This means that, for example, ``import time`` will first attempt to
+resolve from the filesystem, and then failing that will fall back to the
+built-in ``utime``. On the other hand, ``import utime`` will always go directly
+to the built-in.
 
 This allows the user to provide an extended implementation of a built-in library
-(perhaps to provide additional CPython compatibility). The user-provided module
-(in ``module.py``) can still use the built-in functionality by importing
-``umodule`` directly. This is used extensively in :term:`micropython-lib`. See
-:ref:`packages` for more information.
-
-This applies to both the Python standard libraries (e.g. ``os``, ``time``, etc),
-but also the MicroPython libraries too (e.g. ``machine``, ``bluetooth``, etc).
-The main exception is the port-specific libraries (``pyb``, ``esp``, etc).
+(perhaps to provide additional CPython compatibility or missing functionality).
+The user-provided module (in ``module.py``) can still use the built-in
+functionality by importing ``umodule`` directly (e.g. typically an extension
+module ``time.py`` will do ``from utime import *``). This is used extensively
+in :term:`micropython-lib`. See :ref:`packages` for more information.
+
+This extensibility applies to the following Python standard library modules
+which are built-in to the firmware: ``array``, ``binascii``, ``collections``,
+``errno``, ``hashlib``, ``heapq``, ``io``, ``json``, ``os``, ``platform``,
+``random``, ``re``, ``select``, ``socket``, ``ssl``, ``struct``, ``sys``,
+``time``, ``zlib``, as well as the MicroPython-specific libraries: ``bluepy``,
+``bluetooth``, ``machine``, ``timeq``, ``websocket``. All other built-in
+modules cannot be extended from the filesystem.
 
 *Other than when you specifically want to force the use of the built-in module,
 we recommend always using* ``import module`` *rather than* ``import umodule``.
+
+**Note:** In MicroPython v1.21.0 and higher, it is now possible to force an
+import of the built-in module by clearing ``sys.path`` during the import. For
+example, in ``time.py``, you can write::
+
+  _path = sys.path
+  sys.path = ()
+  try:
+    from time import *
+  finally:
+    sys.path = _path
+    del _path
+
+This is now the preferred way (instead of ``from utime import *``), as the
+``u``-prefix will be removed from the names of built-in modules in a future
+version of MicroPython.
diff --git a/examples/usercmodule/cexample/examplemodule.c b/examples/usercmodule/cexample/examplemodule.c
@@ -68,7 +68,7 @@ MP_DEFINE_CONST_OBJ_TYPE(
     locals_dict, &example_Timer_locals_dict
     );
 
-// Define all properties of the module.
+// Define all attributes of the module.
 // Table entries are key/value pairs of the attribute name (a string)
 // and the MicroPython object reference.
 // All identifiers and strings are written as MP_QSTR_xxx and will be

diff --git a/examples/usercmodule/cexample/micropython.mk b/examples/usercmodule/cexample/micropython.mk
@@ -1,9 +1,8 @@
-EXAMPLE_MOD_DIR := $(USERMOD_DIR)
+CEXAMPLE_MOD_DIR := $(USERMOD_DIR)
 
 # Add all C files to SRC_USERMOD.
-SRC_USERMOD += $(EXAMPLE_MOD_DIR)/examplemodule.c
+SRC_USERMOD += $(CEXAMPLE_MOD_DIR)/examplemodule.c
 
 # We can add our module folder to include paths if needed
 # This is not actually needed in this example.
-CFLAGS_USERMOD += -I$(EXAMPLE_MOD_DIR)
-CEXAMPLE_MOD_DIR := $(USERMOD_DIR)
+CFLAGS_USERMOD += -I$(CEXAMPLE_MOD_DIR)
diff --git a/examples/usercmodule/cppexample/examplemodule.c b/examples/usercmodule/cppexample/examplemodule.c
@@ -4,7 +4,7 @@
 // See example.cpp for the definition.
 STATIC MP_DEFINE_CONST_FUN_OBJ_2(cppfunc_obj, cppfunc);
 
-// Define all properties of the module.
+// Define all attributes of the module.
 // Table entries are key/value pairs of the attribute name (a string)
 // and the MicroPython object reference.
 // All identifiers and strings are written as MP_QSTR_xxx and will be

diff --git a/examples/usercmodule/subpackage/README.md b/examples/usercmodule/subpackage/README.md
@@ -0,0 +1 @@
+This is an example of a user C module that includes subpackages.
diff --git a/examples/usercmodule/subpackage/micropython.cmake b/examples/usercmodule/subpackage/micropython.cmake
@@ -0,0 +1,19 @@
+# Create an INTERFACE library for our C module.
+add_library(usermod_subpackage_example INTERFACE)
+
+# Add our source files to the lib
+target_sources(usermod_subpackage_example INTERFACE
+    ${CMAKE_CURRENT_LIST_DIR}/examplemodule.c
+)
+
+# Add the current directory as an include directory.
+target_include_directories(usermod_subpackage_example INTERFACE
+    ${CMAKE_CURRENT_LIST_DIR}
+)
+
+target_compile_definitions(usermod_subpackage_example INTERFACE
+    MICROPY_MODULE_BUILTIN_SUBPACKAGES=1
+)
+
+# Link our INTERFACE library to the usermod target.
+target_link_libraries(usermod INTERFACE usermod_subpackage_example)
diff --git a/examples/usercmodule/subpackage/micropython.mk b/examples/usercmodule/subpackage/micropython.mk
@@ -0,0 +1,10 @@
+SUBPACKAGE_EXAMPLE_MOD_DIR := $(USERMOD_DIR)
+
+# Add all C files to SRC_USERMOD.
+SRC_USERMOD += $(SUBPACKAGE_EXAMPLE_MOD_DIR)/modexamplepackage.c
+
+# We can add our module folder to include paths if needed
+# This is not actually needed in this example.
+CFLAGS_USERMOD += -I$(SUBPACKAGE_EXAMPLE_MOD_DIR) -DMICROPY_MODULE_BUILTIN_SUBPACKAGES=1
+
+QSTR_DEFS += $(SUBPACKAGE_EXAMPLE_MOD_DIR)/qstrdefsexamplepackage.h
diff --git a/examples/usercmodule/subpackage/modexamplepackage.c b/examples/usercmodule/subpackage/modexamplepackage.c
@@ -0,0 +1,84 @@
+// Include MicroPython API.
+#include "py/runtime.h"
+
+// Define example_package.foo.bar.f()
+STATIC mp_obj_t example_package_foo_bar_f(void) {
+    mp_printf(&mp_plat_print, "example_package.foo.bar.f\n");
+    return mp_const_none;
+}
+STATIC MP_DEFINE_CONST_FUN_OBJ_0(example_package_foo_bar_f_obj, example_package_foo_bar_f);
+
+// Define all attributes of the second-level sub-package (example_package.foo.bar).
+STATIC const mp_rom_map_elem_t example_package_foo_bar_globals_table[] = {
+    { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_example_package_dot_foo_dot_bar) },
+    { MP_ROM_QSTR(MP_QSTR_f), MP_ROM_PTR(&example_package_foo_bar_f_obj) },
+};
+STATIC MP_DEFINE_CONST_DICT(example_package_foo_bar_globals, example_package_foo_bar_globals_table);
+
+// Define example_package.foo.bar module object.
+const mp_obj_module_t example_package_foo_bar_user_cmodule = {
+    .base = { &mp_type_module },
+    .globals = (mp_obj_dict_t *)&example_package_foo_bar_globals,
+};
+
+// Define example_package.foo.f()
+STATIC mp_obj_t example_package_foo_f(void) {
+    mp_printf(&mp_plat_print, "example_package.foo.f\n");
+    return mp_const_none;
+}
+STATIC MP_DEFINE_CONST_FUN_OBJ_0(example_package_foo_f_obj, example_package_foo_f);
+
+// Define all attributes of the first-level sub-package (example_package.foo).
+STATIC const mp_rom_map_elem_t example_package_foo_globals_table[] = {
+    { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_example_package_dot_foo) },
+    { MP_ROM_QSTR(MP_QSTR_bar), MP_ROM_PTR(&example_package_foo_bar_user_cmodule) },
+    { MP_ROM_QSTR(MP_QSTR_f), MP_ROM_PTR(&example_package_foo_f_obj) },
+};
+STATIC MP_DEFINE_CONST_DICT(example_package_foo_globals, example_package_foo_globals_table);
+
+// Define example_package.foo module object.
+const mp_obj_module_t example_package_foo_user_cmodule = {
+    .base = { &mp_type_module },
+    .globals = (mp_obj_dict_t *)&example_package_foo_globals,
+};
+
+// Define example_package.f()
+STATIC mp_obj_t example_package_f(void) {
+    mp_printf(&mp_plat_print, "example_package.f\n");
+    return mp_const_none;
+}
+STATIC MP_DEFINE_CONST_FUN_OBJ_0(example_package_f_obj, example_package_f);
+
+STATIC mp_obj_t example_package___init__(void) {
+    if (!MP_STATE_VM(example_package_initialised)) {
+        // __init__ for builtins is called each time the module is imported,
+        //   so ensure that initialisation only happens once.
+        MP_STATE_VM(example_package_initialised) = true;
+        mp_printf(&mp_plat_print, "example_package.__init__\n");
+    }
+    return mp_const_none;
+}
+STATIC MP_DEFINE_CONST_FUN_OBJ_0(example_package___init___obj, example_package___init__);
+
+// The "initialised" state is stored on mp_state so that it is cleared on soft
+// reset.
+MP_REGISTER_ROOT_POINTER(int example_package_initialised);
+
+// Define all attributes of the top-level package (example_package).
+STATIC const mp_rom_map_elem_t example_package_globals_table[] = {
+    { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_example_package) },
+    { MP_ROM_QSTR(MP_QSTR___init__), MP_ROM_PTR(&example_package___init___obj) },
+    { MP_ROM_QSTR(MP_QSTR_foo), MP_ROM_PTR(&example_package_foo_user_cmodule) },
+    { MP_ROM_QSTR(MP_QSTR_f), MP_ROM_PTR(&example_package_f_obj) },
+};
+STATIC MP_DEFINE_CONST_DICT(example_package_globals, example_package_globals_table);
+
+// Define module object.
+const mp_obj_module_t example_package_user_cmodule = {
+    .base = { &mp_type_module },
+    .globals = (mp_obj_dict_t *)&example_package_globals,
+};
+
+// Register the module to make it available in Python.
+// Note: subpackages should not be registered with MP_REGISTER_MODULE.
+MP_REGISTER_MODULE(MP_QSTR_example_package, example_package_user_cmodule);
diff --git a/examples/usercmodule/subpackage/qstrdefsexamplepackage.h b/examples/usercmodule/subpackage/qstrdefsexamplepackage.h
@@ -0,0 +1,2 @@
+Q(example_package.foo)
+Q(example_package.foo.bar)
diff --git a/ports/unix/main.c b/ports/unix/main.c
@@ -664,7 +664,10 @@ MP_NOINLINE int main_(int argc, char **argv) {
                     return handle_uncaught_exception(nlr.ret_val) & 0xff;
                 }
 
-                if (mp_obj_is_package(mod) && !subpkg_tried) {
+                // If this module is a package, see if it has a `__main__.py`.
+                mp_obj_t dest[2];
+                mp_load_method_protected(mod, MP_QSTR___path__, dest, true);
+                if (dest[0] != MP_OBJ_NULL && !subpkg_tried) {
                     subpkg_tried = true;
                     vstr_t vstr;
                     int len = strlen(argv[a + 1]);
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		This is an example of a user C module that includes subpackages.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		Q(example_package.foo)
		Q(example_package.foo.bar)