This is almost ready to go in now.

I found that the best way to extend the block protocol to support littlefs was to make minimal changes:

• add an optional arg to readblocks and writeblocks for the offset into a block
• add a new ioctl to erase a block
  So the extended block protocol is:

class AbstractBlockDev:
    def readblocks(self, block_num, buf, offset=None):
        # if offset is not None then it's a byte offset into the block to start reading from
    def writeblocks(self, block_num, buf, offset=None):
        # if offset is not None then it's a byte offset into the block to start writing to
        # AND in such a case the block should not be erased before writing
    def ioctl(self, op, arg):
        if op == 6: # erase block
            # arg is block number to erase

This approach is simple, requires no new methods, and can coexist with the original block protocol (so the same device can be used for both a FAT FS and littlefs, for example).

I cleaned up the commits here to make it ready to merge.

Final thing to do is test for any regressions of FAT FS on bare-metal ports.

Do you know if anything is still using the old block device protocol with sync and count? Is it still necessary to keep that support?
I guess the code's already there, don't cost much size to have it.

Do you know if anything is still using the old block device protocol with sync and count? Is it still necessary to keep that support?

There may be user code out there. See #3861. Maybe in MicroPython 2.0 we can remove the old block device protocol...

Can this PR be used to create littlefs partition on ESP32? If so, could you please provide example code?

@dmartauz not yet, this is a first step. It currently is only integrated into the Unix port. Integration into the other ports will come in follow up PR's.

@andrewleech Ok, thank you for an explanation.

The last thing to decide here is naming of the new VFS classes.

Existing classes are:

• uos.VfsFat
• uos.VfsPosix

Options for littlefs would be:

• uos.VfsLfs1 and uos.VfsLfs2 -- short, matches existing classes
• uos.VfsLFS1 and uos.VfsLFS2 -- uppercase version of above
• uos.VfsLittle1 and uos.VfsLittle2 -- long version without "FS"
• uos.VfsLittleFs1 and uos.VfsLittleFs2 -- long version with "FS"
• uos.VfsLittleFS1 and uos.VfsLittleFS2 -- long version with uppercase "FS"

The last one is currently used in this PR. It's a bit long and cumbersome to type. I like the first option better (VfsLfs1): it's short, easy to type, matches existing classes, easy to tell what it is, and even though it's short it's not likely to clash with any other known filesystem.

So I'd suggest to use VfsLfs1 and VfsLfs2.

My vote is also for VfsLfs1 and VfsLfs2.

Lfs is already a common/standard shortening for littlefs so I funny think it's likely to cause any confusion. Shorter is easier to type and takes less bytes of qstr.

Ok, rebased and pushed with name changed to VfsLfs1 and VfsLfs2.

It would be good to have a brief description of the littlefs library renaming process used to bring in the two copies. to make it quicker/easier next time someone wants to update them. Even just a couple of lines of a link to the instructions in the respective commit messages?

On a related note, with the extended block device interface:

• add an optional arg to readblocks and writeblocks for the offset into a block

Would you prefer to see existing block drivers updated to expect the offset arg, or support the new argument as optional?
eg.
current

STATIC mp_obj_t pyb_flash_readblocks(mp_obj_t self, mp_obj_t block_num, mp_obj_t buf) {
...
}
STATIC MP_DEFINE_CONST_FUN_OBJ_3(pyb_flash_readblocks_obj, pyb_flash_readblocks);

new
expect new arg:

STATIC mp_obj_t pyb_flash_readblocks(mp_obj_t self, mp_obj_t block_num, mp_obj_t buf, mp_obj_t offset) {
...
}
STATIC MP_DEFINE_CONST_FUN_OBJ_4(pyb_flash_readblocks_obj, pyb_flash_readblocks);

or optional new arg:

STATIC mp_obj_t pyb_flash_readblocks(size_t n_args, const mp_obj_t *args) {
    mp_obj_base_t *self = (mp_obj_base_t*)MP_OBJ_TO_PTR(args[0]);
    mp_int_t block_num = mp_obj_get_int(args[1]);
    mp_obj_t buf = MP_OBJ_TO_PTR(args[2]);
    if (n_args == 4) { ... }
...
}
STATIC MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(pyb_flash_readblocks_obj, 3, 4, pyb_flash_readblocks);

It would be good to have a brief description of the littlefs library renaming process used to bring in the two copies.

Good idea. I pushed a commit here that adds a README.md to the lib/littlefs directory describing the process.

Would you prefer to see existing block drivers updated to expect the offset arg, or support the new argument as optional?

Existing code can stay as-is, everything is backwards compatible. If a block dev only wants to support FAT FS (and similar) then it just needs to implement the standard protocol (without the new offset arg). If a block dev only wants to support littlefs (and similar) then it only needs to implement the 4-arg version of the read/write blocks methods. OTOH, if a block dev wants to support both kinds of filesystems then it must implement the methods with an optional 4th argument.

I've added a new test extmod/vfs_blockdev.py to show how, at the Python level, a block dev can be implemented which supports bth standard and extended protocols at the same time (so 4th arg is optional).

Merged!