Skip to content

rp2: FatFS filesystem is not recreated after erasing the flash. #15779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
iabdalkader opened this issue Sep 3, 2024 · 8 comments · Fixed by #15785
Closed

rp2: FatFS filesystem is not recreated after erasing the flash. #15779

iabdalkader opened this issue Sep 3, 2024 · 8 comments · Fixed by #15785

Comments

@iabdalkader
Copy link
Contributor

iabdalkader commented Sep 3, 2024

Port, board and/or hardware

rp2 port, Arduino Nano-RP2040

MicroPython version

MicroPython v1.24.0-preview.260.g09d070aa5

Reproduction

This particular board has a 16MBytes flash, 2MBytes of which are reserved for the firmware, the rest is for the filesystem. A write of about ~3MBs should be enough to erase the firmware + the filesystem's partition table, but I tested with 16MBytes:

dd if=/dev/zero of=flash.bin bs=1M count=16

Get the board in bootloader mode, with double tap (on Arduino board) or jumper on other boards etc.. then write the binary to flash (erasing everything):

picotool load flash.bin

After it's done, and reset, the ROM bootloader runs. Load MicroPython, for example:

picotool load -x ARDUINO_NANO_RP2040_CONNECT.uf2

Try to use the filesystem:

>>> import os
>>> os.stat(".")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 19] ENODEV

Expected behaviour

I expected it to detect that there's no valid filesystem and re-create it.

Observed behaviour

The board boots MicroPython, the storage device is mounted, however it doesn't have a valid filesystem now:

>>> import os
>>> os.stat(".")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 19] ENODEV

>>> import vfs
>>> import machine, rp2
>>> bdev = rp2.Flash()
>>> fs = vfs.VfsFat(bdev)
>>> vfs.mount(fs, "/")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 19] ENODEV

Additional Information

No, I've provided everything above.

Code of Conduct

Yes, I agree

@iabdalkader iabdalkader added the bug label Sep 3, 2024
@robert-hh
Copy link
Contributor

Can you try to erase the flash by overwriting with 0xff either with the uploaded file for by using the write() or ioctl() methods of the block device.

@iabdalkader
Copy link
Contributor Author

@robert-hh I tried with 0xFF but there's no difference.

@projectgus
Copy link
Contributor

I don't have this exact board, but I couldn't reproduce this with an 8MB board I have: generated an 8MB file of zeroes, flashed with pictool the same way, and then loaded WEACTSTUDIO-FLASH_8M-20240903-v1.24.0-preview.278.g5e692d046.uf2. It booted OK and the filesystem was reformatted and empty.

I have a couple of suggestions to try and narrow down which part of this process might be failing:

First, after loading all zeroes, dump the flash back off and verify it's all zeroes:

picotool load flash.bin
picotool save --range 0x10000000 0x11000000 readback.bin
hexdump -C readback.bin  

Should see this:

00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
01000000

Second, try manually importing the frozen _boot.py file that mounts and formats the flash. i.e. open a REPL and type import _boot. If this succeeds there will be no output, but if formatting fails or mounting after formatting fails then you should see an error.

If manually running _boot.py seems to succeed and the filesystem works after this, then something has gone wrong in the manifest and/or module freezing and maybe the ports/rp2/modules/_boot.py file isn't being frozen into this board's build correctly (I had a quick check on master and it seems to be frozen correctly, but idk.)

@iabdalkader
Copy link
Contributor Author

@projectgus Thanks for testing! Note that this board uses FatFS not LFS2, I have a hunch that LFS2 can detect that there's no filesystem but fatfs can't.

Second, try manually importing the frozen _boot.py

Manually running _boot_fat.py fails, see Observed behaviour in the issue above.

@iabdalkader
Copy link
Contributor Author

@projectgus LFS2 works as expected; it detects invalid filesystem and recreates it automatically. FatFS fails to do so, even if MSC is disabled (it's unrelated but I thought I should rule out any issues with MSC).

@iabdalkader iabdalkader changed the title rp2: Filesystem is not recreated after erasing the flash. rp2: FatFS filesystem is not recreated after erasing the flash. Sep 4, 2024
@dpgeorge
Copy link
Member

dpgeorge commented Sep 4, 2024

I think the problem is that for FAT, running vfs.VfsFat(bdev) won't necessarily raise an exception even if the filesystem is corrupt. Instead, an exception will be raised when you do vfs.mount(bdev, "/").

Possible solutions:

  • in _boot_fat.py put the vfs.mount(fs, "/") line within the try-except, and if that mount fails then reformat the FAT
  • change the C code for vfs.VfsFat constructor to do an integrity check of the bdev to ensure the FAT is valid, and raise an exception if not

@Gadgetoid
Copy link
Contributor

Gadgetoid commented Sep 4, 2024

I think I ran into this with my multi-MSC experiments, and rewrote _boot_fat.py to https://github.com/pimoroni/micropython/blob/34afb41744f0a61edec4d16d8f8d924dc2c26010/ports/rp2/modules/_boot_fat.py

Cutting out all my garbage, _boot_fat.py should probably be:

import vfs
import machine, rp2


# Try to mount the filesystem, and format the flash if it doesn't exist.
bdev = rp2.Flash()
try:
    fs = vfs.VfsFat(bdev)
    vfs.mount(fs, "/")
except:
    vfs.VfsFat.mkfs(bdev)
    fs = vfs.VfsFat(bdev)
    vfs.mount(fs, "/")

del vfs, bdev, fs

I don't know why I didn't raise a PR for this at the time!

Edit:

  • change the C code for vfs.VfsFat constructor to do an integrity check of the bdev to ensure the FAT is valid, and raise an exception if not

I prefer this change since it more closely matches LittleFS.

@iabdalkader
Copy link
Contributor Author

  • change the C code for vfs.VfsFat constructor to do an integrity check of the bdev to ensure the FAT is valid, and raise an exception if not

I prefer this change since it more closely matches LittleFS.

@dpgeorge @Gadgetoid I can fix _boot_fat.py for now as suggested, but I agree that it's better to fix it in the C code, that way it gets fixed globally (for all ports, for any frozen boot.py that may be doing the same).

dpgeorge pushed a commit to iabdalkader/micropython that referenced this issue Sep 19, 2024
This change helps detect if the filesystem is invalid, by also including
the first mount attempt within the try-except.  Then the FAT is reformatted
if needed.

Fixes issue micropython#15779.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
graeme-winter pushed a commit to winter-special-projects/micropython that referenced this issue Sep 21, 2024
This change helps detect if the filesystem is invalid, by also including
the first mount attempt within the try-except.  Then the FAT is reformatted
if needed.

Fixes issue micropython#15779.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
wiznet-grace pushed a commit to wiznet-grace/micropython that referenced this issue Feb 27, 2025
This change helps detect if the filesystem is invalid, by also including
the first mount attempt within the try-except.  Then the FAT is reformatted
if needed.

Fixes issue micropython#15779.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
wiznet-grace pushed a commit to WIZnet-ioNIC/WIZnet-ioNIC-micropython that referenced this issue Feb 28, 2025
This change helps detect if the filesystem is invalid, by also including
the first mount attempt within the try-except.  Then the FAT is reformatted
if needed.

Fixes issue micropython#15779.

Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants