Skip to content
This repository was archived by the owner on Sep 6, 2023. It is now read-only.

Wear levelling #126

Closed
wants to merge 5 commits into from
Closed

Wear levelling #126

wants to merge 5 commits into from

Conversation

loboris
Copy link

@loboris loboris commented Jul 8, 2017

esp-idf offers wear leveling driver which can greatly decrease the danger of wearing the flash with frequent writes.
Here are the changes needed to enable it in MicroPython.

@dpgeorge
Copy link
Member

Thanks for the contribution. A few things:

  • Did you test the changes? If so, what kind of tests and on which board?
  • You added flash_sec_size(), but didn't use it in _boot.py; was that an oversight?
  • Is the sector size tunable with the wear leveling?
  • How much flash does the leveling use for accounting, and where does it put the accounting data?

@loboris
Copy link
Author

loboris commented Jul 10, 2017

I've tested the changes for couple of days with simple write/append/read/verify procedures, with file sizes from 1 byte to 1 MB. I'm also using the wear leveling driver with an application written in C/esp-idf for more than two months, without any problem.
I've tested on SparkFun ESP32 Thing, ESP-WROVER-KIT v3 with WROVER module and on DIYmall ESPio32 (Widora-AIR clone).

flash_sec_size() should be used in flashbdev.py (SEC_SIZE = esp.flash_sec_size()), I'll change it.
Sector size is returned by the driver, that's the reason why I've included the function, but it is always 4096 (fixed in the driver), I think this is the minimal erase page size of the SPI Flash chip.
I think it is good idea to keep the function, as the wear leveling driver could change to include configurable sector size.

I've also tested SPIFFS on ESP32 and I think it is far better choice for file system on Flash. I have it (almost) working with MicroPython (I have some problems integrating it into VFS).

More details on wear leveling driver is available here.

@dpgeorge
Copy link
Member

Thanks for the info. I tested the patch and it indeed works well.

The two main things to consider are:

  • This changes the esp.flash_XXX functions to act on the virtual flash rather than physical. So there'd no longer be a way to read/erase/write an arbitrary flash page. Such functionality is useful, eg a user may want to use a number of flash pages for some other storage (like a simple block of data). And this may need to be in a specific physical address so it can be easily read/writen by a PC (eg to download data).
  • The size of the flash in this patch is hardcoded to 2MB. Users with modules with more than 4MB can't take advantage of the extra flash space.

…ort.h

Added RTC Class to machine module:
Added new methods to machine module:
Changed 'fatfsport.c' so that the correct timestamps are set on file change/create
Added sdcard driver which uses esp-idf sdmmc driver.
Added Makefile option to start esp-idf monitor (terminal emulator)
Added options to stay in bootloader after 'erase' and 'deploy'
Added simple bash script to make MicroPython build proces easier
@loboris
Copy link
Author

loboris commented Jul 12, 2017

Configuration options for wear leveling driver are added to mpconfigport.h.
Start address, size and encription options can be set.

Some additional changes more or less related to FAT file system:

Changed fatfsport.c so that the correct timestamps are set on file change/create

Added RTC Class to machine module:
rtc = machine.RTC()

Methods:
rtc.init((2017, 6, 12, 14, 35, 0, 0, 2))
rtc.now()
rtc.ntp_sync(server="<ntp_server>" [,update_period=])
<ntp_server> can be empty string, then the default server is used ("pool.ntp.org")
rtc.synced()
returns True if time synchronized to NTP server
rtc.wake_on_ext0(Pin, level)
rtc.wake_on_ext1(Pin, level)
wake up from deepsleep on pin level

Added new methods to machine module:
machine.deepsleep(time_ms)
machine.wake_reason()
returns tuple with reset & wakeup reasons
machine.wake_description()
returns tuple with strings describing reset & wakeup reasons

>>> import machine
>>> machine.deepsleep(5000)
ets Jun  8 2016 00:22:57

rst:0x5 (DEEPSLEEP_RESET),boot:0x3e (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0008,len:8
load:0x3fff0010,len:3412
load:0x40078000,len:9488
ho 0 tail 12 room 4
load:0x40080000,len:252
entry 0x40080034
I (1421) cpu_start: Pro cpu up.
I (1421) cpu_start: Single core mode
I (1422) heap_alloc_caps: Initializing. RAM available for dynamic allocation:
I (1435) heap_alloc_caps: At 3FFAE2A0 len 00001D60 (7 KiB): DRAM
I (1456) heap_alloc_caps: At 3FFD54A8 len 0000AB58 (42 KiB): DRAM
I (1477) heap_alloc_caps: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (1498) heap_alloc_caps: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (1520) heap_alloc_caps: At 40092944 len 0000D6BC (53 KiB): IRAM
I (1540) cpu_start: Pro cpu start user code
I (1600) cpu_start: Starting scheduler on PRO CPU.
OSError: [Errno 2] ENOENT
MicroPython v1.9.1-221-gdd6911bd-dirty on 2017-07-12; ESP32 module with ESP32
Type "help()" for more information.
>>> import machine
>>> machine.wake_reason()
(2, 3)
>>> machine.wake_description()
('Deepsleep reset', 'RTC wake')
>>> 

Added sdcard driver which uses esp-idf sdmmc driver.

If this driver is enabled is controled with mpconfig.h option MICROPY_SDMMC_USE_DRIVER
It works much better than SPI sdcard driver, and can be configured to work in 1-line mode (like SPI) or in 4-line mode (SD mode).
Tested in both modes and works without problem (some SDHC/SDXC cards can only work in 4-bit mode).

Usage:

import sdcard, uos, esp
sd = sdcard.SDCard(esp.SD_1LINE)
vfs = uos.VfsFat(sd)
uos.mount(vfs, '/sd')
uos.chdir('/sd')
uos.listdir()

If optional 'automount' parameter is used the card will be automaticaly mounted on /sd

>>> import sdcard,esp,uos
>>> sd = sdcard.SDCard(esp.SD_4LINE, True)
---------------------
Initializing SD Card: OK.
---------------------
 Mode:  SD (4bit)
 Name: SL08G
 Type: SDHC/SDXC
Speed: default speed (25 MHz)
 Size: 7580 MB
  CSD: ver=1, sector_size=512, capacity=15523840 read_bl_len=9
  SCR: sd_spec=2, bus_width=5

>>> uos.listdir()
['overlays', 'bcm2708-rpi-0-w.dtb', ......
>>> 

Some Makefile changes
Added Makefile option to start esp-idf monitor (terminal emulator)
Added options to stay in bootloader after 'erase' and 'deploy'

Added simple bash script to make MicroPython build proces easier
BUILD.sh in esp32 directory

@dpgeorge
Copy link
Member

Sorry but I can't accept the additional commits made to this PR. First, the usual way we do things is that a single PR should be for a single bit of added functionality, so this PR should have just the wear-levelling mods. Second, there's no way that GPL licensed code can be added because it clashes with the MIT license.

If you want to make progress here then 1) remove the changes unrelated to wear levelling; 2) open other PRs for the additional changes (as long as they don't involve GPL code, or code that is in any way connected to GPL code).

@loboris
Copy link
Author

loboris commented Jul 14, 2017

OK, thank you, I'll try do do it the right way,

@loboris loboris closed this Jul 14, 2017
c0d3z3r0 added a commit to c0d3z3r0/pycopy that referenced this pull request Aug 9, 2019
c0d3z3r0 added a commit to c0d3z3r0/pycopy that referenced this pull request Aug 11, 2019
This is a rework of loboris' wear leveling patch
micropython/micropython-esp32#126

Signed-off-by: Michael Niewöhner <foss@mniewoehner.de>
c0d3z3r0 added a commit to c0d3z3r0/pycopy that referenced this pull request Aug 11, 2019
This is a rework of loboris' wear leveling patch
micropython/micropython-esp32#126

Signed-off-by: Michael Niewöhner <foss@mniewoehner.de>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants