Skip to content

esp32/modesp32.c: add esp32 non volatile storage support #4707

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

upyfx
Copy link

@upyfx upyfx commented Apr 20, 2019

quick ref:

import esp32

esp32.nvs_set('testkey', 'testvalue')
esp32.nvs_get('testkey')
esp32.nvs_erase('testkey')
esp32.nvs_erase_all()

@upyfx
Copy link
Author

upyfx commented Apr 20, 2019

Please what is the right way to deliver a test (ex: directory, similar test) ?

@junhuanchen
Copy link

This is fun!

@dpgeorge
Copy link
Member

Thanks for the contribution, it looks quite good, to just read/write bytes objects.

@Lisa999 how does this compare to your implementation?

@dpgeorge
Copy link
Member

I see that the ESP IDF has functions to get/set a range of signed/unsigned integer types (8-64 bit). I don't know, is it worth exposing all of that functionality?

@Lisa999
Copy link

Lisa999 commented Apr 26, 2019

@dpgeorge: This is a nice try, but i have to disagree with this approach.
For example:
nvs_set_blob(nvs_handle, key, value, strlen(value));
It's assuming that value is a STRING, but it's using a blob function to store the string.

Here are my functions:
checkNVS()
mod_esp_nvs_set_int (mp_obj_t _key, mp_obj_t _value)
--> only using the i32 function...
mod_esp_nvs_get_int (mp_obj_t _key)
mod_esp_nvs_set_str (mp_obj_t _key, mp_obj_t _value)
--> Using: nvs_set_str(mpy_nvs_handle, key, value)
mod_esp_nvs_get_str (mp_obj_t _key)
mod_esp_nvs_erase (mp_obj_t _key)
mod_esp_nvs_erase_all (void)

example (code to see which reset has been performed for the next cycle):
esp.nvs_setstr('reset', 'rebootasync')

The only thing i'm doubting about is whether to put it into ESP or into ESP32...

I happy to make a PR, IF you promise to pull #3576 after @andynd has fixed it.

@upyfx
Copy link
Author

upyfx commented Apr 26, 2019

This follow Micropython's Django Principle.

@Lisa999
Copy link

Lisa999 commented Apr 26, 2019

Django? Django is all-inclusive, Flask Lightweight.
I would say that Micropython is more like Flask, everything should be lightweight and modular to avoid using huge amounts of memory.

Even so, blob's should be used to store binary data. If you want to store strings, then use the nvs_set_str function. https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/storage/nvs_flash.html

esp_err_tnvs_set_blob(nvs_handlehandle, const char *key, const void *value, size_t length)
set variable length binary value for given key

@dpgeorge
Copy link
Member

Thanks @Lisa999 for the input. It's probably a good idea to at least add "type" annotations to the method names so others can be added in the future if needed; such as nvs_set_blob(), nvs_set_i32(), etc.

Looking more at what the ESP IDF provides, I'd also suggest to make the NVS interface an object, rather than just functions in a module. This is because it's possible to open multiple NVS handles. It's like a mini database. Eg:

nvs = esp32.NVS("my_storage", "w") # open read/write
nvs.set_i32("key", 1234)
nvs2 = esp32.NVS("other_storage", "r") # open read-only
blob = nvs2.get_blob("key2")

@mattytrentini
Copy link
Contributor

See #4436 for a related issue.

@mattytrentini
Copy link
Contributor

ESP-IDF Non-volatile storage library docs can be found here:

https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/storage/nvs_flash.html

(Link above is broken though the text is correct)

@Lisa999
Copy link

Lisa999 commented Jul 20, 2019

Thanks @Lisa999 for the input. It's probably a good idea to at least add "type" annotations to the method names so others can be added in the future if needed; such as nvs_set_blob(), nvs_set_i32(), etc.

Looking more at what the ESP IDF provides, I'd also suggest to make the NVS interface an object, rather than just functions in a module. This is because it's possible to open multiple NVS handles. It's like a mini database. Eg:

nvs = esp32.NVS("my_storage", "w") # open read/write
nvs.set_i32("key", 1234)
nvs2 = esp32.NVS("other_storage", "r") # open read-only
blob = nvs2.get_blob("key2")

@dpgeorge: Since you build the Partition class for the ESP32, i'll keep my part of the bargain and have NVS working for Micropython v1.11 . However it's still functions based, do you have an example of a good object based class so i can rewrite my code?

@dpgeorge
Copy link
Member

do you have an example of a good object based class so i can rewrite my code?

I think the esp32.Partition() class is a really good example to base NVS on, see #4910.

Also, it might be neater to use subscription to read/write NVS values, similar to how machine.mem32[..] works. Eg:

nvs = esp32.NVS('storage', 'w')
nvs.i32['key'] = 1234 # store value
print(nvs.blob['key2']) # retrieve value

See extmod/machine_mem.c for code to implement that kind of API.

@nevercast
Copy link
Contributor

Its not clear to me why Partition class needed to be merged before implementing NVS. There is very little interaction between OTA/Partition functions and nvs functions.

@Lisa999 what was your intention in waiting for Partition functionality to be merged? Also are you working on this PR now ?

@Lisa999
Copy link

Lisa999 commented Aug 16, 2019

@nevercast : the Partition PR is not connected to this PR. I already have a working NVS class, however not object oriented but function oriented. @dpgeorge refered to the Partition class as an example how to build something like an object oriented class. The NVS OO class is on my to do list, probably going to start with the next few weeks.

@dmartauz
Copy link

@Lisa999 Any plans to finish OO implementation of NVS?

@Lisa999
Copy link

Lisa999 commented Oct 26, 2019

@Lisa999 Any plans to finish OO implementation of NVS?

Absolutely, as promised. Is on my todo list. However it gets overridden all the time by urgent matters that take up all of my time. I hope to make a start the next few weeks, i need to get this off my todo list. :-)

@pumelo
Copy link

pumelo commented Apr 9, 2020

@Lisa999 would you mind publishing your work on NVS somewhere? probably I can step in to finish the OO work, as I'm interested in NVS too ...

@remibert
Copy link

I do the merge on the commit 60f5b94, I attached the files merged, but I don't know to do this with github.
nvs.zip

dpgeorge pushed a commit that referenced this pull request Feb 19, 2021
This commit implements basic NVS support for the esp32.  It follows the
pattern of the esp32.Partition class and exposes an NVS object per NVS
namespace.  The initial support provided is only for signed 32-bit integers
and binary blobs.  It's easy (albeit a bit tedious) to add support for
more types.

See discussions in: #4436, #4707, #6780
@dpgeorge
Copy link
Member

This is superseded by the NVS supporte added in c10d431

@dpgeorge dpgeorge closed this Feb 19, 2021
tannewt pushed a commit to tannewt/circuitpython that referenced this pull request May 6, 2021
qstr: Use `const` consistently to avoid a cast.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants