Skip to content

RP2: Add Audio support for Arduino Nano RP2040. #8016

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

RP2: Add Audio support for Arduino Nano RP2040. #8016

wants to merge 2 commits into from
+1,501 −0

Conversation

iabdalkader
Copy link
Contributor

@iabdalkader iabdalkader commented Nov 19, 2021 •
edited
Loading

This PR adds the following:

  • openpdm: an opensource PDM to PCM library from ST. Note: a pre-computed LUT stored in flash was added to the library. The LUT saves about 500uS per conversion (as tested on RP2040), and can be enabled by defining USE_LUT=1.
  • Audio support for Nano-RP2040 using openpdm, PIO program, and a C-level audio module. The audio module supports a dynamically configurable queue of PCM buffers, and can detect and (optionally) abort on overflows when they occur. The API supports both a blocking mode to read buffers from the queue using get_buffer(), and a non-blocking mode with Python user callbacks. The module can be compile-time configured to use asynchronous events with pendsv (if available) or synchronous events with mp_sched_schedule for the Python callbacks.

Example usage:

Recording 5 seconds 16Khz audio in blocking mode, with 128 PCM buffers, and abort on overflow enabled.

import audio, time
audio.init(channels=1, frequency=16000, gain_db=8, buffers=128, overflow=True)

# Start audio streaming
audio.start_streaming(None)

print("recording audio...")
start = time.ticks_ms()
buf = bytearray()
while ((time.ticks_ms() - start) < 5000):
    buf.extend(audio.get_buffer(timeout=10))

# Stop streaming
audio.stop_streaming()
print("samples captured: {} overflow: {}".format(audio.samples(), audio.overflow()))

# Write out recorded PCM samples.
with open('test.pcm', 'w') as fout:
    fout.write(buf)

@iabdalkader iabdalkader mentioned this pull request Dec 29, 2021
Open
@iabdalkader
lib/openpdm: Add ST openpdm library.
f4d7ca1 
* An opensource PDM to PCM library from ST. Note, a pre-computed LUT
stored in flash was added to the library. The LUT saves about 500uS
per conversion (tested onRP2040), and can be enabled by defining USE_LUT=1.
@iabdalkader
rp2/boards/ARDUINO_NANO_RP2040: Add Audio support.
6d73b03
@iabdalkader iabdalkader force-pushed the arduino_nano_rp2040_audio branch from 958170f to 6d73b03 Compare January 20, 2022 19:09
@iabdalkader
Copy link
Contributor Author

Hi, @dpgeorge this has been ready for a while, it's the last missing feature for Nano-RP2040. Other than the openpdm lib, this PR/feature is completely self-contained to the board files. Note this was rebased on master, and split to 2 commits.

@iabdalkader
Copy link
Contributor Author

Can I get feedback on this @dpgeorge ?

@dpgeorge
Copy link
Member

dpgeorge commented Jun 3, 2022

Sorry this took a while to get to.

Let me start with a high-level review (rather than technical coding details). With a module called audio I would assume it was a general purpose audio module that would work on lots of boards and so belong in extmod/modaudio.c, and be similar to machine.I2S. (Note also that the microbit has a module called audio.)

But as far as I can tell, this PR adds a very specific driver for the MP34DT05 MEMS microphone, with an implementation that is tightly coupled to the rp2 port because it uses PIO (as well as PR2040 DMA). Is that PIO code 100% necessary? In other words, could this driver be adapted to work on an MCU without PIO?

Considering it's a driver for MP34DT05, it would be better to call it the mp34dt05 module (or something similar), and the code probably fits better in the drivers/ directory, maybe in drivers/codec/. You can then add a nice Python wrapper for the Nano connect board if you want to make it have a more convenient name/interface on that board (eg select the correct pins automatically).

Regarding the API for this module/class: it would be worth considering matching (as much as possible) the machine.I2S API, which has already been designed and tested to work in blocking, non-blocking and asyncio modes. It has streaming in and out modes. I think that I2S API would match well to the features of the MP34DT05.

In summary the theme here is:

  1. Writing code that's general and reusable across multiple ports and boards.
  2. Using an API that matches an existing API, and not having to invent something new.

@iabdalkader
Copy link
Contributor Author

But as far as I can tell, this PR adds a very specific driver for the MP34DT05 MEMS microphone, with an implementation that is tightly coupled to the rp2 port because it uses PIO (as well as PR2040 DMA). Is that PIO code 100% necessary? In other words, could this driver be adapted to work on an MCU without PIO?

This isn't a specific driver for MP34DT05, it should work with any digital MEMS microphone that outputs PDM. The PIO is only necessary on the rp2 port to sample the output, on the stm32 port, SAI or DFSDM can be used, so I don't think it should be moved it to something like codec/mp34 etc..

I realize it's tightly coupled to rp2, we could add extmod/modaudio.c and decouple the peripheral/DMA/port somehow (maybe with mpaudioport.c ?) Then microbit (is that a board or port), rp2, stm32 can all share a common audio API.

tannewt added a commit to tannewt/circuitpython that referenced this pull request May 25, 2023
@tannewt
Merge pull request micropython#8016 from charlieh0tel/new-mimxrt1060-…
5967732 
…evkb-board

Adds MIMXRT1060-EVKB as new board.
@iabdalkader iabdalkader deleted the arduino_nano_rp2040_audio branch June 24, 2023 07:46
@jonnor
Copy link
Contributor

jonnor commented Apr 7, 2024

Is there anything happening with this MR? It would be great to have PDM support for RP2040.

@iabdalkader
Copy link
Contributor Author

No, I just don't have the bandwidth to update this, but feel free to reuse the code and/or send it in another PR if you like (please keep the licenses if you do).

@jonnor
Copy link
Contributor

jonnor commented Apr 7, 2024

No problem. Thank you for the quick reply, and the code. I do not have capacity to work on this either at the moment. But now everyone knows that this is up for grabs to finish :)

@jonnor jonnor mentioned this pull request Apr 7, 2024
Closed
@jonnor jonnor mentioned this pull request May 6, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
port-rp2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@iabdalkader @dpgeorge @jonnor