Skip to content

tools/test_serial.py: Add test for serial throughput. #15909

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

dpgeorge
Copy link
Member

Summary

This is a test script used to test USB CDC (or USB UART) serial reliability and throughput.

Testing

Run against any MicroPython board with:

$ python test_serial.py <device>

@dpgeorge dpgeorge added the tools Relates to tools/ directory in source, or other tooling label Sep 25, 2024
@dpgeorge dpgeorge marked this pull request as draft September 25, 2024 07:49
Copy link

codecov bot commented Sep 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.54%. Comparing base (f9a755c) to head (37e86d6).

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #15909   +/-   ##
=======================================
  Coverage   98.54%   98.54%           
=======================================
  Files         169      169           
  Lines       21890    21890           
=======================================
  Hits        21572    21572           
  Misses        318      318           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@robert-hh
Copy link
Contributor

I've checked a few boards & ports with that test. Most work fine. SOme minor observations:

  • nrf: Sometimes fails. When the test is repeated, it works.
  • ESP32S3 at the CDC interface: Lots of timeouts. The UArt interface works fine.
  • ESP32C3 at the CDC interface: Data IN runs at 0.09MBit/s, Data OUT at 0.66/~3 MBit/s. That's odd.
  • ESP32S2: Locks often up in the DATA OUT, verify=1 sequence at larger buffer sizes
  • STM32F411 Blackpill: Locks up DATA OUT, verify=1 sequence at larger buffer sizes

@andrewleech
Copy link
Contributor

I've just tested on a esp32-S3 CDC with my shared/tinyusb migration PR: #15108
It works correctly without timeouts (@dpgeorge reported similar results previously on that PR)
DATA IN: 0.09 MBits/sec / DATA OUT: verify=0, bufsize=9999 1.14 MBits/sec

@robert-hh I've seen the same 0.09MBit/s DATA IN rates on renesas board, not sure why they're so low.

I also get lock-ups with the black pill board. I suspect it might be due to the 25mhz crystal causing instability - that is said to be the reason the built in DFU struggles to connect to usb more often than not.

On other stm32 boards though, I get much better speeds. WB55 for instance:

DATA IN: bufsize=256, read 32768 bytes in 63.21 msec = 506.22 kibytes/sec = 4.15 MBits/sec
DATA OUT: verify=0, bufsize=9999, wrote 639936 bytes in 1286.46 msec = 485.78 kibytes/sec = 3.98 MBits/sec

However for this I needed to increase the delay before the initial drain as the soft reboot took a little longer to complete.

def drain_input(ser):
    time.sleep(0.5)  # <---
    while ser.inWaiting() > 0:
        data = ser.read(ser.inWaiting())
        time.sleep(0.1)

rp2 (tinyusb)

DATA IN: bufsize=256, read 32768 bytes in 67.66 msec = 472.96 kibytes/sec = 3.87 MBits/sec
DATA OUT: verify=0, bufsize=9999, wrote 639936 bytes in 4998.38 msec = 125.03 kibytes/sec = 1.02 MBits/sec

@andrewleech
Copy link
Contributor

Ah... some of the slow data rates are caused by UART!
On renesas-ra, while I'm using the USB port only, the stdout goes to all ports and uart is a blocking write. Disabling uart repl raised the DATA IN rates at least from 0.09 MBits/sec up to 3-5 MBits/sec. The DATA OUT rates are still slow though.

@dpgeorge
Copy link
Member Author

Ah... some of the slow data rates are caused by UART!

Oh, yes, that's bitten me a few times. You need to disable UART if you want to test USB throughput.

You could also use this tool to test UART reliability, if you connect up a USB-UART convertor.

This is a test script used to test USB CDC (or USB UART) serial reliability
and throughput.

Run against any MicroPython board with:

    $ python test_serial.py <device>

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge force-pushed the tools-test-serial branch from 4e748c8 to 37e86d6 Compare April 30, 2025 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tools Relates to tools/ directory in source, or other tooling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants