Add SSL support to WebREPL #5611
Conversation
extmod/webrepl/webrepl.py
Outdated
| websocket_helper.server_handshake(cl) | ||
| if websslrepl: | ||
| if hasattr(uos, 'dupterm_notify'): | ||
| cl.setsockopt(socket.SOL_SOCKET, 20, uos.dupterm_notify) |
There was a problem hiding this comment.
What is this dupterm_notify about? (Naive question)
There was a problem hiding this comment.
My idea was to try to modify webrepl.py as little as possible to get SSL working and therefore increase the least amount of bytes so it does not have an impact in more constrained devices (esp8266). So yes since this was already commented I decided to not duplicate it.
extmod/webrepl/webrepl.py
Outdated
| cl = ssl.wrap_socket(cl, server_side=True, key=key, cert=cert) | ||
| websocket_helper.server_handshake(cl, ssl=True) | ||
| else: | ||
| websocket_helper.server_handshake(cl) | ||
| ws = uwebsocket.websocket(cl, True) | ||
| ws = _webrepl._webrepl(ws) | ||
| cl.setblocking(False) | ||
| # notify REPL on socket incoming data (ESP32/ESP8266-only) |
There was a problem hiding this comment.
Ah, I guess this comment answers my question above... It would have been nice to keep the ssl/non-ssl code together with the comment...
extmod/webrepl/websocket_helper.py
Outdated
| l = cl.readline() | ||
| # print(l) | ||
| while 1: | ||
| if not ssl: |
There was a problem hiding this comment.
Dumb question: why does the handshake have to be different for wss?
There was a problem hiding this comment.
The main reason of a big percentage of changes is due to sockets and ssl sockets not having the same methods. SSL socket doesn't have a .makefile neither send or recv as a normal socket.
extmod/webrepl/webrepl.py
Outdated
|
|
||
|
|
||
| def set_ssl(flag): | ||
| with open('ssl_flag.py', 'wb') as sslconfig: |
There was a problem hiding this comment.
Is there a reason to create a separate config file just for the SSL flag? Maybe I'm misunderstanding...
There was a problem hiding this comment.
This is optional I think. Although is useful to change from ws to wss without modifying main.py
e.g. if you include this in main.py :
...
from ssl_flag import SSL
if SSL:
webrepl.start(ssl=True)
else:
webrepl.start()
So at the repl you can do at any time webrepl.set_ssl(True) or webrepl.set_ssl(False)
and after a soft/hard reset webrepl will start on ws or wss.
|
Hi @tve thanks for reviewing this. Take this PR as a "white canvas" since here are decisions that shouldn't be made by just one person.
Yes definitely, I already write something about that in here #5266, so as soon as https access is solved, I can put together the docs before merging. (Also there is #5543, so I have to do tests with latest releases too..) |
Don't use reset reason to skip writing boot_out.txt
|
Hi all. This PR has more than two years. Sorry, but I would like to know why it was not merged yet to mainstream? Add SSL (secure websocket) is a great feature. Thank you so much. |
|
Hi @Carglglz
That's great. I will test those examples. Anyway, now I'm curious, why after so long time, this PR still not in the mainstream? Maybe are there some conflict with this PR and some else? Actually I and everyone that use the binary release are using the Web REPL without secure. Well, I will learn how to create my own binary Micropython firmware and try to add add this PR. Thank you so much. |
@beyonlo you don't need to create your own firmware, check
So my version requires that a key and a certificate to be present in the device following that naming pattern (
You can check for context my initial issue #5543, lack of support of WebREPL website #5266, and the awesome ongoing work of @tve at #5840 and other folks, e.g. #7315. |
|
We definitely want to have a way of doing a secure network connection to a remote device. And this is a good starting point for that. But there are things to resolve first, before merging this. Eg how to provision a device with cert/keys. Also we want to move to So let's keep this open. |
I agree
About that, I've put together a
I didn't make a PR/Draft since I don't know where this should be included or if this should be included by default. Also for future work about provisioning a device with certs signed by a CA, I've looked into implement a protocol ie ACME to automate cert renewal, but I'm not sure exactly since it seems to be server oriented only (ie HTTPS). |
I'm a big fan of HTTPS but the moment you touch web browsers it's a total PITA for embedded devices. I would highly recommend testing various set-ups using an rPi or other linux box before hacking mpy! I recently worked on an embedded rPi project and reached the conclusion that the only viable approach is http://local-ip.co (which is a slightly simplified version of https://words.filippo.io/how-plex-is-doing-https-for-all-its-users/). The reason for all this is that all browsers restrict the bypassing of HTTPS checks more and more. It used to be just a warning, now you have to click through multiple warnings, except that some browsers (on windows in particular) no longer offer a bypass. There are a couple of good stackoverflow Q&A. Of course, if you assume everyone who will use the webrepl is a hacker and can find a browser that still works or uses a non-browser client then any solution will be fine... |
You may want to have a look at microdot efforts to make it work, and I think right now both HTTPS and WSS work fine. So there should no problem using browser + WebREPL once this
is figured out. |
|
I've had TLS + MQTT + uasyncio + REPL working for several years on esp32, I have a good number of devices in "production" with great results. I'm not doing any non-TLS comm anymore. I like to use MQTT 'cause this way there is no issue about certs on the device, the certs are only on the server and I have the infrastructure to maintain LE certs there. The experience I've had is that I used self-signed certs in an rPi project. It worked great for me on linux and android. But as soon as I handed the code out to "normal people" to test all hell broke loose. Some had trouble with all the hoops you have to jump through, others has systems/browsers that simply blocked the self-signed certs without offering any hoops to jump through. I then switched to the local-ip method and that solved pretty much all problems for me. YMMV... |
|
I'm using In my opinion to have the possibility to run the |
|
Blindly trusting self-signed certificates, exceptions, local-ip.co, etc are all completely insecure and defeat the entire purpose of TLS. Does anyone have a real solution? :/ |
|
"completely insecure" is not accurate, you're just lumping everything into one bucket. Using something like local-ip.co with PFC cipher suites gives you a lot of security at the encryption level. What it doesn't give you is authentication of the server, e.g., someone else could take over the IP address and claim to be the server you want to connect to. There are other ways to ensure that the server you ended up connecting to is the one you wanted. In any case, it raises the barrier a whole lot over non-encrypted connections. If you want to go all the way, it's not that difficult to set-up DNS and the DNS-based challenges for Let's encrypt. |
|
Encrypting to an unknown and unverifiable key doesn't accomplish anything. Cleartext is fine if you have no risk of MITM, and if there's a MITM, they can just sit in between both encrypted machines with unauthenticated TLS. (Perhaps there's a benefit in the case of a merely passive MITM, but that's not typically a real scenario.) DNS and Let's Encrypt doesn't provide a solution for local networks AFAIK? |
You're not doing that if you're using ciphers with PFC. |
|
@luke-jr can you explain what is your use case?
So now if I want to connect to a device, both parts needs to present a valid certificate signed by the CA i.e. mTLS If you want to allow anyone to connect to your device in your local network then you need to provision
Although I haven't tested the latter but I guess it should be possible... PD: @dpgeorge I closed this by mistake while updating my master branch... |
|
@Carglglz Yes, creating your own CA works. My objection is only to suggestions that don't have a way to authenticate. My use case is to ensure someone cannot replace my devices with malicious ones or intercept the passwords for them, even if they physically MITM their network connections. |
@luke-jr passwords are sent after the TLS handshake is done, but if the devices are physically accessible, then you probably should use flash encryption and secure boot (although AFAIK this is platform dependent and not part of MicroPython), otherwise someone could replace/copy the contents which will be as dangerous as no encryption at all... |
The two first commits are MicroPython related, the other three are a Python Websocket class that works both with WebREPL (ws) and WebREPL over SSL (wss).
Related to webrepl.py:
'SSL_certificate{}.der'.format(hexlify(unique_id()).decode())I use the unique id to be able to have unique key-cert for each device.
Related to websocket_helper.py:
Related to ws_protocol.py, ws_client_handshake.py and websocket_client.py:
These are not part of the PR really, just for testing (although they could be included in this or another PR).
ws_protocol.pyandws_client_handshake.pyare adaptations from https://github.com/danni/uwebsockets/tree/esp8266/uwebsockets, and there is no external dependency, just "pure" Python.websocket_client.pyis a simple class to sends commands and receive output. ( for both ws and wss). This could be adapted or included in some way inpyboard.py