Skip to content

[STM32-F2] - mqtt connect fails on latest builds. #5071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pacmac opened this issue Sep 5, 2019 · 6 comments
Closed

[STM32-F2] - mqtt connect fails on latest builds. #5071

pacmac opened this issue Sep 5, 2019 · 6 comments

Comments

@pacmac
Copy link

pacmac commented Sep 5, 2019

When using build MicroPython v1.9.3-1869-ga9b1d3ca3, the mqtt transaction succeeds:

>>> import mqtest
dbug-msg: 0x9 b'04:4d:51:54:54:04:02:00:00'
dbug-resp: b' \x02\x00\x00'
connect: 0
publish: None
disconnect: None

On latest release: MicroPython v1.11-274-g06661890d - it fails:

>>> import mqtest
dbug-msg: 0x9 b'04:4d:51:54:54:04:02:00:00'
dbug-resp: b''
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mqtest.py", line 37, in <module>
  File "mqtest.py", line 33, in go
  File "umqtt_robust.py", line 101, in connect
IndexError: bytes index out of range

This is my test code:

from umqtt_robust import MQTTClient

id      = b'484a300173f0'
host    = b'mqtt.thingspeak.com'
port    = 1883
retain  = False
topic   = "channels/MY-CHANNEL-ID/publish/MY-ACCESS-KEY"
data    = 'field6=100&field5=4019&field4=64.20001&field3=29.5&field2=73.1&field1=29.5&field8={"last_stamp": 1495097691048, "update_secs": 300, "name": "Room", "devid": "484a300173f0"}&status=MQTTPUBLISH&field7=-9318.205'

client = MQTTClient(id,host,port=port)

def go():
  print('connect:',client.connect())
  print('publish:',client.publish(topic,data,retain=retain))
  print('disconnect:',client.disconnect());

go()

This is the code in mqtt_simple / robust that is debugged & failing:

    def connect(self, clean_session=True):
        self.sock = socket.socket()
        addr = socket.getaddrinfo(self.server, self.port)[0][-1]
        self.sock.connect(addr)
        if self.ssl:
            import ussl
            self.sock = ussl.wrap_socket(self.sock, **self.ssl_params)
        premsg = bytearray(b"\x10\0\0\0\0\0")
        msg = bytearray(b"\x04MQTT\x04\x02\0\0")

        sz = 10 + 2 + len(self.client_id)
        msg[6] = clean_session << 1
        if self.user is not None:
            sz += 2 + len(self.user) + 2 + len(self.pswd)
            msg[6] |= 0xC0
        if self.keepalive:
            assert self.keepalive < 65536
            msg[7] |= self.keepalive >> 8
            msg[8] |= self.keepalive & 0x00FF
        if self.lw_topic:
            sz += 2 + len(self.lw_topic) + 2 + len(self.lw_msg)
            msg[6] |= 0x4 | (self.lw_qos & 0x1) << 3 | (self.lw_qos & 0x2) << 3
            msg[6] |= self.lw_retain << 5

        i = 1
        while sz > 0x7f:
            premsg[i] = (sz & 0x7f) | 0x80
            sz >>= 7
            i += 1
        premsg[i] = sz

        self.sock.write(premsg, i + 2)
        self.sock.write(msg)
        print('dbug-msg:',hex(len(msg)), hexlify(msg, ":"))
        self._send_str(self.client_id)
        if self.lw_topic:
            self._send_str(self.lw_topic)
            self._send_str(self.lw_msg)
        if self.user is not None:
            self._send_str(self.user)
            self._send_str(self.pswd)
        resp = self.sock.read(4)
        print('dbug-resp:',resp)
        assert resp[0] == 0x20 and resp[1] == 0x02
        if resp[3] != 0:
            raise MQTTException(resp[3])
        return resp[2] & 1

This is also reported here: https://forum.micropython.org/viewtopic.php?f=2&t=6890

@dpgeorge
Copy link
Member

Thanks for the report. I can reproduce something similar with socket read not returning data. It looks like it might have something to do with mDNS and the latest version of lwIP. As a temporary workaround you can try disabling mDNS via:

--- a/ports/stm32/lwip_inc/lwipopts.h
+++ b/ports/stm32/lwip_inc/lwipopts.h
@@ -28,8 +28,8 @@
 #define LWIP_DHCP_CHECK_LINK_UP         1
 #define DHCP_DOES_ARP_CHECK             0 // to speed DHCP up
 #define LWIP_DNS                        1
-#define LWIP_DNS_SUPPORT_MDNS_QUERIES   1
-#define LWIP_MDNS_RESPONDER             1
+#define LWIP_DNS_SUPPORT_MDNS_QUERIES   0
+#define LWIP_MDNS_RESPONDER             0
 #define LWIP_IGMP                       1
 
 #define LWIP_NUM_NETIF_CLIENT_DATA      1 // mDNS responder requires 1

@dpgeorge
Copy link
Member

It turns out the lwIP TCP timer was not being scheduled because there was not enough buffers left to allocate its slot in the system timer list, and so TCP data was never being sent (UDP was fine). This was because mDNS took an extra slot in the timer list. This bug should be fixed by b0e17bb

@WadeJA
Copy link

WadeJA commented Sep 13, 2019

First, my apologies in following up; new to esp32 but been around (lots) in other arenas. Was using esp32-20190802-v1.11-187-g00e7fe8ab.bin in my initial moves to esp32 fr rpis (esp is more straightforward for some analog work). Wasn't getting simple mqtt to connect (socket failures). Found this post (just now - 9/13 ~6AM EST). Using HUZZAH32 – ESP32 Feather Board (hope acceptable to state). Expecting perhaps new bin will fix. But (newbie I guess): What specific bin to use? I think I 'could' go back to esp32-20180511-v1.9.4 which herein states does work. Or move ahead to esp32-20190913-v1.11-312-g22099ab88 which I've d/l'd from http://www.micropython.org/download. But is fix mentioned in 312 build? Don't know how to determine. Thank you for your patience in replying.

@WadeJA
Copy link

WadeJA commented Sep 13, 2019

Follow-up (FWIW): I loaded the 1.11 312 build and the adaptation of pacmac's simple test into my code and it works just fine (using mosquitto's test site). So to all, thank you - 312 appears to fix this. FYI, I wrote several data collectors using mostly rpiz-Ws with GOLANG. The collector code runs on any of W10s, Macs, Ubuntu (17, 18), also in GOLANG. So my next step is to do more analog 'stuff' using the ESP32s (been mostly DS18b20s n I2C data gathering). Given the collector / local controller comm uses MQTT with formatted data reporting, I needed to extend that infrastructure to include several ESP32s. The DB uses MySQL; have ~60m rows in it. Again, thank you. Had been trying this on 1.11 187 for several weeks (on n off). Given my newbie 'status' I was exhausting everything I could think of before posting here. Then saw this post. Was seeing socket problems but just 'was sure' it was my problem somewhere.

@dpgeorge
Copy link
Member

@WadeJA note that the error reported here, and the fix, is for stm32, not esp32 boards.

@pacmac
Copy link
Author

pacmac commented Sep 16, 2019

Thanks Damien and sorry for the delay, yes confirmed the issue is now fixed.

@pacmac pacmac closed this as completed Sep 16, 2019
tannewt added a commit to tannewt/circuitpython that referenced this issue Jul 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants