Hi Patrick,

thanks for the PR.

I did not implement logging via logging because I did not see the point. I will explain this further but I may be missing something so please correct me if my assumptions and understanding are wrong.

This is the code of logging in micropython-lib. It is a simplified version of logging that works as a drop-in replacement for full-featured logging. It does not support anything else but sending messages to stdout - it cannot write logs into persistent storage (files, NVS) and it cannot send logs to a logging server.

Therefore, for a new project for MicroPython I don't see added value in using logging over print at the moment. It just dumps messages to stderr and according to MicroPython documentation "Overriding sys.stdin, sys.stdout and sys.stderr not possible".

Are you planning to use uOTA with Python rather than MicroPython? Do you use a custom logging handler that does something more than writing to stderr? Please help me understand your use-case with storing/collecting logs in remotely deployed devices.

Hi Martin,

Thanks for your message.

For my use-case I am using uota with micropython (ESP32S2). I am deploying IOT devices remotely that are connected through the cellular/satellite network. Most of them are at sea or in remote regional areas.

I use a custom log handler to write logs to flash and ftp to upload the logs to a server. This seems to be the only way I can monitor remote devices as sys.stdout/err is not accessible remotely (without a permanent webrepl connection).

The idea of adding logging to uota would be to allow checking/validation that an update installed correctly and to help debug any errors that occur. At the moment the output of print() is lost on remote devices.

What are your thoughts? Am I approaching this the wrong way? I could monkey-patch the print() method to redirect to logging but I'm not sure this is very pythonic.

Hi Patrick,

now I understand, the custom logging handler is what makes logging support worth having. Would you mind sharing the handler it with the rest of us, for our benefit and inspiration? You could add a link to your repository into uOTA docs so that everyone can add the handler into their projects. Also I'd like to write a handler that forwards logs into a syslog server over the network and an existing handler compatible with MicroPython would be a good place to start.

Your approach with logging is of course a good one, better than abusing print(). I was just missing a pice of the puzzle at first. Please let me know if we can link to this piece in the documentation to make it obvious to everyone that they need to add a handler to make this do something useful.

Alright, so I've written a basic custom handler to send logs to a central server over the network. I am struggling with making it act as a FileIO object in order to make it universal for different versions of logging that are out there so I've asked a question in MicroPython forums. Hopefully I'll resolve the issue in one way or another, update the docs and merge the PR this weekend.

What approach are you using to send the logs to the server? I sometimes use a simple UDP connection to broadcast over the local LAN.

import socket

class BroadcastHandler(LoggingHandler):

    def __init__(self, address, port=3000):
        """Create an instance.

        :param filename: the name of the file to which to write messages

        """
        self._address = address
        self._port = port
        try:
            self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        except:
            pass

    def emit(self, level, msg):
        """Generate the message and write it to the UART.

        :param level: The level at which to log
        :param msg: The core message

        """
        try:
            self._socket.sendto(self.format(level, msg), (self._address, self._port))
        except:
            pass

and then on the server

import socket
import os

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

s.bind(('192.168.17.255', 3000))

f = open("log.log", "a")

while True:
    msg, addr = s.recvfrom(3000)
    print(msg)
    f.write(msg.decode("utf-8"))
    f.write("\n")

f.close()

I can definitely share my file handler and put some background info in the docs if you like, it's pretty simple.

import logging
logging.basicConfig(level=logging.DEBUG)

log = logging.getLogger(__name__)

class FileHandler(logging.Handler):

    def __init__(self, filename):
        """Create an instance.

        :param filename: the name of the file to which to write messages

        """
        self._filename = filename
        self.file = open(filename, 'a')

    def emit(self, msg):
        """Generate the message and write it to file.

        :param msg: The core message

        """
        
        current_date = time.localtime()
        (_year, _month, _mday, _hour, _minute, _second, _wday, _yday) = current_date
        uptime_seconds = '%03d' % round(time.ticks_ms()/1000)
        
        date_string = '%04d-%02d-%02d %02d:%02d:%02d' % (_year, _month, _mday, _hour, _minute, _second)
        
        try:
            self.file.write('{0} | {1} | {2}\t | {3}\t | {4}\n'.format(date_string, uptime_seconds, msg.levelname, msg.name, msg.message))
        except Exception as e:
            sys.print_exception(e)
        self.file.flush()

            
fh = FileHandler('/log.log')
log.addHandler(fh)

What approach are you using to send the logs to the server?

Same as you, i.e. writing into a UDP socket, although I use unicast instead of broadcast for the transmission. For the server part I use rsyslog configured to receive messages on the standard port 514 as I like to use reliable industry standard components where they exist.

Meanwhile I learned that subclassing basic types is not supported in MicroPython so I'll get to cleaning up my handler and updating the docs. Thank you for sharing your code.

Interesting, I need to look at rsyslog.

Where is the best place for handlers to live, is it best to contribute a basic file/network handler to micropython-lib or should it be up to the individual to create their own handlers? What do you think?

Rsyslog is almost a trivial piece of software. Install it from a repository, uncomment the following two lines in /etc/rsyslog.conf, restart the process and you're all set to receive syslog on UDP/514.

module(load="imudp")
input(type="imudp" port="514")

At fist I was thinking of creating an extra repository for the handlers but it makes sense to offer them to micropython-lib. They handlers are small enough not to be much of a burden to the lightweight logging library itself. I'm in the middle of turning the FileHandler into a circular buffer to make it more robust and prevent filling up the filesystem. I'll finish it over the weekend and shoot a PR to micropython-lib. It will be cool if they accept it, if not I'll create a new repository for the handlers and link it from the docs, either way I'll merge this PR after the logging dependency is sorted out.

Thanks Martin sounds good!

If you need any help implementing future features let me know.

I've submitted a PR to micropython/micropython-lib with a more useful version of logging.py. You can check it out, especially the SyslogHandler and CircularFileHandler. With that I'm merging this PR.

Thanks for merging. The mp PR looks great, very comprehensive with lots of examples, hopefully they will merge it!