-
Notifications
You must be signed in to change notification settings - Fork 1k
python-stdlib/logging: add formatter, handlers #482
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
add formatting support and embedded device oriented handlers (circular log file, socket and syslog)
Thanks for this, I agree the existing I certainly support pulling in updates from pycopy when they make sense. As that fork has a common history with this one I prefer to cherry-pick / rebase commits from there to preserve the author attribution - when good work is done I try to acknowledge it directly. For example I've recently pulled in pycopy changed to unittest in #487 which I then build on top of in #488 If you'd like I'd be happy to help pull in pycopy commits to logging in a separete PR that you could rebase on top of? Separately though, I like your rationalle for the two handlers! However being different to any cpython logging handlers I think they'll need to be documented in some way (maybe just adding a readme to the folder) with a note that these are non-compliant with cpython and some basic details on their usage (separate to the example provided). Similarly, these handlers probably wont be suitable for use on all platforms / ports - some don't have socket for instance. As such I think they should go in separate python files that users can choose to install / deploy to device or not as needed. This would allow easier exclusion of the code size/functionality in applciations where it won't be needed. |
With the In my mind it feels like the circular overwriting of a single file would be harder to read as you have to scan through the file to find the oldest messages then read on from there? I might be missing something here though :-) |
I've just had a chat with others and there's a guidance that everything in Anything that's a micropython specific change should go in a separate module in |
@@ -7,6 +10,14 @@ | |||
DEBUG = 10 | |||
NOTSET = 0 | |||
|
|||
SYSLOG_FORMAT = "<%(pri)s>1 %(asctime)s %(ip)s - - - - %(message)s" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should these go in the respective classes using it instead of globally?
@@ -15,7 +26,13 @@ | |||
DEBUG: "DEBUG", | |||
} | |||
|
|||
_stream = sys.stderr | |||
_syslog_severity = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same question
@@ -108,21 +272,34 @@ def getLogger(name="root"): | |||
_loggers[name] = l | |||
return l | |||
|
|||
def critical(msg, *args): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can log
and exception
also be added while at it?
Below is an implementation of the CPython RotatingFileHandler for consideration. class RotatingFileHandler(Handler):
def __init__(self, filename, maxsize=128_000, backupCount=5):
if maxsize < 256:
raise ValueError("maxsize must be at least 256 B")
self.filename = filename
self.maxsize = maxsize
self.backupCount = backupCount
self.formatter = Formatter()
try:
# checks if file exists and prevents overwriting it
self.pos = os.stat(self.filename)[6]
self.file = open(filename, "r+")
except OSError:
self.pos = 0
self.file = open(filename, "w+")
self.file.seek(self.pos)
def shouldRollover(self, record):
"""
Determine if rollover should occur.
Basically, see if the supplied record would cause the file to exceed
the size limit we have.
"""
message = self.format(record)
if len(message) + self.pos > self.maxsize:
return True
else:
return False
def doRollover(self):
"""
Do a rollover.
"""
# Close current file first
self.file.close()
if self.backupCount > 0:
for i in range(self.backupCount - 1, 0, -1):
sfn = "%s.%d" % (self.filename, i)
dfn = "%s.%d" % (self.filename, i + 1)
try:
os.rename(sfn, dfn)
except:
print(f"Cant move file {sfn} to {dfn}")
dfn = self.filename + ".1"
try:
os.rename(self.filename, dfn)
except:
print(f"Couldnt move file {self.filename} to {dfn}")
print("Rollover complete")
self.pos = 0
self.file = open(self.filename, "w+")
self.file.seek(self.pos) |
Closing due to inactivity. Significant improvements were made to |
The current version 0.3 of logging.py is very lightweight. Too lightweight, in fact, as all it does out of the box is print into sys.stderr.
I'm suggesting to add features focused on embedded devices where MicroPython is mostly used. What I mean by that are specifically
CircularFileHandler
andSysLogHandler
.CircularFileHandler
is a handler that writes logs into a file and starts at the beginning of the file once it reaches a defined limit. This prevents the exhaustion of free space that is a risk of naive file handlers. This handler does not overwrite existing log file which preserves logs across device reboots.SysLogHandler
extendsSocketHandler
and mostly implements RFC 5424 - The Syslog Protocol. I say "mostly" because it does not implement TLS transport due to memory constraints. It does implement the protocol message format, and supports both TCP and UDP transport.To implement
SyslogHandler
custom log message formatting is required. I borrowedFormatter
from pycopy-lib/logging/logging/init.py with minimal changes. I added support fordefaults
argument which allows to pass system IP or hostname into the formatter used for SysLogHandler without adding undesired dependency oflogging
onnetwork
. Both %-style and {-style string formats are supported.Finally, I extended
basicConfig
to supportfilename
(usesCircularFileHandler
),format
anddatefmt
arguments which simplifies the usage of most of the functionality of the module.