Skip to content

gh-116263: Do not rollover empty files in RotatingFileHandler #122788

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Aug 7, 2024

rh.close()

@unittest.skipIf(support.is_wasi, "WASI does not have /dev/null.")
def test_should_not_rollover_non_file(self):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not know what to do with this test. It is now passed not because the file is not a regular file, but because tell() returns 0. It always returns 0 for /dev/null on Linux.

So this test no longer tests what it was intended to test, and I do not know how to fix this. Is the check for regular file is even needed in shouldRollover()?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd need to check the behaviour on macOS and Windows too, though it seems like the zero should be returned on all platforms. This check can remain to test that we never rollover special files, no matter the internal reason. If we can guarantee that tell() returns 0 in all cases for all special files (not just /dev/null), perhaps the check for a regular file could be removed.

@@ -196,9 +196,12 @@ def shouldRollover(self, record):
if self.stream is None: # delay was set...
self.stream = self._open()
if self.maxBytes > 0: # are we rolling over?
pos = self.stream.tell()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section will clash with the other PR for gh-116267, perhaps wait until that is finalised?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is simpler, and it adds some tests for delay=True, so I think that it would be better to merge it first.

msg = "%s\n" % self.format(record)
self.stream.seek(0, 2) #due to non-posix-compliant Windows feature
if self.stream.tell() + len(msg) >= self.maxBytes:
if pos + len(msg) >= self.maxBytes:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The preceding line hardcodes the \n terminator - in the PR for gh-116267, I've changed it to avoid that. same comment as above re. clash with the PR for gh-116267.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it will be not difficult to resolve conflict.

rh.close()

@unittest.skipIf(support.is_wasi, "WASI does not have /dev/null.")
def test_should_not_rollover_non_file(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd need to check the behaviour on macOS and Windows too, though it seems like the zero should be returned on all platforms. This check can remain to test that we never rollover special files, no matter the internal reason. If we can guarantee that tell() returns 0 in all cases for all special files (not just /dev/null), perhaps the check for a regular file could be removed.

@@ -196,9 +196,12 @@ def shouldRollover(self, record):
if self.stream is None: # delay was set...
self.stream = self._open()
if self.maxBytes > 0: # are we rolling over?
pos = self.stream.tell()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is simpler, and it adds some tests for delay=True, so I think that it would be better to merge it first.

msg = "%s\n" % self.format(record)
self.stream.seek(0, 2) #due to non-posix-compliant Windows feature
if self.stream.tell() + len(msg) >= self.maxBytes:
if pos + len(msg) >= self.maxBytes:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it will be not difficult to resolve conflict.

@serhiy-storchaka serhiy-storchaka merged commit 6094c6f into python:main Aug 8, 2024
36 checks passed
@miss-islington-app
Copy link

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

@serhiy-storchaka serhiy-storchaka deleted the RotatingFileHandler-empty-rollover branch August 8, 2024 06:48
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Aug 8, 2024
…ythonGH-122788)

(cherry picked from commit 6094c6f)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-app
Copy link

bedevere-app bot commented Aug 8, 2024

GH-122814 is a backport of this pull request to the 3.13 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Aug 8, 2024
…ythonGH-122788)

(cherry picked from commit 6094c6f)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Aug 8, 2024
@bedevere-app
Copy link

bedevere-app bot commented Aug 8, 2024

GH-122815 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 only security fixes label Aug 8, 2024
Yhg1s pushed a commit that referenced this pull request Sep 2, 2024
…H-122788) (#122814)

gh-116263: Do not rollover empty files in RotatingFileHandler (GH-122788)
(cherry picked from commit 6094c6f)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
ambv pushed a commit that referenced this pull request Sep 6, 2024
…H-122788) (#122815)

(cherry picked from commit 6094c6f)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants