Skip to content

Close DB connection when Database is garbage collected #132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

mbhutton
Copy link
Contributor

@mbhutton mbhutton commented Feb 22, 2025

Register a weakref.finalize callback to close the underlying DB connection when the Database instance is garbage collected.

This address a ResourceWarning which was introduced in Python 3.13 (python/cpython#105539).

@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 98.91%. Comparing base (ff54025) to head (210a746).
Report is 40 commits behind head on main.

Files with missing lines Patch % Lines
things/database.py 75.00% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #132      +/-   ##
==========================================
- Coverage   99.34%   98.91%   -0.43%     
==========================================
  Files           3        3              
  Lines         455      462       +7     
==========================================
+ Hits          452      457       +5     
- Misses          3        5       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mbhutton mbhutton marked this pull request as draft February 22, 2025 23:09
@mbhutton mbhutton force-pushed the fix/close-db-connection-on-weakref-finalize branch 2 times, most recently from f149a70 to 1844154 Compare February 22, 2025 23:26
@mbhutton mbhutton marked this pull request as ready for review February 22, 2025 23:28
@AlexanderWillner
Copy link
Contributor

So this registers a cleanup function (sqlite3.Connection.close) when the DB object is garbage collected. The tests are fine, the warning is gone....lgtm

@mikez
Copy link
Contributor

mikez commented Feb 23, 2025

@mbhutton Thank you!

Two minor things, otherwise LGTM.

  1. Is the assignment to self._finalizer strictly necessary? According to the docs

Unlike an ordinary weak reference, a finalizer will always survive until the reference object is collected, greatly simplifying lifecycle management.

  1. Do you think it might be helpful to add a brief comment above weakref.finalize to explain the context here for future people?

@mbhutton mbhutton force-pushed the fix/close-db-connection-on-weakref-finalize branch from 1844154 to d3d796e Compare February 24, 2025 19:02
@mbhutton
Copy link
Contributor Author

Hi @mikez,

  1. Is the assignment to self._finalizer strictly necessary? According to the docs

Unlike an ordinary weak reference, a finalizer will always survive until the reference object is collected, greatly simplifying lifecycle management.

Yes I think so. The only options I can see are either:

  • Ignore the warning
  • Use a separate connection for each transaction (slower)
  • Provide a manual close() method for use by callers, pushing responsibility to close to the API client
  • Create a context manager for the Database to automatically close when going out of scope (requiring changes to the caller)
  • This approach: clean up on GC. Chosen as the least invasive option which keeps the performance gains.

If there were things.py clients which used large numbers of Database instances then one of the other options would be more appropriate, to force earlier cleanup, but I know this isn't an issue in practice based on the fact that this library has historically used a single connection.

The reference to self is required for the finalizer to work, as well as the connection reference to support the Connection.close() call.

  1. Do you think it might be helpful to add a brief comment above weakref.finalize to explain the context here for future people?

I've added the following comment above the finalizer call:

# Close the underlying SQLite connection when this Database object is garbage collected

Does that sound okay?

@mikez
Copy link
Contributor

mikez commented Feb 25, 2025

@mbhutton Quick clarification on point 1. What I meant with referring to that citation in the docs was doing only the function call

weakref.finalize(self, sqlite3.Connection.close, self.connection)

as opposed to the assignment

self._finalizer = weakref.finalize(self, sqlite3.Connection.close, self.connection)

The strategy itself I like.

  1. Sounds good to me.

Register a weakref.finalize callback to close the underlying DB
connection when the Database instance is garbage collected.

This address a ResourceWarning which was introduced in Python 3.13
(python/cpython#105539).
@mbhutton mbhutton force-pushed the fix/close-db-connection-on-weakref-finalize branch from d3d796e to dee457e Compare February 25, 2025 18:37
@mbhutton
Copy link
Contributor Author

@mikez Thanks for clarifying, self._finalizer assignment now removed, much nicer without it!

I believe that's everything resolved?

@mikez
Copy link
Contributor

mikez commented Feb 25, 2025

@mbhutton Looks great to me! Thanks for your work here. 💛

@mikez mikez merged commit e128744 into thingsapi:main Feb 25, 2025
7 checks passed
@mbhutton mbhutton deleted the fix/close-db-connection-on-weakref-finalize branch February 26, 2025 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants