Skip to content

gh-83638: Add sqlite3.Connection.autocommit for PEP 249 compliant behaviour #93823

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 82 commits into from
Nov 12, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
10d3a65
Add sqlite3.Connection.autocommit for PEP 249 compliant behaviour
erlend-aasland Jun 11, 2022
6ca9045
Add basic tests
erlend-aasland Jun 14, 2022
5772676
Add documentation
erlend-aasland Jun 14, 2022
06609f3
Add NEWS
erlend-aasland Jun 14, 2022
8b6b866
Add What's New
erlend-aasland Jun 14, 2022
a604a57
Document DEPRECATED_TRANSACTION_CONTROL
erlend-aasland Jun 14, 2022
cbc1041
Fix setattr behaviour
erlend-aasland Jun 14, 2022
41b898c
Suggest new behaviour starting with Python 3.14
erlend-aasland Jun 15, 2022
b11ebd2
Simplify implicit rollback test
erlend-aasland Jun 15, 2022
d3009d4
Address Alex' review
erlend-aasland Jun 15, 2022
fb98615
Reorder if condition to add emphasis to deprecated behaviour
erlend-aasland Jun 15, 2022
5e4626e
Add context manager tests
erlend-aasland Jun 15, 2022
a8e3b9e
Improve 5e4626ecf47a679ba5fa8846472af16fb325cac5
erlend-aasland Jun 15, 2022
d46e9be
Add executescript tests
erlend-aasland Jun 15, 2022
c2e2f1a
Improve docs
erlend-aasland Jun 15, 2022
3bf3ee1
Add missing versionadded
erlend-aasland Jun 16, 2022
757f70b
Try to further improve the docs
erlend-aasland Jun 16, 2022
a5ef40c
Nano-optimisation
erlend-aasland Jun 16, 2022
ae37fbb
Explicit downcast
erlend-aasland Jun 17, 2022
341e0dd
Clarify execute*, commit, rollback, and close
erlend-aasland Jun 17, 2022
77651c0
Incorporate gh-93890 changes
erlend-aasland Jun 17, 2022
0ac782c
Improve
erlend-aasland Jun 17, 2022
7743f34
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jun 19, 2022
6628e30
commit/rollback wordings
erlend-aasland Jun 19, 2022
060bd3a
Fix merge
erlend-aasland Jun 19, 2022
c96adbf
Doc: reword context manager section
erlend-aasland Jun 20, 2022
a7c7c4c
Autocommit is keyword only
erlend-aasland Jun 24, 2022
f1c5843
Isolation level is not yet deprecated (soon it will be though)
erlend-aasland Jun 24, 2022
491426e
Merge remote-tracking branch 'upstream/main' into sqlite-autocommit
erlend-aasland Jun 24, 2022
05879a9
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jun 27, 2022
97ad734
Use macro as C default
erlend-aasland Jun 27, 2022
bf3a857
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 8, 2022
60f7224
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 18, 2022
7f645ee
Pass keywords to factory function
erlend-aasland Jul 20, 2022
88fb2b9
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 20, 2022
4deed17
Adjust commit/rollback and remove redundant info
erlend-aasland Jul 20, 2022
f3a9e37
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 23, 2022
2981a6b
Fix docstring
erlend-aasland Jul 23, 2022
a5b7864
Revert unneeded doc changes
erlend-aasland Jul 23, 2022
fd6659a
Fix autocommit enum
erlend-aasland Jul 23, 2022
52e3736
Sync with main bco. recent doc updates
erlend-aasland Jul 26, 2022
6e92073
Merge branch 'main' into sqlite-autocommit
erlend-aasland Jul 30, 2022
59fbd29
Merge branch 'main' into sqlite-autocommit
erlend-aasland Aug 3, 2022
894fcce
Add autocommit to connect signature
erlend-aasland Aug 3, 2022
fff4307
Merge branch 'main' into sqlite-autocommit
erlend-aasland Aug 8, 2022
4ac0fa0
Address Alex's second review
erlend-aasland Aug 8, 2022
64c6482
Adjust headings
erlend-aasland Aug 8, 2022
e4947d0
Use versionadded iso. versionchanged
erlend-aasland Aug 8, 2022
5981536
Remove some no-ops
erlend-aasland Aug 8, 2022
527e6a3
Three adjustments:
erlend-aasland Aug 8, 2022
697b65a
Address Alex's last round of review
erlend-aasland Aug 15, 2022
4299874
Sync with main bco. recent Argument Clinic changes
erlend-aasland Aug 15, 2022
76122be
Regen all
erlend-aasland Aug 15, 2022
6902738
Explicit logic in set_autocommit
erlend-aasland Aug 15, 2022
c37ebb5
Tweak naming: connection_txn_stmt => connection_exec_stmt
erlend-aasland Aug 15, 2022
2e9083c
Merge branch 'main' into sqlite-autocommit
erlend-aasland Aug 18, 2022
b944a90
Address review: improve wording for connection close()
erlend-aasland Aug 18, 2022
bce921f
Sync with main
erlend-aasland Oct 28, 2022
78e04f0
Fix bad merge
erlend-aasland Oct 28, 2022
bd91e34
Remove implicit commit for cx.autocommit = False
erlend-aasland Oct 28, 2022
9d66daa
Doc: make the autocommit reference more to the point; expand in the e…
erlend-aasland Oct 28, 2022
d4c0583
Address most of CAM's review
erlend-aasland Oct 29, 2022
a3aa9aa
Better naming for the constant; don't mention deprecations (yet)
erlend-aasland Oct 29, 2022
c9b05ff
Fix default role
erlend-aasland Oct 30, 2022
d5e33f1
Make it explicit that isolation_level is ignored if autocommit is Tru…
erlend-aasland Oct 30, 2022
0cf022a
Reflow
erlend-aasland Oct 30, 2022
5009372
Address review: COMPAT_TRANSACTIONAL_CONTROL => LEGACY_TRANSACTION_CO…
erlend-aasland Oct 31, 2022
2c9b117
Address more of CAM's review
erlend-aasland Oct 31, 2022
de3bd4e
Try to address the rest of CAM's review
erlend-aasland Oct 31, 2022
fca9c56
Sync with main
erlend-aasland Nov 6, 2022
7de6bf3
Address more of CAM's remarks
erlend-aasland Nov 6, 2022
e0fa134
Prevent segfault if implicitly rolling back during interpreter shutdown
erlend-aasland Nov 6, 2022
e0bedf6
Update Doc/library/sqlite3.rst
Nov 7, 2022
e33946a
Address last batch of CAM's comments
erlend-aasland Nov 7, 2022
cd15505
Last bit
erlend-aasland Nov 8, 2022
4feefa1
Update Doc/library/sqlite3.rst
Nov 11, 2022
720ac11
Sync with main
erlend-aasland Nov 11, 2022
f7c2ae4
Update Doc/library/sqlite3.rst
Nov 11, 2022
88e1420
Sync with main
erlend-aasland Nov 12, 2022
7b87830
Address review: remove unneeded assert in test_autocommit_setget_invalid
erlend-aasland Nov 12, 2022
8322683
Address review: add test for rollback() if autocommit is True
erlend-aasland Nov 12, 2022
aca1e22
Address review: fixup context manager docs
erlend-aasland Nov 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
171 changes: 154 additions & 17 deletions Doc/library/sqlite3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,8 @@ Module functions
.. function:: connect(database, timeout=5.0, detect_types=0, \
isolation_level="DEFERRED", check_same_thread=True, \
factory=sqlite3.Connection, cached_statements=128, \
uri=False)
uri=False, \*, \
autocommit=sqlite3.LEGACY_TRANSACTION_CONTROL)

Open a connection to an SQLite database.

Expand Down Expand Up @@ -290,11 +291,12 @@ Module functions
By default (``0``), type detection is disabled.

:param isolation_level:
The :attr:`~Connection.isolation_level` of the connection,
controlling whether and how transactions are implicitly opened.
See :attr:`Connection.isolation_level` and
:ref:`sqlite3-transaction-control-isolation-level` for more information.
Can be ``"DEFERRED"`` (default), ``"EXCLUSIVE"`` or ``"IMMEDIATE"``;
or ``None`` to disable opening transactions implicitly.
See :ref:`sqlite3-controlling-transactions` for more.
Has no effect unless :attr:`Connection.autocommit` is set to
:data:`~sqlite3.LEGACY_TRANSACTION_CONTROL` (the default).
:type isolation_level: str | None

:param bool check_same_thread:
Expand All @@ -321,6 +323,14 @@ Module functions
The query string allows passing parameters to SQLite,
enabling various :ref:`sqlite3-uri-tricks`.

:param autocommit:
See :attr:`Connection.autocommit` and
:ref:`sqlite3-transaction-control-autocommit` for more information.
*autocommit* currently defaults to
:data:`~sqlite3.LEGACY_TRANSACTION_CONTROL`.
The default will change to ``False`` in a future Python release.
:type autocommit: bool

:rtype: Connection

.. audit-event:: sqlite3.connect database sqlite3.connect
Expand All @@ -335,6 +345,9 @@ Module functions
.. versionadded:: 3.10
The ``sqlite3.connect/handle`` auditing event.

.. versionadded:: 3.12
The *autocommit* parameter.

.. function:: complete_statement(statement)

Return ``True`` if the string *statement* appears to contain
Expand Down Expand Up @@ -418,6 +431,12 @@ Module functions
Module constants
^^^^^^^^^^^^^^^^

.. data:: LEGACY_TRANSACTION_CONTROL

Set :attr:`~Connection.autocommit` to this constant to select
old style (pre-Python 3.12) transaction control behaviour.
See :ref:`sqlite3-transaction-control-isolation-level` for more information.

.. data:: PARSE_COLNAMES

Pass this flag value to the *detect_types* parameter of
Expand Down Expand Up @@ -616,18 +635,27 @@ Connection objects
.. method:: commit()

Commit any pending transaction to the database.
If there is no open transaction, this method is a no-op.
If :attr:`autocommit` is ``True``, or there is no open transaction,
this method does nothing.
If :attr:`!autocommit` is ``False``, a new transaction is implicitly
opened if a pending transaction was committed by this method.

.. method:: rollback()

Roll back to the start of any pending transaction.
If there is no open transaction, this method is a no-op.
If :attr:`autocommit` is ``True``, or there is no open transaction,
this method does nothing.
If :attr:`!autocommit` is ``False``, a new transaction is implicitly
opened if a pending transaction was rolled back by this method.

.. method:: close()

Close the database connection.
Any pending transaction is not committed implicitly;
make sure to :meth:`commit` before closing
If :attr:`autocommit` is ``False``,
any pending transaction is implicitly rolled back.
If :attr:`!autocommit` is ``True`` or :data:`LEGACY_TRANSACTION_CONTROL`,
no implicit transaction control is executed.
Make sure to :meth:`commit` before closing
to avoid losing pending changes.

.. method:: execute(sql, parameters=(), /)
Expand Down Expand Up @@ -1224,6 +1252,38 @@ Connection objects

.. versionadded:: 3.11

.. attribute:: autocommit

This attribute controls :pep:`249`-compliant transaction behaviour.
:attr:`!autocommit` has three allowed values:

* ``False``: Select :pep:`249`-compliant transaction behaviour,
implying that :mod:`!sqlite3` ensures a transaction is always open.
Use :meth:`commit` and :meth:`rollback` to close transactions.

This is the recommended value of :attr:`!autocommit`.

* ``True``: Use SQLite's `autocommit mode`_.
:meth:`commit` and :meth:`rollback` have no effect in this mode.

* :data:`LEGACY_TRANSACTION_CONTROL`:
Pre-Python 3.12 (non-:pep:`249`-compliant) transaction control.
See :attr:`isolation_level` for more details.

This is currently the default value of :attr:`!autocommit`.

Changing :attr:`!autocommit` to ``False`` will open a new transaction,
and changing it to ``True`` will commit any pending transaction.

See :ref:`sqlite3-transaction-control-autocommit` for more details.

.. note::

The :attr:`isolation_level` attribute has no effect unless
:attr:`autocommit` is :data:`LEGACY_TRANSACTION_CONTROL`.

.. versionadded:: 3.12

.. attribute:: in_transaction

This read-only attribute corresponds to the low-level SQLite
Expand All @@ -1236,17 +1296,24 @@ Connection objects

.. attribute:: isolation_level

This attribute controls the :ref:`transaction handling
<sqlite3-controlling-transactions>` performed by :mod:`!sqlite3`.
Controls the :ref:`legacy transaction handling mode
<sqlite3-transaction-control-isolation-level>` of :mod:`!sqlite3`.
If set to ``None``, transactions are never implicitly opened.
If set to one of ``"DEFERRED"``, ``"IMMEDIATE"``, or ``"EXCLUSIVE"``,
corresponding to the underlying `SQLite transaction behaviour`_,
implicit :ref:`transaction management
<sqlite3-controlling-transactions>` is performed.
:ref:`implicit transaction management
<sqlite3-transaction-control-isolation-level>` is performed.

If not overridden by the *isolation_level* parameter of :func:`connect`,
the default is ``""``, which is an alias for ``"DEFERRED"``.

.. note::

Using :attr:`autocommit` to control transaction handling is
recommended over using :attr:`!isolation_level`.
:attr:`!isolation_level` has no effect unless :attr:`autocommit` is
set to :data:`LEGACY_TRANSACTION_CONTROL` (the default).

.. attribute:: row_factory

A callable that accepts two arguments,
Expand Down Expand Up @@ -1375,7 +1442,9 @@ Cursor objects
:meth:`executescript` if you want to execute multiple SQL statements with one
call.

If :attr:`~Connection.isolation_level` is not ``None``,
If :attr:`~Connection.autocommit` is
:data:`LEGACY_TRANSACTION_CONTROL`,
:attr:`~Connection.isolation_level` is not ``None``,
*sql* is an ``INSERT``, ``UPDATE``, ``DELETE``, or ``REPLACE`` statement,
and there is no open transaction,
a transaction is implicitly opened before executing *sql*.
Expand Down Expand Up @@ -1403,7 +1472,9 @@ Cursor objects
.. method:: executescript(sql_script, /)

Execute the SQL statements in *sql_script*.
If there is a pending transaction,
If the :attr:`~Connection.autocommit` is
:data:`LEGACY_TRANSACTION_CONTROL`
and there is a pending transaction,
an implicit ``COMMIT`` statement is executed first.
No other implicit transaction control is performed;
any transaction control must be added to *sql_script*.
Expand Down Expand Up @@ -2183,9 +2254,12 @@ the transaction is committed.
If this commit fails,
or if the body of the ``with`` statement raises an uncaught exception,
the transaction is rolled back.
If :attr:`~Connection.autocommit` is ``False``,
a new transaction is implicitly opened after committing or rolling back.

If there is no open transaction upon leaving the body of the ``with`` statement,
the context manager is a no-op.
or if :attr:`~Connection.autocommit` is ``True``,
the context manager does nothing.

.. note::

Expand Down Expand Up @@ -2270,13 +2344,72 @@ can be found in the `SQLite URI documentation`_.
Explanation
-----------

.. _sqlite3-transaction-control:
.. _sqlite3-controlling-transactions:

Transaction control
^^^^^^^^^^^^^^^^^^^

The :mod:`!sqlite3` module does not adhere to the transaction handling recommended
by :pep:`249`.
:mod:`!sqlite3` offers multiple methods of controlling whether,
when and how database transactions are opened and closed.
:ref:`sqlite3-transaction-control-autocommit` is recommended,
while :ref:`sqlite3-transaction-control-isolation-level`
retains the pre-Python 3.12 behaviour.

.. _sqlite3-transaction-control-autocommit:

Transaction control via the ``autocommit`` attribute
""""""""""""""""""""""""""""""""""""""""""""""""""""

The recommended way of controlling transaction behaviour is through
the :attr:`Connection.autocommit` attribute,
which should preferrably be set using the *autocommit* parameter
of :func:`connect`.

It is suggested to set *autocommit* to ``False``,
which implies :pep:`249`-compliant transaction control.
This means:

* :mod:`!sqlite3` ensures that a transaction is always open,
so :meth:`Connection.commit` and :meth:`Connection.rollback`
will implicitly open a new transaction immediately after closing
the pending one.
:mod:`!sqlite3` uses ``BEGIN DEFERRED`` statements when opening transactions.
* Transactions should be committed explicitly using :meth:`!commit`.
* Transactions should be rolled back explicitly using :meth:`!rollback`.
* An implicit rollback is performed if the database is
:meth:`~Connection.close`-ed with pending changes.

Set *autocommit* to ``True`` to enable SQLite's `autocommit mode`_.
In this mode, :meth:`Connection.commit` and :meth:`Connection.rollback`
have no effect.
Note that SQLite's autocommit mode is distinct from
the :pep:`249`-compliant :attr:`Connection.autocommit` attribute;
use :attr:`Connection.in_transaction` to query
the low-level SQLite autocommit mode.

Set *autocommit* to :data:`LEGACY_TRANSACTION_CONTROL`
to leave transaction control behaviour to the
:attr:`Connection.isolation_level` attribute.
See :ref:`sqlite3-transaction-control-isolation-level` for more information.


.. _sqlite3-transaction-control-isolation-level:

Transaction control via the ``isolation_level`` attribute
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""

.. note::

The recommended way of controlling transactions is via the
:attr:`~Connection.autocommit` attribute.
See :ref:`sqlite3-transaction-control-autocommit`.

If :attr:`Connection.autocommit` is set to
:data:`LEGACY_TRANSACTION_CONTROL` (the default),
transaction behaviour is controlled using
the :attr:`Connection.isolation_level` attribute.
Otherwise, :attr:`!isolation_level` has no effect.

If the connection attribute :attr:`~Connection.isolation_level`
is not ``None``,
Expand Down Expand Up @@ -2307,6 +2440,10 @@ regardless of the value of :attr:`~Connection.isolation_level`.
:mod:`!sqlite3` used to implicitly commit an open transaction before DDL
statements. This is no longer the case.

.. versionchanged:: 3.12
The recommended way of controlling transactions is now via the
:attr:`~Connection.autocommit` attribute.

.. _autocommit mode:
https://www.sqlite.org/lang_transaction.html#implicit_versus_explicit_transactions

Expand Down
7 changes: 7 additions & 0 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,13 @@ sqlite3
* Add a :ref:`command-line interface <sqlite3-cli>`.
(Contributed by Erlend E. Aasland in :gh:`77617`.)

* Add the :attr:`~sqlite3.Connection.autocommit` attribute
to :class:`~sqlite3.Connection`
and the *autocommit* parameter to :func:`~sqlite3.connect`
to control :pep:`249`-compliant
:ref:`transaction handling <sqlite3-transaction-control-autocommit>`.
(Contributed by Erlend E. Aasland in :gh:`83638`.)

threading
---------

Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_global_objects_fini_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Include/internal/pycore_global_strings.h
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(as_integer_ratio)
STRUCT_FOR_ID(attribute)
STRUCT_FOR_ID(authorizer_callback)
STRUCT_FOR_ID(autocommit)
STRUCT_FOR_ID(b)
STRUCT_FOR_ID(backtick)
STRUCT_FOR_ID(base)
Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_runtime_init_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Include/internal/pycore_unicodeobject_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading