Skip to content

gh-128505: Expose an interface to sqlite3_file_control #128507

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions Doc/library/sqlite3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -579,6 +579,61 @@ Module constants
.. deprecated-removed:: 3.12 3.14
The :data:`!version` and :data:`!version_info` constants.

.. _sqlite3-fcntl-constants:

.. data:: SQLITE_FCNTL_LOCKSTATE
SQLITE_FCNTL_GET_LOCKPROXYFILE
SQLITE_FCNTL_SET_LOCKPROXYFILE
SQLITE_FCNTL_LAST_ERRNO
SQLITE_FCNTL_SIZE_HINT
SQLITE_FCNTL_CHUNK_SIZE
SQLITE_FCNTL_FILE_POINTER
SQLITE_FCNTL_SYNC_OMITTED
SQLITE_FCNTL_WIN32_AV_RETRY
SQLITE_FCNTL_PERSIST_WAL
SQLITE_FCNTL_OVERWRITE
SQLITE_FCNTL_POWERSAFE_OVERWRITE
SQLITE_FCNTL_PRAGMA
SQLITE_FCNTL_BUSYHANDLER
SQLITE_FCNTL_MMAP_SIZE
SQLITE_FCNTL_TRACE
SQLITE_FCNTL_HAS_MOVED
SQLITE_FCNTL_SYNC
SQLITE_FCNTL_COMMIT_PHASETWO
SQLITE_FCNTL_WIN32_SET_HANDLE
SQLITE_FCNTL_WAL_BLOCK
SQLITE_FCNTL_ZIPVFS
SQLITE_FCNTL_RBU
SQLITE_FCNTL_VFS_POINTER
SQLITE_FCNTL_JOURNAL_POINTER
SQLITE_FCNTL_WIN32_GET_HANDLE
SQLITE_FCNTL_PDB
SQLITE_FCNTL_BEGIN_ATOMIC_WRITE
SQLITE_FCNTL_COMMIT_ATOMIC_WRITE
SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE
SQLITE_FCNTL_LOCK_TIMEOUT
SQLITE_FCNTL_DATA_VERSION
SQLITE_FCNTL_SIZE_LIMIT
SQLITE_FCNTL_CKPT_DONE
SQLITE_FCNTL_RESERVE_BYTES
SQLITE_FCNTL_CKPT_START
SQLITE_FCNTL_EXTERNAL_READER
SQLITE_FCNTL_CKSM_FILE
SQLITE_FCNTL_RESET_CACHE
SQLITE_FCNTL_NULL_IO

These constants are used for the :meth:`Connection.file_control` method.

The availability of these constants varies depending on the version of SQLite
Python was compiled with.

.. versionadded:: 3.14

.. seealso::

https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html
SQLite docs: Standard File Control Opcodes

.. _sqlite3-connection-objects:

Connection objects
Expand Down Expand Up @@ -1288,6 +1343,24 @@ Connection objects

.. versionadded:: 3.12

.. method:: file_control(op, val, /, name="main")

Invoke a file control method on the database.
Opcodes which take non-integer arguments are not supported.

:param int op:
The :ref:`SQLITE_FCNTL_* constant <sqlite3-fcntl-constants>` to invoke.

:param int arg:
The argument to pass to the operation.

:param str name:
the database name to operate against.

:rtype: int

.. versionadded:: 3.14

.. method:: serialize(*, name="main")

Serialize a database into a :class:`bytes` object. For an
Expand Down
26 changes: 26 additions & 0 deletions Lib/test/test_sqlite3/test_dbapi.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import sqlite3 as sqlite
import subprocess
import sys
import tempfile
import threading
import unittest
import urllib.parse
Expand Down Expand Up @@ -724,6 +725,31 @@ def test_database_keyword(self):
with contextlib.closing(sqlite.connect(database=":memory:")) as cx:
self.assertEqual(type(cx), sqlite.Connection)

@unittest.skipIf(sys.platform == "darwin", "skipped on macOS")
def test_wal_preservation(self):
with tempfile.TemporaryDirectory() as dirname:
path = os.path.join(dirname, "db.sqlite")
with contextlib.closing(sqlite.connect(path)) as cx:
cx.file_control(sqlite.SQLITE_FCNTL_PERSIST_WAL, 1)
cu = cx.cursor()
cu.execute("PRAGMA journal_mode = WAL")
cu.execute("CREATE TABLE foo (id int)")
cu.execute("INSERT INTO foo (id) VALUES (1)")
self.assertTrue(os.path.exists(path + "-wal"))
self.assertTrue(os.path.exists(path + "-wal"))

with contextlib.closing(sqlite.connect(path)) as cx:
cu = cx.cursor()
self.assertTrue(os.path.exists(path + "-wal"))
cu.execute("INSERT INTO foo (id) VALUES (2)")
self.assertFalse(os.path.exists(path + "-wal"))


def test_file_control_raises(self):
with memory_database() as cx:
with self.assertRaises(sqlite.InternalError):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

InternalError sounds suspicious. What's the underlying SQLite error code? SQLITE_MISUSE?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the failed test run, the full exception is sqlite3.InternalError: unknown operation, which appears to correspond to SQLITE_NOTFOUND.

cx.file_control(sqlite.SQLITE_FCNTL_PERSIST_WAL, 1)


class CursorTests(unittest.TestCase):
def setUp(self):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
sqlite Connection objects now expose a method
:meth:`sqlite3.Connection.file_control`, which is a thin wrapper for
`sqlite3_file_control <https://www.sqlite.org/c3ref/file_control.html>`_.
2 changes: 1 addition & 1 deletion Modules/_sqlite/blob.c
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ static void
blob_seterror(pysqlite_Blob *self, int rc)
{
assert(self->connection != NULL);
_pysqlite_seterror(self->connection->state, self->connection->db);
set_error_from_db(self->connection->state, self->connection->db);
}

static PyObject *
Expand Down
95 changes: 94 additions & 1 deletion Modules/_sqlite/clinic/connection.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading