Skip to content

gh-99813: Start using SSL_sendfile when available #99907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 55 commits into from
Jul 12, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
a018beb
Add methods for checking whether kTLS is used
illia-v Sep 11, 2022
72e7f5e
Start using `SSL_sendfile` when available
illia-v Nov 30, 2022
cf24d78
Try fixing warnings
illia-v Dec 5, 2022
e9ef747
Use unsigned `size_t` instead of `Py_ssize_t`
illia-v Dec 5, 2022
646d328
Merge branch 'main' into SSL_sendfile
illia-v Dec 5, 2022
76353c7
Merge branch 'main'
illia-v Mar 13, 2023
14832de
Make `sendfile_impl` more similar to `write_impl`
illia-v Mar 13, 2023
6b41d31
Modify `test_ssl` to test `SSL_sendfile` calls
illia-v Mar 13, 2023
8ac6ff9
Modify documentation and add a news entry
illia-v Mar 13, 2023
5cf9483
Fix a test
illia-v Mar 13, 2023
fc8e82f
Rename `uses_ktls_for_write` to `uses_ktls_for_send`
illia-v Mar 13, 2023
7fc9b50
Merge branch 'main' into SSL_sendfile
illia-v Jul 25, 2023
1ba04ae
Update `versionchanged`
illia-v Jul 25, 2023
365e0c0
Update a test to get successful HTTP responses
illia-v Jul 25, 2023
8fbc955
Add setting errors
illia-v Jul 25, 2023
21ea0ca
Fix conditions
illia-v Jul 25, 2023
17d9685
Add handling of `SSL_R_UNINITIALIZED`
illia-v Jul 26, 2023
a92ae7e
Merge branch 'main' into SSL_sendfile
illia-v Jul 26, 2023
4e8b6d8
Modify `test_sendfile` to avoid the internet
illia-v Jul 26, 2023
379d242
Refactor `test_sendfile` a bit
illia-v Jul 26, 2023
7de3dcb
Merge branch 'main' into SSL_sendfile
illia-v Oct 4, 2023
bbe21c0
Merge branch 'main' into SSL_sendfile
illia-v Oct 19, 2023
7323ec0
Try to fix new warnings
illia-v Oct 19, 2023
2dc6947
Merge branch 'main' into SSL_sendfile
illia-v Feb 27, 2024
6e902ad
Merge branch 'main' into SSL_sendfile
illia-v Mar 27, 2024
e435b9a
Apply a change from ea9a296fce2f786b4cf43c7924e5de01061f27ca
illia-v Mar 27, 2024
ac1b2b2
Merge branch 'main' into SSL_sendfile
illia-v Jun 6, 2024
7d11a59
Merge branch 'main' into SSL_sendfile
illia-v Jul 31, 2024
99e89d3
Merge branch 'main' into SSL_sendfile
illia-v Sep 28, 2024
dc626c8
Merge branch 'main' into SSL_sendfile
illia-v Dec 27, 2024
291a5b7
Set `versionchanged` to next in docs
illia-v Dec 27, 2024
3193ba4
Merge branch 'main' into SSL_sendfile
illia-v Feb 3, 2025
d870f92
Merge branch 'main' into SSL_sendfile
illia-v Apr 10, 2025
94e522e
Apply some suggestions from code review
illia-v Apr 12, 2025
25bde6f
Apply suggestions to Python code
illia-v Apr 12, 2025
9a6a120
Improve style of `_ssl__SSLSocket_sendfile_impl`
illia-v Apr 12, 2025
eaa0b2c
Merge branch 'main' into SSL_sendfile
illia-v Apr 12, 2025
05a0c6c
Drop `_sendfile_use_ssl_sendfile`
illia-v Apr 13, 2025
6b4eed6
Merge remote-tracking branch 'python/main' into SSL_sendfile
illia-v Apr 13, 2025
31ed52d
Apply suggestions from code review
illia-v Apr 18, 2025
5a22e6b
Merge branch 'main' into SSL_sendfile
illia-v Apr 18, 2025
ada3f30
Improve kTLS checks for older OpenSSL
illia-v Apr 18, 2025
3350854
Merge branch 'main' into SSL_sendfile
illia-v Apr 28, 2025
3339673
Rename `uses_ktls_for_read` to `uses_ktls_for_recv`
illia-v Apr 28, 2025
ee573ad
Apply PEP 7
illia-v Apr 28, 2025
6795f62
Use an alternative method of returning booleans
illia-v Apr 28, 2025
6a35ac6
Mark new functions with `critical_section`
illia-v May 2, 2025
9dffdbd
Merge two if blocks
illia-v May 2, 2025
19d5746
Reword docs
illia-v May 2, 2025
ef3744e
Update the news entry
illia-v May 2, 2025
e64f329
Use `Py_RETURN_FALSE` again
illia-v May 2, 2025
028067a
Merge remote-tracking branch 'python/main' into SSL_sendfile
illia-v May 2, 2025
6e099fa
Merge branch 'main' into SSL_sendfile
illia-v May 5, 2025
f363ec3
Merge branch 'main' into SSL_sendfile
illia-v May 19, 2025
7c071fd
Merge branch 'main' into SSL_sendfile
picnixz Jul 12, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Apply some suggestions from code review
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
  • Loading branch information
illia-v and picnixz authored Apr 12, 2025
commit 94e522ee82d0328bbd3e8db281310d32245514f9
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Python now uses ``SSL_sendifle`` internally when it is possible (see
Python now uses ``SSL_sendfile`` internally when it is possible (see
:data:`~ssl.OP_ENABLE_KTLS`.) The function sends a file more efficiently
because it performs TLS encryption in the kernel to avoid additional context
switches. Patch by Illia Volochii.
29 changes: 15 additions & 14 deletions Modules/_ssl.c
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,6 @@
#endif



#ifdef BIO_get_ktls_send
#ifdef MS_WINDOWS
typedef long long Py_off_t;
Expand All @@ -90,9 +89,7 @@ Py_off_t_converter(PyObject *arg, void *addr)
#else
*((Py_off_t *)addr) = PyLong_AsLong(arg);
#endif
if (PyErr_Occurred())
return 0;
return 1;
return PyErr_Occurred() ? 0 : 1;
}

/*[python input]
Expand Down Expand Up @@ -2482,9 +2479,9 @@ _ssl__SSLSocket_uses_ktls_for_send_impl(PySSLSocket *self)
{
#ifdef BIO_get_ktls_send
int uses = BIO_get_ktls_send(SSL_get_wbio(self->ssl));
return PyBool_FromLong((long)uses);
return PyBool_FromLong(uses);
#else
return Py_False;
Py_RETURN_FALSE;
#endif
}

Expand All @@ -2500,9 +2497,9 @@ _ssl__SSLSocket_uses_ktls_for_read_impl(PySSLSocket *self)
{
#ifdef BIO_get_ktls_recv
int uses = BIO_get_ktls_recv(SSL_get_rbio(self->ssl));
return PyBool_FromLong((long)uses);
return PyBool_FromLong(uses);
#else
return Py_False;
Py_RETURN_FALSE;
#endif
}

Expand Down Expand Up @@ -2539,7 +2536,7 @@ _ssl__SSLSocket_sendfile_impl(PySSLSocket *self, int fd, Py_off_t offset,
int has_timeout;

if (sock != NULL) {
if (((PyObject*)sock) == Py_None) {
if ((PyObject *)sock == Py_None) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a real issue, but thinking out loud for me and Bénédikt: does Py_Is come with _PyObject_CAST? If so, it seems more useful than == here. If not, we should add it.

_setSSLError(get_state_sock(self),
"Underlying socket connection gone",
PY_SSL_ERROR_NO_SOCKET, __FILE__, __LINE__);
Expand Down Expand Up @@ -2583,8 +2580,9 @@ _ssl__SSLSocket_sendfile_impl(PySSLSocket *self, int fd, Py_off_t offset,
PySSL_END_ALLOW_THREADS
self->err = err;

if (PyErr_CheckSignals())
if (PyErr_CheckSignals()) {
goto error;
}

if (has_timeout) {
timeout = _PyDeadline_Get(deadline);
Expand Down Expand Up @@ -2613,7 +2611,8 @@ _ssl__SSLSocket_sendfile_impl(PySSLSocket *self, int fd, Py_off_t offset,
err.ssl == SSL_ERROR_WANT_WRITE);

if (err.ssl == SSL_ERROR_SSL
&& ERR_GET_REASON(ERR_peek_error()) == SSL_R_UNINITIALIZED) {
&& ERR_GET_REASON(ERR_peek_error()) == SSL_R_UNINITIALIZED)
{
/* OpenSSL fails to return SSL_ERROR_SYSCALL if an error
* happens in sendfile(), and returns SSL_ERROR_SSL with
* SSL_R_UNINITIALIZED reason instead. */
Expand All @@ -2623,14 +2622,16 @@ _ssl__SSLSocket_sendfile_impl(PySSLSocket *self, int fd, Py_off_t offset,
goto error;
}
Py_XDECREF(sock);
if (retval < 0)
if (retval < 0) {
return PySSL_SetError(self, __FILE__, __LINE__);
if (PySSL_ChainExceptions(self) < 0)
}
if (PySSL_ChainExceptions(self) < 0) {
return NULL;
}
return PyLong_FromSize_t(retval);
error:
Py_XDECREF(sock);
PySSL_ChainExceptions(self);
(void)PySSL_ChainExceptions(self);
return NULL;
}
#endif /* BIO_get_ktls_send */
Expand Down
Loading