Skip to content

gh-101558: Add time.sleep_until() #101559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions Doc/library/time.rst
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,22 @@ Functions
by a signal, except if the signal handler raises an exception (see
:pep:`475` for the rationale).

.. function:: sleep_until(secs)

Like :func:`sleep`, but sleep until the specified time of the system clock (as
returned by :func:`time`). This can be used, for example, to schedule events
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced from this documentation what is the benefit of using this function instead of existing code:

dt = deadline - time.time()
if dt > 0: time.sleep(dt)

How does it behave differently? If it does behave the same, the function has no benefit?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is your point about improving the documentation or are you questioning the usefulness of the function in general? I'd be happy to improve the documentation if that's all that's missing, but I'd prefer to finalize the implementation first (see above).

The code you showed does differ from sleep_until in at least one IMHO significant way: The fact that sleep_until (when backed by the SetWaitableTimerEx or clock_nanosleep implementations) is sensitive to changes to the system clock is actually desirable in some situations. Imagine for example a measurement system consisting of multiple processes on a single machine, or spread across several machines, where time is kept synchronized by NTP, running for several days with a measurement interval on the order of minutes. On this timescale, differences in the clocks of a few ms don't matter, but if one of the measurement processes were to drift off relative to the others, as it could with the code you showed, that wouldn't be too good.

Other than that, for short measurement intervals, in theory the code you showed introduces a small offset due to there being several instructions in between taking the current time and scheduling the sleep, which may make a difference on a heavily loaded machine.

at a specific timestamp obtained from
:meth:`datetime.timestamp <datetime.datetime.timestamp>`.

See the notes in :func:`sleep` on the behavior when interrupted and on accuracy.
Additional potential sources of inaccuracies include:

* Because this function uses the system clock as a reference, this means the
reference clock is adjustable and may jump backwards.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear to me what is the expected behavior when the system clock jumps backward or forward. Does the function sleeps longer/shorter?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a caller asks to sleep until an absolute time on a given clock and that clock then "jumps", the caller should still wake up when that absolute time has elapsed. For example, if the clock is set backwards after the caller blocks, then the caller should sleep for a longer duration because it will take a longer time for the clock to reach the chosen absolute value. This is not a violation of the caller's request: they asked to block until an absolute time, not to block for a given duration.

If sleeping a longer/shorter duration is unacceptable for a caller, then either (a) the caller should use a relative sleep, or (b) the caller should schedule the sleep against a clock that cannot "jump".

* On Unix, if ``clock_nanosleep()`` is not available, the absolute timeout
is emulated using ``nanosleep()`` or ``select()``.
Comment on lines +403 to +404
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you consider that it's important to document the implementation, I suggest to do it in a separated section, since it's unclear to me what's the relationship between the chosen implementation and inaccuracies. If you care about the theorical clock resolution, you should document it, it's not obviously to users what is the resolution of these functions.


.. versionadded:: 3.12

.. index::
single: % (percent); datetime format
Expand Down
12 changes: 12 additions & 0 deletions Lib/test/test_time.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,18 @@ def test_sleep(self):
self.assertRaises(ValueError, time.sleep, -1)
time.sleep(1.2)

def test_sleep_until(self):
start = time.time()
deadline = start + 2
time.sleep_until(deadline)
stop = time.time()
delta = stop - deadline
# cargo-cult these 50ms from test_monotonic (bpo-20101)
self.assertGreater(delta, -0.050)
# allow sleep_until to take up to 1s longer than planned
# (e.g. in case the system is under heavy load during testing)
self.assertLess(delta, 1.000)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my experience, such test tends to fail on busy buildbots. I would prefer to not test the "performance" (accuracy) of the function in such unit test.

I'm not sure that it's useful to write an unit test for this function. It's very hard to write a reliable on the clock accuracy.

What if the system clock changes during the test? :-(

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree, and I actually already had to increase that final assertLess because the test failed on a busy runner with a smaller value. When writing the test I took my cue from tests like test_monotonic, but I'm also fine with simplifying the test case, let me know.


def test_epoch(self):
# bpo-43869: Make sure that Python use the same Epoch on all platforms:
# January 1, 1970, 00:00:00 (UTC).
Expand Down
1 change: 1 addition & 0 deletions Misc/ACKS
Original file line number Diff line number Diff line change
Expand Up @@ -403,6 +403,7 @@ Lisandro Dalcin
Darren Dale
Andrew Dalke
Lars Damerow
Hauke Dämpfling
Evan Dandrea
Eric Daniel
Scott David Daniels
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Added the :func:`time.sleep_until` function, which allows sleeping until the
specified absolute time.
86 changes: 70 additions & 16 deletions Modules/timemodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ _PyTime_Init(void)


/* Forward declarations */
static int pysleep(_PyTime_t timeout);
static int pysleep(_PyTime_t timeout, int absolute);


typedef struct {
Expand Down Expand Up @@ -422,7 +422,7 @@ time_sleep(PyObject *self, PyObject *timeout_obj)
"sleep length must be non-negative");
return NULL;
}
if (pysleep(timeout) != 0) {
if (pysleep(timeout, 0) != 0) {
return NULL;
}
Py_RETURN_NONE;
Expand All @@ -434,6 +434,29 @@ PyDoc_STRVAR(sleep_doc,
Delay execution for a given number of seconds. The argument may be\n\
a floating point number for subsecond precision.");

static PyObject *
time_sleep_until(PyObject *self, PyObject *deadline_obj)
{
_PyTime_t deadline;
if (_PyTime_FromSecondsObject(&deadline, deadline_obj, _PyTime_ROUND_TIMEOUT)) {
return NULL;
}
if (deadline < 0) {
PyErr_SetString(PyExc_ValueError,
"sleep_until deadline must be non-negative");
return NULL;
}
if (pysleep(deadline, 1) != 0) {
return NULL;
}
Py_RETURN_NONE;
}

PyDoc_STRVAR(sleep_until_doc,
"sleep_until(seconds)\n\
\n\
Delay execution until the specified system clock time.");

static PyStructSequence_Field struct_time_type_fields[] = {
{"tm_year", "year, for example, 1993"},
{"tm_mon", "month of year, range [1, 12]"},
Expand Down Expand Up @@ -1868,6 +1891,7 @@ static PyMethodDef time_methods[] = {
{"pthread_getcpuclockid", time_pthread_getcpuclockid, METH_VARARGS, pthread_getcpuclockid_doc},
#endif
{"sleep", time_sleep, METH_O, sleep_doc},
{"sleep_until", time_sleep_until, METH_O, sleep_until_doc},
{"gmtime", time_gmtime, METH_VARARGS, gmtime_doc},
{"localtime", time_localtime, METH_VARARGS, localtime_doc},
{"asctime", time_asctime, METH_VARARGS, asctime_doc},
Expand Down Expand Up @@ -2132,8 +2156,9 @@ PyInit_time(void)
// time.sleep() implementation.
// On error, raise an exception and return -1.
// On success, return 0.
// If absolute==0, timeout is relative; otherwise timeout is absolute.
static int
pysleep(_PyTime_t timeout)
pysleep(_PyTime_t timeout, int absolute)
{
assert(timeout >= 0);

Expand All @@ -2145,13 +2170,27 @@ pysleep(_PyTime_t timeout)
#else
struct timeval timeout_tv;
#endif
_PyTime_t deadline, monotonic;
_PyTime_t deadline, reference;
int err = 0;

if (get_monotonic(&monotonic) < 0) {
return -1;
if (absolute) {
deadline = timeout;
#ifndef HAVE_CLOCK_NANOSLEEP
if (get_system_time(&reference) < 0) {
return -1;
}
timeout = deadline - reference;
if (timeout < 0) {
return 0;
}
#endif
}
else {
if (get_monotonic(&reference) < 0) {
return -1;
}
deadline = reference + timeout;
}
deadline = monotonic + timeout;
#ifdef HAVE_CLOCK_NANOSLEEP
if (_PyTime_AsTimespec(deadline, &timeout_abs) < 0) {
return -1;
Expand All @@ -2174,7 +2213,8 @@ pysleep(_PyTime_t timeout)
int ret;
Py_BEGIN_ALLOW_THREADS
#ifdef HAVE_CLOCK_NANOSLEEP
ret = clock_nanosleep(CLOCK_MONOTONIC, TIMER_ABSTIME, &timeout_abs, NULL);
ret = clock_nanosleep(absolute ? CLOCK_REALTIME : CLOCK_MONOTONIC,
TIMER_ABSTIME, &timeout_abs, NULL);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why need to use CLOCK_REALTIME? @vstinner mentioned in a mailing thread that CLOCK_REALTIME can jump forwards and backwards as the system time-of-day clock is changed.

Copy link
Author

@haukex haukex Feb 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quoting my reply from the email:

Yes, using the system clock has its caveats, which is why I mentioned it in the documentation, and I wouldn't mind highlighting these even more.

The use case in my specific case is long-running recurring measurements, for example a measurement every minute or every ten minutes over a period of many hours or often days. In these cases it makes much more sense to tie the measurements to wall-clock time, because often there are other measurement systems also using wall-clock time.

Although of course not everyone uses it, an NTP daemon like Chrony is able to adjust the system clock by changing the system clock frequency and thereby adjusting the system clock gradually, which keeps the system clock monotonic. Having multiple measurement computers stay synchronized via NTP is then much more reasonable. In the past I've built a system consisting of multiple machines, one of them using a GPS clock and acting as an NTP server, that does exactly this.

Since both clock_nanosleep and SetWaitableTimerEx allow sleeping until a specific time, my reasoning is simply that this functionality available in the lower-level API could be made available to the user so they can write simple loops - aside from the precision of clock_nanosleep appealing to my perfectionist side 😉

While there are schedulers like cron or similar, sometimes, a measurement may need to happen e.g. every second, where then the (admittedly small) inaccuracy introduced by calculating the sleep() timeout based on the current time can start to make a difference.

err = ret;
#elif defined(HAVE_NANOSLEEP)
ret = nanosleep(&timeout_ts, NULL);
Expand All @@ -2201,10 +2241,17 @@ pysleep(_PyTime_t timeout)
}

#ifndef HAVE_CLOCK_NANOSLEEP
if (get_monotonic(&monotonic) < 0) {
return -1;
if (absolute) {
if (get_system_time(&reference) < 0) {
return -1;
}
}
timeout = deadline - monotonic;
else {
if (get_monotonic(&reference) < 0) {
return -1;
}
}
timeout = deadline - reference;
if (timeout < 0) {
break;
}
Expand All @@ -2229,11 +2276,18 @@ pysleep(_PyTime_t timeout)
return 0;
}

LARGE_INTEGER relative_timeout;
LARGE_INTEGER due_time;
// No need to check for integer overflow, both types are signed
assert(sizeof(relative_timeout) == sizeof(timeout_100ns));
// SetWaitableTimer(): a negative due time indicates relative time
relative_timeout.QuadPart = -timeout_100ns;
assert(sizeof(due_time) == sizeof(timeout_100ns));
if (absolute) {
// Adjust from Unix time (1970-01-01) to Windows time (1601-01-01)
// (the inverse of what is done in py_get_system_clock)
due_time.QuadPart = timeout_100ns + 116444736000000000;
}
else {
// SetWaitableTimer(): a negative due time indicates relative time
due_time.QuadPart = -timeout_100ns;
}

HANDLE timer = CreateWaitableTimerExW(NULL, NULL, timer_flags,
TIMER_ALL_ACCESS);
Expand All @@ -2242,7 +2296,7 @@ pysleep(_PyTime_t timeout)
return -1;
}

if (!SetWaitableTimerEx(timer, &relative_timeout,
if (!SetWaitableTimerEx(timer, &due_time,
0, // no period; the timer is signaled once
NULL, NULL, // no completion routine
NULL, // no wake context; do not resume from suspend
Expand Down