Skip to content

Commit ac56fcc

Browse files
pgansslepganssle
committed
PEP 615: Tests and implementation for zoneinfo
This is the initial implementation of PEP 615, the zoneinfo module, ported from the standalone reference implementation (see https://www.python.org/dev/peps/pep-0615/#reference-implementation for a link, which has a more detailed commit history). This includes (hopefully) all functional elements described in the PEP, but documentation is found in a separate PR. This includes: 1. A pure python implementation of the ZoneInfo class 2. A C accelerated implementation of the ZoneInfo class 3. Tests with 100% branch coverage for the Python code (though C code coverage is less than 100%). 4. A compile-time configuration option on Linux (though not on Windows) Differences from the reference implementation: - The module is arranged slightly differently: the accelerated module is `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates some changes in the test support function. (Suggested by Victor Stinner and Steve Dower.) - The tests are arranged slightly differently and do not include the property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py because we may do some refactoring in the future that would likely require this separation anyway; we may: - include the property tests - automatically run all the tests against both pure Python and C, rather than manually constructing C and Python test classes (similar to the way this works with test_datetime.py, which generates C and Python test cases from datetimetester.py). - This includes a compile-time configuration option on Linux (though not on Windows); added with much help from Thomas Wouters. - Integration into the CPython build system is obviously different from building a standalone zoneinfo module wheel. - This includes configuration to install the tzdata package as part of CI, though only on the coverage jobs. Introducing a PyPI dependency as part of the CI build was controversial, and this is seen as less of a major change, since the coverage jobs already depend on pip and PyPI. Additional changes that were introduced as part of this PR, most / all of which were backported to the reference implementation: - Fixed reference and memory leaks With much debugging help from Pablo Galindo - Added smoke tests ensuring that the C and Python modules are built The import machinery can be somewhat fragile, and the "seamlessly falls back to pure Python" nature of this module makes it so that a problem building the C extension or a failure to import the pure Python version might easily go unnoticed. - Adjustments to zoneinfo.__dir__ Suggested by Petr Viktorin. - Slight refactorings as suggested by Steve Dower. - Removed unnecessary if check on std_abbr Discovered this because of a missing line in branch coverage.
1 parent 6e57237 commit ac56fcc

27 files changed

+6383
-2
lines changed

‎.github/workflows/coverage.yml

+1
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ jobs:
4848
./python -m venv .venv
4949
source ./.venv/bin/activate
5050
python -m pip install -U coverage
51+
python -m pip install -r Misc/requirements-test.txt
5152
python -m test.pythoninfo
5253
- name: 'Tests with coverage'
5354
run: >

‎.travis.yml

+1
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ matrix:
8787
# Need a venv that can parse covered code.
8888
- ./python -m venv venv
8989
- ./venv/bin/python -m pip install -U coverage
90+
- ./venv/bin/python -m pip install -r Misc/requirements-test.txt
9091
- ./venv/bin/python -m test.pythoninfo
9192
script:
9293
# Skip tests that re-run the entire test suite.

‎Lib/sysconfig.py

+1
Original file line numberDiff line numberDiff line change
@@ -546,6 +546,7 @@ def get_config_vars(*args):
546546

547547
if os.name == 'nt':
548548
_init_non_posix(_CONFIG_VARS)
549+
_CONFIG_VARS['TZPATH'] = ''
549550
if os.name == 'posix':
550551
_init_posix(_CONFIG_VARS)
551552
# For backward compatibility, see issue19555

‎Lib/test/test_zoneinfo/__init__.py

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
from .test_zoneinfo import *

‎Lib/test/test_zoneinfo/__main__.py

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
import unittest
2+
3+
unittest.main('test.test_zoneinfo')

‎Lib/test/test_zoneinfo/_support.py

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
import contextlib
2+
import functools
3+
import sys
4+
import threading
5+
import unittest
6+
from test.support import import_fresh_module
7+
8+
OS_ENV_LOCK = threading.Lock()
9+
TZPATH_LOCK = threading.Lock()
10+
TZPATH_TEST_LOCK = threading.Lock()
11+
12+
13+
def call_once(f):
14+
"""Decorator that ensures a function is only ever called once."""
15+
lock = threading.Lock()
16+
cached = functools.lru_cache(None)(f)
17+
18+
@functools.wraps(f)
19+
def inner():
20+
with lock:
21+
return cached()
22+
23+
return inner
24+
25+
26+
@call_once
27+
def get_modules():
28+
"""Retrieve two copies of zoneinfo: pure Python and C accelerated.
29+
30+
Because this function manipulates the import system in a way that might
31+
be fragile or do unexpected things if it is run many times, it uses a
32+
`call_once` decorator to ensure that this is only ever called exactly
33+
one time — in other words, when using this function you will only ever
34+
get one copy of each module rather than a fresh import each time.
35+
"""
36+
import zoneinfo as c_module
37+
38+
py_module = import_fresh_module("zoneinfo", blocked=["_zoneinfo"])
39+
40+
return py_module, c_module
41+
42+
43+
@contextlib.contextmanager
44+
def set_zoneinfo_module(module):
45+
"""Make sure sys.modules["zoneinfo"] refers to `module`.
46+
47+
This is necessary because `pickle` will refuse to serialize
48+
an type calling itself `zoneinfo.ZoneInfo` unless `zoneinfo.ZoneInfo`
49+
refers to the same object.
50+
"""
51+
52+
NOT_PRESENT = object()
53+
old_zoneinfo = sys.modules.get("zoneinfo", NOT_PRESENT)
54+
sys.modules["zoneinfo"] = module
55+
yield
56+
if old_zoneinfo is not NOT_PRESENT:
57+
sys.modules["zoneinfo"] = old_zoneinfo
58+
else: # pragma: nocover
59+
sys.modules.pop("zoneinfo")
60+
61+
62+
class ZoneInfoTestBase(unittest.TestCase):
63+
@classmethod
64+
def setUpClass(cls):
65+
cls.klass = cls.module.ZoneInfo
66+
super().setUpClass()
67+
68+
@contextlib.contextmanager
69+
def tzpath_context(self, tzpath, lock=TZPATH_LOCK):
70+
with lock:
71+
old_path = self.module.TZPATH
72+
try:
73+
self.module.reset_tzpath(tzpath)
74+
yield
75+
finally:
76+
self.module.reset_tzpath(old_path)

‎Lib/test/test_zoneinfo/data/update_test_data.py

+122
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
"""
2+
Script to automatically generate a JSON file containing time zone information.
3+
4+
This is done to allow "pinning" a small subset of the tzdata in the tests,
5+
since we are testing properties of a file that may be subject to change. For
6+
example, the behavior in the far future of any given zone is likely to change,
7+
but "does this give the right answer for this file in 2040" is still an
8+
important property to test.
9+
10+
This must be run from a computer with zoneinfo data installed.
11+
"""
12+
from __future__ import annotations
13+
14+
import base64
15+
import functools
16+
import json
17+
import lzma
18+
import pathlib
19+
import textwrap
20+
import typing
21+
22+
import zoneinfo
23+
24+
KEYS = [
25+
"Africa/Abidjan",
26+
"Africa/Casablanca",
27+
"America/Los_Angeles",
28+
"America/Santiago",
29+
"Asia/Tokyo",
30+
"Australia/Sydney",
31+
"Europe/Dublin",
32+
"Europe/Lisbon",
33+
"Europe/London",
34+
"Pacific/Kiritimati",
35+
"UTC",
36+
]
37+
38+
TEST_DATA_LOC = pathlib.Path(__file__).parent
39+
40+
41+
@functools.lru_cache(maxsize=None)
42+
def get_zoneinfo_path() -> pathlib.Path:
43+
"""Get the first zoneinfo directory on TZPATH containing the "UTC" zone."""
44+
key = "UTC"
45+
for path in map(pathlib.Path, zoneinfo.TZPATH):
46+
if (path / key).exists():
47+
return path
48+
else:
49+
raise OSError("Cannot find time zone data.")
50+
51+
52+
def get_zoneinfo_metadata() -> typing.Dict[str, str]:
53+
path = get_zoneinfo_path()
54+
55+
tzdata_zi = path / "tzdata.zi"
56+
if not tzdata_zi.exists():
57+
# tzdata.zi is necessary to get the version information
58+
raise OSError("Time zone data does not include tzdata.zi.")
59+
60+
with open(tzdata_zi, "r") as f:
61+
version_line = next(f)
62+
63+
_, version = version_line.strip().rsplit(" ", 1)
64+
65+
if (
66+
not version[0:4].isdigit()
67+
or len(version) < 5
68+
or not version[4:].isalpha()
69+
):
70+
raise ValueError(
71+
"Version string should be YYYYx, "
72+
+ "where YYYY is the year and x is a letter; "
73+
+ f"found: {version}"
74+
)
75+
76+
return {"version": version}
77+
78+
79+
def get_zoneinfo(key: str) -> bytes:
80+
path = get_zoneinfo_path()
81+
82+
with open(path / key, "rb") as f:
83+
return f.read()
84+
85+
86+
def encode_compressed(data: bytes) -> typing.List[str]:
87+
compressed_zone = lzma.compress(data)
88+
raw = base64.b85encode(compressed_zone)
89+
90+
raw_data_str = raw.decode("utf-8")
91+
92+
data_str = textwrap.wrap(raw_data_str, width=70)
93+
return data_str
94+
95+
96+
def load_compressed_keys() -> typing.Dict[str, typing.List[str]]:
97+
output = {key: encode_compressed(get_zoneinfo(key)) for key in KEYS}
98+
99+
return output
100+
101+
102+
def update_test_data(fname: str = "zoneinfo_data.json") -> None:
103+
TEST_DATA_LOC.mkdir(exist_ok=True, parents=True)
104+
105+
# Annotation required: https://github.com/python/mypy/issues/8772
106+
json_kwargs: typing.Dict[str, typing.Any] = dict(
107+
indent=2, sort_keys=True,
108+
)
109+
110+
compressed_keys = load_compressed_keys()
111+
metadata = get_zoneinfo_metadata()
112+
output = {
113+
"metadata": metadata,
114+
"data": compressed_keys,
115+
}
116+
117+
with open(TEST_DATA_LOC / fname, "w") as f:
118+
json.dump(output, f, **json_kwargs)
119+
120+
121+
if __name__ == "__main__":
122+
update_test_data()

0 commit comments

Comments
 (0)