Skip to content

gh-132983: [DRAFT] Test _zstd code with Python test suite #133185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 39 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
9814e3b
Add _zstd module
emmatyping Apr 26, 2025
fda87c8
Add _zstd to modules
emmatyping Apr 26, 2025
887e564
Fix path for compression.zstd module
emmatyping Apr 26, 2025
cdba656
Ignore _zstd module like _io
emmatyping Apr 26, 2025
6b67e9b
Expand module state macros to improve code quality
emmatyping Apr 26, 2025
a99a5d2
Remove backticks suggested in review
emmatyping Apr 27, 2025
02cd17a
Use critical sections to lock object state
emmatyping Apr 27, 2025
54eca74
Remove compress/decompress and mark module as not reliant on the GIL
emmatyping Apr 27, 2025
f605956
Lift critical section to avoid clang warning
emmatyping Apr 27, 2025
2eadc65
Respond to comments by picnixz
emmatyping Apr 27, 2025
8eac354
Call out pyzstd explicitly in license description
emmatyping Apr 27, 2025
26775be
Use a much more robust implementation...
emmatyping Apr 27, 2025
eae460f
Use PyList_GetItemRef for thread safety purposes
emmatyping Apr 27, 2025
2ab5e4a
Use a macro for the minimum supported version
emmatyping Apr 27, 2025
d5bf1c1
remove const from primivite types
emmatyping Apr 27, 2025
9e92b9f
Use PyMem_New in another spot
emmatyping Apr 27, 2025
47f815a
Simplify error handling in _get_frame_size
emmatyping Apr 27, 2025
6a4f7b8
Another simplification of error handling in get_frame_info
emmatyping Apr 27, 2025
d7b3805
Rename _module_state to mod_state
emmatyping Apr 27, 2025
c225ea6
Rewrite comment explaining the context of the code
emmatyping Apr 28, 2025
6e8c61c
Add link to pyzstd
emmatyping Apr 28, 2025
e52ad06
Add TODO about refactoring dict training code
emmatyping Apr 28, 2025
2a1ad8b
Use PyModule_AddObjectRef over PyModule_AddObject
emmatyping Apr 28, 2025
94473b9
Check result of OutputBufferGrow
emmatyping Apr 28, 2025
e2b2515
Simplify return logic in `add_constant_to_type`
emmatyping Apr 29, 2025
cd2f085
Ignore return value of _zstd_clear()
emmatyping Apr 29, 2025
79e174f
Remove redundant comments
emmatyping Apr 29, 2025
ce6f79c
Remove __reduce__ from ZstdDict
emmatyping Apr 29, 2025
e15dd85
Use PyUnicode_FromFormat instead of a buffer
emmatyping Apr 29, 2025
685a3d1
Don't use C constants/types in error messages
emmatyping Apr 29, 2025
1b9f786
Make error messages easier to understand for Python users
emmatyping Apr 29, 2025
40c653c
Lower minimum required version 1.4.0
emmatyping Apr 30, 2025
428677d
Use casts and make slot function signatures correct
emmatyping Apr 30, 2025
0962bbb
Be consistent with CPython on const usage
emmatyping Apr 30, 2025
85efc18
Make else clauses in line with PEP 7
emmatyping Apr 30, 2025
cadf6e4
Fix over-indented blocks in argument clinic
emmatyping Apr 30, 2025
e45c22a
Merge branch 'main' into 3.14-zstd-c-code
emmatyping Apr 30, 2025
5c1e360
Add Python files
emmatyping Apr 30, 2025
1ad29f2
Fix byteswarning in test
emmatyping Apr 30, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove __reduce__ from ZstdDict
We should instead document that to pickle a dictionary a user should use
the `.dict_content` attribute.
  • Loading branch information
emmatyping committed Apr 29, 2025
commit ce6f79c37c981f1fe64a153887ef81d0a202f83d
20 changes: 1 addition & 19 deletions Modules/_zstd/clinic/zdict.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 0 additions & 27 deletions Modules/_zstd/zdict.c
Original file line number Diff line number Diff line change
Expand Up @@ -129,36 +129,10 @@ _zstd_ZstdDict___init___impl(ZstdDict *self, PyObject *dict_content,
return 0;
}


/*[clinic input]
_zstd.ZstdDict.__reduce__


Intentionally not supporting pickle.
[clinic start generated code]*/

static PyObject *
_zstd_ZstdDict___reduce___impl(ZstdDict *self)
/*[clinic end generated code: output=5c9b8a3550429417 input=2d69a2b4775ff76d]*/
{
PyErr_SetString(PyExc_TypeError,
"ZstdDict object intentionally doesn't support pickle. If you need "
"to save zstd dictionary to disk, please save .dict_content "
"attribute, it's a bytes object. So that the zstd dictionary "
"can be used with other programs.");
return NULL;
}

#define clinic_state() (get_zstd_state(type))
#include "clinic/zdict.c.h"
#undef clinic_state

static PyMethodDef ZstdDict_methods[] = {
_ZSTD_ZSTDDICT___REDUCE___METHODDEF

{0}
};

PyDoc_STRVAR(ZstdDict_dictid_doc,
"ID of zstd dictionary, a 32-bit unsigned int value.\n\n"
"Non-zero means ordinary dictionary, was created by zstd functions, follow\n"
Expand Down Expand Up @@ -270,7 +244,6 @@ ZstdDict_length(ZstdDict *self)
}

static PyType_Slot zstddict_slots[] = {
{Py_tp_methods, ZstdDict_methods},
{Py_tp_members, ZstdDict_members},
{Py_tp_getset, ZstdDict_getset},
{Py_tp_new, _zstd_ZstdDict_new},
Expand Down