Skip to content

Commit b10b386

Browse files
committed
Add caveat about inconsistent pickles
1 parent d55a50e commit b10b386

File tree

1 file changed

+12
-2
lines changed

1 file changed

+12
-2
lines changed

docs/tutorial.rst

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -863,8 +863,17 @@ protocol`_ is not used. Neither the `__hash__` nor `__eq__` methods are used
863863
for lookups. Instead lookups depend on the serialization method defined by
864864
:class:`Disk <diskcache.Disk>` objects. For strings, bytes, integers, and
865865
floats, equality matches Python's definition. But large integers and all other
866-
types will be converted to bytes using pickling and the bytes representation
867-
will define equality.
866+
types will be converted to bytes and the bytes representation will define
867+
equality.
868+
869+
The default :class:`diskcache.Disk` serialization uses pickling for both keys
870+
and values. Unfortunately, pickling produces inconsistencies sometimes when
871+
applied to container data types like tuples. Two equal tuples may serialize to
872+
different bytes objects using pickle. The likelihood of differences is reduced
873+
by using `pickletools.optimize` but still inconsistencies occur (`#54`_). The
874+
inconsistent serialized pickle values is particularly problematic when applied
875+
to the key in the cache. Consider using an alternative Disk type, like
876+
:class:`JSONDisk <diskcache.JSONDisk>`, for consistent serialization of keys.
868877

869878
SQLite is used to synchronize database access between threads and processes and
870879
as such inherits all SQLite caveats. Most notably SQLite is `not recommended`_
@@ -898,6 +907,7 @@ does not account the size of directories themselves or other filesystem
898907
metadata. If directory count or size is a concern then consider implementing an
899908
alternative :class:`Disk <diskcache.Disk>`.
900909

910+
.. _`#54`: https://github.com/grantjenks/python-diskcache/issues/54
901911
.. _`hash protocol`: https://docs.python.org/library/functions.html#hash
902912
.. _`not recommended`: https://www.sqlite.org/faq.html#q5
903913
.. _`performs poorly`: https://www.pythonanywhere.com/forums/topic/1847/

0 commit comments

Comments
 (0)