Closed
Description
The dtype description in an NPY file can be written in a way that prevents the file from being read. The attached array in bad_array.pkl.zip exhibits this problem. In this case the written dtype is [('id', '|V16'), ('has_fit', '|b1'), ('timeax', ('|O', {'vlen': dtype('float64')})), ('', '|V8'), ('profile', ('|O', {'vlen': dtype('float64')})), ('', '|V8'), ('area', '<f8')].
(Note that it was obtained from via h5py - if this is an issue with the format of the arrays produced by that library, I'm happy to file a bug there instead.)
Reproducing code example:
import numpy as np
import pickle
import io
with open('bad_array.pkl', 'rb') as f:
arr = pickle.load(f)
buf = io.BytesIO()
np.save(buf, arr)
buf.seek(0)
np.load(buf)Error message:
Traceback (most recent call last):
File "scripts/badnpy.py", line 11, in <module>
np.load(buf)
File "/Users/colingavin/Code/908/clarent2/algo/py/.venv/lib/python3.7/site-packages/numpy/lib/npyio.py", line 453, in load
pickle_kwargs=pickle_kwargs)
File "/Users/colingavin/Code/908/clarent2/algo/py/.venv/lib/python3.7/site-packages/numpy/lib/format.py", line 712, in read_array
shape, fortran_order, dtype = _read_array_header(fp, version)
File "/Users/colingavin/Code/908/clarent2/algo/py/.venv/lib/python3.7/site-packages/numpy/lib/format.py", line 578, in _read_array_header
d = safe_eval(header)
File "/Users/colingavin/Code/908/clarent2/algo/py/.venv/lib/python3.7/site-packages/numpy/lib/utils.py", line 1139, in safe_eval
return ast.literal_eval(source)
File "/Users/colingavin/.pyenv/versions/3.7.2/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 91, in literal_eval
return _convert(node_or_string)
File "/Users/colingavin/.pyenv/versions/3.7.2/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 79, in _convert
map(_convert, node.values)))
File "/Users/colingavin/.pyenv/versions/3.7.2/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 74, in _convert
return list(map(_convert, node.elts))
File "/Users/colingavin/.pyenv/versions/3.7.2/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 72, in _convert
return tuple(map(_convert, node.elts))
File "/Users/colingavin/.pyenv/versions/3.7.2/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 72, in _convert
return tuple(map(_convert, node.elts))
File "/Users/colingavin/.pyenv/versions/3.7.2/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 79, in _convert
map(_convert, node.values)))
File "/Users/colingavin/.pyenv/versions/3.7.2/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 90, in _convert
return _convert_signed_num(node)
File "/Users/colingavin/.pyenv/versions/3.7.2/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 63, in _convert_signed_num
return _convert_num(node)
File "/Users/colingavin/.pyenv/versions/3.7.2/Python.framework/Versions/3.7/lib/python3.7/ast.py", line 55, in _convert_num
raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Call object at 0x10b965a58>
Numpy/Python version information:
1.17.0 3.7.2 (default, Jul 14 2019, 10:22:38)
[Clang 9.0.0 (clang-900.0.39.2)]