Closed
Description
For a reason which escapes me, population_session.h5 is 3Mb (while the same data in .xlsx is 11.8Kb) and demography.* are a bit large too, making the total size of the larray package larger than I would prefer.
{{ message }}
For a reason which escapes me, population_session.h5 is 3Mb (while the same data in .xlsx is 11.8Kb) and demography.* are a bit large too, making the total size of the larray package larger than I would prefer.
Activity
alixdamman commentedon Jun 27, 2019
In the PR #771 I have started to replace the use of demography dataset by the demography_eurostat dataset (which is actually population_session renamed).
Is demography used in the unittests?
gdementen commentedon Jun 27, 2019
I don't remember...
alixdamman commentedon Jun 27, 2019
I temporarily changed the name of demography, ran the tests and found that demography is only used in the
load_example_data()function and in the tutorial.However, there is still a weird problem with demography_eurostat.h5. I'll take a look.
alixdamman commentedon Jun 27, 2019
After some tests I discovered that the size of demography_eurostat.h5 is huge because the session contains Axis objects with labels of dtype '<U'.
There is clearly a problem when dumping axes with unicode labels to an HDF5 file.
gdementen commentedon Jun 27, 2019
can we solve this?
alixdamman commentedon Jun 27, 2019
We need to convert unicode labels to bytes
- added module to generate the files contained in the test/data/ dire…
- added module to generate the files contained in the test/data/ dire…
- added module to generate the files contained in the test/data/ dire…
- added module to generate the files contained in the test/data/ dire…
- added module to generate the files contained in the test/data/ dire…
5 remaining items