Update 2020-08-11-efficient-pytorch-io-library-for-large-datasets-many-files-many-gpus.md

wookim3 · web-flow · commit fa4c276b527b · 2020-08-11T16:16:03.000-07:00
I noticed another small issue: at the very end, the blog post links to webdataset-examples; that’s an old repo.

It should just say “pytorch-imagenet-wds contains an example of how to use WebDataset with ImageNet, based on the PyTorch ImageNet example”; the link should go to github.com/tmbdev/pytorch-imagenet-wds
diff --git a/_posts/2020-08-11-efficient-pytorch-io-library-for-large-datasets-many-files-many-gpus.md b/_posts/2020-08-11-efficient-pytorch-io-library-for-large-datasets-many-files-many-gpus.md
@@ -149,7 +149,7 @@ For a general introduction to how we handle large scale training with WebDataset
 
 * [tensorcom](https://github.com/NVLabs/tensorcom) is a library supporting distributed data augmentation and RDMA to GPU.
 
-* [webdataset-examples](https://github.com/tmbdev/webdataset-examples) contains an example (and soon more examples) of how to use WebDataset in practice.
+* [pytorch-imagenet-wds](https://github.com/tmbdev/pytorch-imagenet-wds) contains an example of how to use WebDataset with ImageNet, based on the PyTorch ImageNet example.
 
 * [Bigdata 2019 Paper with Benchmarks](https://arxiv.org/abs/2001.01858)
 
