Update on "Remove usage of fsspec in HF consolidation script"

ankitageorge · ankitageorge · commit fac79145abe9 · 2025-08-01T10:27:50.000-07:00
Moving towards just supporting local storage to take advantage of HF apis such as safe_open. This PR removes fsspec usages and relies on local storage only Differential Revision: [D78997975](https://our.internmc.facebook.com/intern/diff/D78997975/) cc H-Huang awgu wanchaol fegin fduwjj wz337 wconstab d4l3k pragupta [ghstack-poisoned]
diff --git a/torch/distributed/checkpoint/_consolidate_hf_safetensors.py b/torch/distributed/checkpoint/_consolidate_hf_safetensors.py
@@ -541,7 +541,7 @@ def consolidate_safetensors_files(
         output_files_data[output_path] = _OutputFileData()
 
     # Find all safetensors files in the input directory
-    safetensors_files = glob.glob(os.path.join(input_dir,f"*{SUFFIX}"))
+    safetensors_files = glob.glob(os.path.join(input_dir, f"*{SUFFIX}"))
 
     # Read metadata from all input files
     input_files_data: dict[str, _InputFileData] = {}
