I'm not well versed with Composer at all, but would it be possible to switch to one of Prestissimo's async file downloaders if they are available? On my computer the scaffolding process takes 16sec. For reference, using prestissimo when downloading drupal-composer/drupal-project dependency packages.json files decreased the build step by 235sec.