Delete HTML build sources to save on artefact upload time #14629
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Does this actually make a difference? I would have assumed the biggest contributor to the size was the images. |
|
I think the bottleneck isn't the size of the files, but the number of files as each is uploaded individually. Looking at a handful of builds, previously artifact upload was ~15 mins, with this PR it goes down to ~ 10 mins, so ~ 5 min/build time saved. |
|
Yes, that's right, files are uploaded individually. And especially if they're small, this causes a significant overhead (if there were a way to upload a tarball and explode it on S3, we'd probably save 8-9 minutes of the remaining 10.) |
8d2f719 to
1a4d691
Compare
jklymak
approved these changes
Jul 15, 2019
timhoffm
approved these changes
Jul 15, 2019
|
Well, lets merge and if the sources were somehow needed, we will find out soon enough and can revert this. None of this is user facing anyhow... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently uploading the doc build artefacts takes circleCI a long time because it uploads every single file individually. I don't think the files in
build/html/_sourcesare needed for viewing the generated HTML docs though, so try deleting them to save time on the artefact upload.