CLOUDP-295785 - using build_info.json in new atomic pipeline #317
Conversation
Fix build scenario Remove create and push manifests Continue improvement to main Simplify main and build_context missed Pass Build Configuration object directly Use legacy and new pipeline Fix Remove --include Rename MCO test image Multi platform builds, with buildx TODOs Implement is_release_step_executed() Fix init appdb image Import sort black formatting Some cleaning and version adjustments Adapt main to new build config Add buildscenario to buildconfig Handle build env Renaming, usage of high level config All images build pass on EVG Lint Explicit image type, support custom build_path Replace old by new pipeline in EVG Add documentation Split in multiple files, cleanup WIP, passing builds on staging temp + multi arch manifests Replace usage of sonar Remove namespace Remove pin_at and build_id Copied pipeline, removed daily builds and --exclude
This reverts commit 426e522.
This reverts commit d2a6153.
This reverts commit c6fc163.
…d-versions # Conflicts: # scripts/release/atomic_pipeline.py # scripts/release/build/image_build_process.py # scripts/release/build_context.py
…d-versions # Conflicts: # scripts/release/atomic_pipeline.py # scripts/release/build/image_build_process.py # scripts/release/pipeline_main.py
| @@ -6,9 +6,7 @@ | |||
| # | |||
| # Ref: https://cryptography.io/en/latest/installation/#building-cryptography-on-linux | |||
| # | |||
| ARG GOLANG_VERSION | |||
There was a problem hiding this comment.
to align with other Dockerfiles, where Golang version is specified in base image
|
|
||
| expected_build_info = BuildInfo( | ||
| images={ | ||
| "mongodbOperator": ImageInfo( | ||
| "operator": ImageInfo( |
There was a problem hiding this comment.
did you change the name to match what you've defined in build_info? was it generally a unification effort to extract all into shared const?
There was a problem hiding this comment.
should we use the const here instead of strings?
There was a problem hiding this comment.
did you change the name to match what you've defined in build_info? was it generally a unification effort to extract all into shared const?
yes, I'm using for examplescripts.release.build.build_info.OPERATOR_IMAGEconst vars
this is only the unit test. I can use constants or for better verbosity use hardcoded strings.
| elif is_patch or is_evg: | ||
| scenario = BuildScenario.PATCH | ||
| logger.info(f"Build scenario: {scenario} (patch_id: {patch_id})") | ||
| # TODO: Uncomment the following lines when starting to work on staging builds |
There was a problem hiding this comment.
but this is unrelated to patches, right? Its only for master merges?
There was a problem hiding this comment.
yes, this is to not break the master merges. The next PR will target staging builds particularly
| DEVELOPMENT = "development" # Local build on a developer machine | ||
|
|
||
| @classmethod | ||
| def infer_scenario_from_environment(cls) -> "BuildScenario": |
There was a problem hiding this comment.
i know we have a ticket for this, but wanted to say this again - such high important infer scenarios - we should have metrics for them
There was a problem hiding this comment.
I would also like to provide scenario and version from the outside of the script. This is on my list and we will do this in Atomic Releases definitely - I've added it to our tasks
| """ | ||
| Builds image used to run tests. | ||
| """ | ||
|
|
There was a problem hiding this comment.
so we don't need the image_name here in the function anymore since we tightly couple this (and other) functions to a specific build_configuration which gets its information from build_info , right?
There was a problem hiding this comment.
yes, we will move most of the configuration to build_info.json, including Dockerfile location and docker args
| def build_om_image(build_configuration: ImageBuildConfiguration): | ||
| # Make this a parameter for the Evergreen build | ||
| # https://github.com/evergreen-ci/evergreen/wiki/Parameterized-Builds | ||
| om_version = os.environ.get("om_version") |
There was a problem hiding this comment.
can we somehow extract this out of the pipeline?
There was a problem hiding this comment.
yes! All of the versions should be provided from outside, not from within the pipeline. This will be done in subsequent iterations and is added to our task list.
# Conflicts: # scripts/release/atomic_pipeline.py # scripts/release/pipeline_main.py
# Conflicts: # scripts/release/atomic_pipeline.py
Summary
This PR focuses on using
build_info.jsoninformation inatomic_pipeline.pycode. The logic is mostly hidden inimage_build_config_from_argsfunction. It is binding the parts from:build_info.jsonwith information for building imagesSection in Atomic Releases TD that describes the
build_info.jsonand how it is used in the image building process.build_info.jsonmeko-testsandmco-testsimages inbuild_info.jsonagentandops-managerimages inbuild_info.jsonimages_signing.pyfromscripts/evergreen/releaseProof of Work
CI tests passing + updated unit tests
Checklist
skip-changeloglabel if not needed