Skip to content

CLOUDP-295785 - using build_info.json in new atomic pipeline #317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 89 commits into
base: master
Choose a base branch
from

Conversation

MaciejKaras
Copy link
Collaborator

@MaciejKaras MaciejKaras commented Aug 6, 2025

Summary

This PR focuses on using build_info.json information in atomic_pipeline.py code. The logic is mostly hidden in image_build_config_from_args function. It is binding the parts from:

Section in Atomic Releases TD that describes the build_info.json and how it is used in the image building process.

  • Added signing configuration in build_info.json
  • Added missing meko-tests and mco-tests images in build_info.json
  • Added configuration for agent and ops-manager images in build_info.json
  • Moved images_signing.py from scripts/evergreen/release
  • Simplifications in the existing code

Proof of Work

CI tests passing + updated unit tests

Checklist

  • Have you linked a jira ticket and/or is the ticket in the title?
  • Have you checked whether your jira ticket required DOCSP changes?
  • Have you added changelog file?

Fix build scenario

Remove create and push manifests

Continue improvement to main

Simplify main and build_context

missed

Pass Build Configuration object directly

Use legacy and new pipeline

Fix

Remove --include

Rename MCO test image

Multi platform builds, with buildx

TODOs

Implement is_release_step_executed()

Fix init appdb image

Import sort

black formatting

Some cleaning and version adjustments

Adapt main to new build config

Add buildscenario to buildconfig

Handle build env

Renaming, usage of high level config

All images build pass on EVG

Lint

Explicit image type, support custom build_path

Replace old by new pipeline in EVG

Add documentation

Split in multiple files, cleanup

WIP, passing builds on staging temp + multi arch manifests

Replace usage of sonar

Remove namespace

Remove pin_at and build_id

Copied pipeline, removed daily builds and --exclude
This reverts commit 426e522.
MaciejKaras and others added 3 commits August 8, 2025 15:27
…d-versions

# Conflicts:
#	scripts/release/atomic_pipeline.py
#	scripts/release/build/image_build_process.py
#	scripts/release/build_context.py
…d-versions

# Conflicts:
#	scripts/release/atomic_pipeline.py
#	scripts/release/build/image_build_process.py
#	scripts/release/pipeline_main.py
@MaciejKaras MaciejKaras marked this pull request as ready for review August 8, 2025 14:18
@MaciejKaras MaciejKaras requested a review from a team as a code owner August 8, 2025 14:18
@MaciejKaras MaciejKaras requested review from anandsyncs, Julien-Ben, nammn, lucian-tosa and viveksinghggits and removed request for a team and anandsyncs August 8, 2025 14:18
Base automatically changed from julienben/redesign-pipeline to master August 8, 2025 14:49
@@ -6,9 +6,7 @@
#
# Ref: https://cryptography.io/en/latest/installation/#building-cryptography-on-linux
#
ARG GOLANG_VERSION
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this chagne?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to align with other Dockerfiles, where Golang version is specified in base image

# Conflicts:
#	scripts/release/atomic_pipeline.py
#	scripts/release/pipeline_main.py
@Julien-Ben Julien-Ben mentioned this pull request Aug 11, 2025
@MaciejKaras MaciejKaras requested a review from nammn August 11, 2025 10:28
MaciejKaras pushed a commit that referenced this pull request Aug 11, 2025
# Summary

Master patches are broken because they build with `latest` tag only,
while the patch still need the `patch_id` tagged images.
This PR fixes this issue to get master green again, but remove builds to
`latest`.
They will be added again in
#317, using
build_info.
Not pushing to latest only impacts local testing for dev, as we use them
to have the most recently built versions from master. It won't break
local development, but it means the image used will be slightly
outdated.

## Proof of Work
I tested it by forcing to return the patch ID and dumping the relevant
variables in this patch:

https://spruce.mongodb.com/version/6899d033f94ad4000694cf89/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC

But the relevant environment variable (`is_patch` being false, as
described in the added documentation) will be set to the right value
only when merging to master.
Copy link
Contributor

@viveksinghggits viveksinghggits left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -57,7 +57,7 @@ def load_build_info(scenario: BuildScenario,
f"""
Load build information based on the specified scenario.

:param scenario: BuildScenario enum value indicating the build scenario (e.g., PATCH, STAGING, RELEASE).
:param scenario: BuildScenario enum value indicating the build scenario (e.g. "development", "patch", "staging", "release"). "development" scenario will return build info for "patch" scenario.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: do we really need to introduce a new build scenario development if we are just treating it as patch. Should we consider adding a comment on the method that for development just pass patch?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The development scenario is useful for local development, when you want to publish with latest tag etc. It is synthetic and we could later decide to drop it if we extract the version and scenario out of the atomic_pipeline.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip-changelog Use this label in Pull Request to not require new changelog entry file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants