CLOUDP-295785 - add `pipeline_main.py` extended help #334

MaciejKaras · 2025-08-11T18:39:17Z

Summary

Adding more help information for the scripts.release.pipeline_main python script.

Proof of Work

(venv) ➜  mongodb-kubernetes git:(maciejk/use-ar-related-versions) ✗ python3 -m scripts.release.pipeline_main --help
usage: pipeline_main.py [-h] [-b ] [-p ] [-v ] [-r ] [-s] [--parallel] [--parallel-factor ] image

Builder tool for container images. It allows to push and sign images with multiple architectures using Docker Buildx. By default build information is read from 'build_info.json' file in the project root directory based on the build scenario. It can be inferred from the environment variables or overridden with the '--build-scenario' option.

positional arguments:
  image                 Image name to build. Supported images: meko-tests, operator, mco-tests, readiness-probe, upgrade-hook, database, agent, init-appdb, init-database, init-ops-manager, ops-manager

options:
  -h, --help            show this help message and exit
  -b, --build-scenario
                        Override the build scenario instead of inferring from environment. Options: release, patch, staging, development. Default is to infer from environment variables. For 'development' the 'patch' scenario is used to read values from 'build_info.json'
  -p, --platform        Override the platforms instead of resolving from build scenario. Multi-arch builds are comma-separated. Example: linux/amd64,linux/arm64
  -v, --version         Override the version/tag instead of resolving from build scenario. Default is to infer from environment variables based on the selected scenario.
  -r, --registry        Override the base registry instead of resolving from build scenario
  -s, --sign            If set force image signing. Default is to infer from build scenario.
  --parallel            Build agent images in parallel.
  --parallel-factor     Number of agent builds to run in parallel, defaults to number of cores

Checklist

Have you linked a jira ticket and/or is the ticket in the title?
Have you checked whether your jira ticket required DOCSP changes?
Have you added changelog file?
- use skip-changelog label if not needed
- refer to Changelog files and Release Notes section in CONTRIBUTING.md for more details

Fix build scenario Remove create and push manifests Continue improvement to main Simplify main and build_context missed Pass Build Configuration object directly Use legacy and new pipeline Fix Remove --include Rename MCO test image Multi platform builds, with buildx TODOs Implement is_release_step_executed() Fix init appdb image Import sort black formatting Some cleaning and version adjustments Adapt main to new build config Add buildscenario to buildconfig Handle build env Renaming, usage of high level config All images build pass on EVG Lint Explicit image type, support custom build_path Replace old by new pipeline in EVG Add documentation Split in multiple files, cleanup WIP, passing builds on staging temp + multi arch manifests Replace usage of sonar Remove namespace Remove pin_at and build_id Copied pipeline, removed daily builds and --exclude

This reverts commit 426e522.

…d-versions # Conflicts: # scripts/release/atomic_pipeline.py # scripts/release/build/image_build_process.py # scripts/release/pipeline_main.py

# Conflicts: # scripts/release/atomic_pipeline.py # scripts/release/pipeline_main.py

# Conflicts: # scripts/release/atomic_pipeline.py

# Conflicts: # scripts/release/build_context.py

github-actions · 2025-08-11T18:40:03Z

⚠️ (this preview might not be accurate if the PR is not rebased on current master branch)

MCK 1.3.0 Release Notes

Bug Fixes

This change fixes the current complex and difficult-to-maintain architecture for stateful set containers, which relies on an "agent matrix" to map operator and agent versions which led to a sheer amount of images.
We solve this by shifting to a 3-container setup. This new design eliminates the need for the operator-version/agent-version matrix by adding one additional container containing all required binaries. This architecture maps to what we already do with the mongodb-database container.

Other Changes

Optional permissions for PersistentVolumeClaim moved to a separate role. When managing the operator with Helm it is possible to disable permissions for PersistentVolumeClaim resources by setting operator.enablePVCResize value to false (true by default). When enabled, previously these permissions were part of the primary operator role. With this change, permissions have a separate role.
subresourceEnabled Helm value was removed. This setting used to be true by default and made it possible to exclude subresource permissions from the operator role by specifying false as the value. We are removing this configuration option, making the operator roles always have subresource permissions. This setting was introduced as a temporary solution for this OpenShift issue. The issue has since been resolved and the setting is no longer needed.

# Conflicts: # scripts/release/atomic_pipeline.py # scripts/release/pipeline_main.py

Julien-Ben · 2025-08-12T11:41:09Z

scripts/release/pipeline_main.py

+        type=str,
+        choices=supported_scenarios,
+        help=f"""Override the build scenario instead of inferring from environment. Options: {", ".join(supported_scenarios)}.
+Default is to infer from environment variables. For '{BuildScenario.DEVELOPMENT}' the '{BuildScenario.PATCH}' scenario is used to read values from 'build_info.json'""",


nit: I think

For '{BuildScenario.DEVELOPMENT}' the '{BuildScenario.PATCH}' scenario is used to read values from 'build_info.json

is confusing.

Depending on if we decide to change this behaviour or keep it, we may want to add a sentence to explain why.

I'm still not entirely sure what direction we should go. DEVELOPMENT makes sense, but we rarely build our own images, rather instruct evg to build it for us. We can reuse PATCH configuration to build everything we want by overriding other settings like registry and build scenario

I will add a task to our todo list to clean this up later

Julien-Ben added 30 commits July 29, 2025 17:30

Remove file

675bee4

Put lib back in dependencies

833e25f

add todo

15e7f51

Fix

120c1af

Remove multi arch call, fix test image path

c9ceabf

Fix agent version for default case

fb87f4d

Lindt

c05e180

isort

747c4ba

Cleanup TODOs

03fd9b8

Rename arch -> platform

1fbb8d5

Don't rely on exception to check for builder existence

e9a524f

Remove unused variables

fa6b899

Pre commit

426e522

Cleanup

6890858

Correct build envs

aab9592

Lindt

33173bb

Update Makefile

74e867c

Add TODO

b13b054

Revert "Pre commit"

832ce61

This reverts commit 426e522.

Remove agent unused functions

bc23827

Remove debug

03062c3

Remove args iteration for multi platform

4d4e4e9

Cleanup unused stuff

7c9967a

Cleanup

a7c63c9

Rename file

742e784

Remove cli sbom

1f0a21b

Renamed image building file

813d539

Freeze python on whales

c06061b

Lint

5f9d49a

Julien-Ben and others added 11 commits August 8, 2025 15:41

Remove comment

2125cb7

Merge branch 'julienben/redesign-pipeline' into maciejk/use-ar-relate…

d047de5

…d-versions # Conflicts: # scripts/release/atomic_pipeline.py # scripts/release/build/image_build_process.py # scripts/release/pipeline_main.py

Merge branch 'master' into maciejk/use-ar-related-versions

c8b66fe

# Conflicts: # scripts/release/atomic_pipeline.py # scripts/release/pipeline_main.py

Review fixes

fa9c7ee

Merge branch 'master' into maciejk/use-ar-related-versions

e56591c

# Conflicts: # scripts/release/atomic_pipeline.py

Added comment to get_version_id() method

597edf2

Revert parallel max_workers = 1

857f705

Agent image fix

59e4420

Merge branch 'master' into maciejk/use-ar-related-versions

2eebf0f

Merge branch 'master' into maciejk/use-ar-related-versions

bec8626

# Conflicts: # scripts/release/build_context.py

CLOUDP-295785 - add pipeline_main.py extended help

22f1330

MaciejKaras changed the base branch from master to maciejk/ar-use-calculate-version August 11, 2025 18:40

MaciejKaras changed the base branch from maciejk/ar-use-calculate-version to maciejk/use-ar-related-versions August 11, 2025 18:41

MaciejKaras changed the title ~~CLOUDP-295785 - add pipeline_main.py extended help~~ CLOUDP-295785 - add pipeline_main.py extended help Aug 11, 2025

MaciejKaras added skip-changelog Use this label in Pull Request to not require new changelog entry file and removed skip-changelog Use this label in Pull Request to not require new changelog entry file labels Aug 11, 2025

MaciejKaras marked this pull request as ready for review August 11, 2025 18:55

MaciejKaras requested a review from a team as a code owner August 11, 2025 18:55

MaciejKaras requested review from fealebenpae and lucian-tosa and removed request for a team August 11, 2025 18:55

Base automatically changed from maciejk/use-ar-related-versions to master August 12, 2025 10:58

Merge branch 'master' into maciejk/ar-pipeline-help

cfd66b6

# Conflicts: # scripts/release/atomic_pipeline.py # scripts/release/pipeline_main.py

nammn approved these changes Aug 12, 2025

View reviewed changes

Julien-Ben reviewed Aug 12, 2025

View reviewed changes

Julien-Ben approved these changes Aug 12, 2025

View reviewed changes

MaciejKaras merged commit 3d0c8e7 into master Aug 12, 2025
35 of 38 checks passed

MaciejKaras deleted the maciejk/ar-pipeline-help branch August 12, 2025 13:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CLOUDP-295785 - add `pipeline_main.py` extended help #334

CLOUDP-295785 - add `pipeline_main.py` extended help #334

MaciejKaras commented Aug 11, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Aug 11, 2025

Uh oh!

Julien-Ben Aug 12, 2025

Uh oh!

MaciejKaras Aug 12, 2025

Uh oh!

MaciejKaras Aug 12, 2025

Uh oh!

Uh oh!

Uh oh!

CLOUDP-295785 - add pipeline_main.py extended help #334

CLOUDP-295785 - add pipeline_main.py extended help #334

Conversation

MaciejKaras commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Proof of Work

Checklist

Uh oh!

github-actions bot commented Aug 11, 2025

MCK 1.3.0 Release Notes

Bug Fixes

Other Changes

Uh oh!

Julien-Ben Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

MaciejKaras Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

MaciejKaras Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

CLOUDP-295785 - add `pipeline_main.py` extended help #334

CLOUDP-295785 - add `pipeline_main.py` extended help #334

MaciejKaras commented Aug 11, 2025 •

edited

Loading