-
Notifications
You must be signed in to change notification settings - Fork 66
⚠️ OPRUN-4075: Move to a helm-based configuration #2145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
✅ Deploy Preview for olmv1 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2145 +/- ##
==========================================
- Coverage 72.79% 72.64% -0.16%
==========================================
Files 79 79
Lines 7340 7347 +7
==========================================
- Hits 5343 5337 -6
- Misses 1651 1662 +11
- Partials 346 348 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
@@ -24,7 +24,7 @@ To enable the Helm Chart support feature gate, you need to patch the `operator-c | |||
2. **Wait for the controller manager pods to be ready:** | |||
|
|||
```bash | |||
$ kubectl -n olmv1-system wait --for condition=ready pods -l control-plane=operator-controller-controller-manager | |||
$ kubectl -n olmv1-system wait --for condition=ready pods -l apps.kubernetes.io/name=operator-controller |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you added it in the RFC, but could it not be a breaking change?
Should we keep both?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's a breaking change, it's only in our draft documentation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean, people could teorically have scripts and etc to filter things with control-plane=operator-controller-controller-manager
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
theoretically... but I think our user base is very small and OCP focused. Also, I don't think we consider our YAML a published API.
7beca6e
to
70474d0
Compare
hack/test/install-prometheus.sh
Outdated
|
||
TMPDIR="$(mktemp -d)" | ||
trap 'echo "Cleaning up $TMPDIR"; rm -rf "$TMPDIR"' EXIT | ||
#trap 'echo "Cleaning up $TMPDIR"; rm -rf "$TMPDIR"' EXIT |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should it be really commented?
helm/olmv1/Chart.yaml
Outdated
# incremented each time you make changes to the application. Versions are not expected to | ||
# follow Semantic Versioning. They should reflect the version the application is using. | ||
# It is recommended to use it with quotes. | ||
appVersion: "1.3.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is interesting, how we update this value when we do a new release?
We might want a github action that will open a PR for we do not forget that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure how we update this, or if we even care? Because it's the commit we'd want the 1.x
tag to be on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since it's optional, I removed it.
olm.operatorframework.io/feature-set: experimental | ||
name: operator-controller-controller-manager | ||
namespace: olmv1-system | ||
olm.operatorframework.io/feature-set: experimental-e2e |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems that we are indeed fixing it 👍
app.kubernetes.io/name: e2e | ||
app.kubernetes.io/part-of: olm | ||
name: e2e-registries-conf | ||
namespace: olmv1-system |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool it is showing up only in the e2e ones 👍
6a6f688
to
7f7fab8
Compare
This does not remove the kustomize config, but instead puts a helm chart into the repo, that should give very close (but not identical) results. * Adds a new chart: helm/olmv1/ - standard - experimental - openshift - cert-manager - e2e - tilt * Adds "values" files in helm/ * Adds helm executable to .bingo/ * Updates documents int docs/drafts/ * Update tests in tests/ * Update `make manifests` to use helm chart - Update the checked-in manifests - Use a tool like `dyff` to properly diff the manifests * Pull RBAC and WebHook config out of the goland code - controller-tools is not longer used to generate RBAC/Wehbooks - These resources are not part of the helm chart - The CRDs are still generated via kubebuilder Significant changes to the resulting manifests are listed in the RFC. Signed-off-by: Todd Short <tshort@redhat.com> Assisted-by: Gemini (research) Assisted-by: Claude (analysis)
Signed-off-by: Todd Short <tshort@redhat.com> Assisted-by: Gemini (research) Assisted-by: Claude (analysis)
Signed-off-by: Todd Short <tshort@redhat.com> Assisted-by: Gemini (research) Assisted-by: Claude (analysis)
This is currently separate due to the ordering of application. If we change the order, this could be included in the main Helm Chart. Signed-off-by: Todd Short <tshort@redhat.com> Assisted-by: Gemini (research) Assisted-by: Claude (analysis)
From Claude Code: ● Helm Charts Analysis This repository contains 3 Helm charts for the OLMv1 (Operator Lifecycle Manager v1) project:
Security Best Practices Found: ✅ Strong Security Posture:
✅ Operational Best Practices:
The charts follow Kubernetes and Helm best practices with a strong focus on security and operational excellence. |
Recommend the use of https://github.com/homeport/dyff to evaluate the manifest YAML differences. |
algorithm: ECDSA | ||
rotationPolicy: Always | ||
size: 256 | ||
secretName: operator-controller-cert |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tmshort we changed the name, it was before olmv1-cert
But I think it is fine, downstream would we need to do something when upgrade to delete the old secret?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an upstream-only (i.e. cert-manager). Downstream, we use the service-ca.
@@ -1,20 +1,122 @@ | |||
--- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Resource Type | File: standard.yaml |
File: standard-helm.yaml |
Difference | Review & Considerations |
---|---|---|---|---|
Namespace | Simple namespace manifest. | Namespace manifest is templated with additional labels and security annotations. | Templated with additional metadata. | ✅ the additional shows fine |
ServiceAccount | Only one ServiceAccount has labels. | Both ServiceAccounts have app.kubernetes.io/name and app.kubernetes.io/part-of labels. |
More consistent and descriptive labeling. | ✅ |
Service Selector | Uses control-plane: <name> for the pod selector. |
Uses app.kubernetes.io/name: <name> for the pod selector. |
Uses a different labeling convention. | ✅ I am little concern because technically it is a breaking change but we discussed it here #2145 (comment) and I agree with |
Deployment Selector | Uses control-plane: <name> in the selector . |
Uses app.kubernetes.io/name: <name> in the selector . |
Uses a different labeling convention. | ✅ - Same of #2145 (comment) |
Deployment Args | Uses $(POD_NAMESPACE) variable for the external-address argument. |
Uses a hardcoded catalogd-service.olmv1-system.svc for the external-address argument. |
Hardcoded value instead of a variable. | ✅ That is fine we get from the values: --external-address=catalogd-service.{{ .Values.namespaces.olmv1.name }}.svc |
Volumes | Uses secret name olmv1-cert for operator controller certificates. |
Uses secret name operator-controller-cert for operator controller certificates. |
Different naming convention for secrets. | ✅ It seems fine; raised here: https://github.com/operator-framework/operator-controller/pull/2145/files#r2301592489 |
@tmshort look the delta ALL shows fine. 👍
@@ -11,10 +11,6 @@ import ( | |||
ocv1 "github.com/operator-framework/operator-controller/api/v1" | |||
) | |||
|
|||
// +kubebuilder:webhook:admissionReviewVersions={v1},failurePolicy=Fail,groups=olm.operatorframework.io,mutating=true,name=inject-metadata-name.olm.operatorframework.io,path=/mutate-olm-operatorframework-io-v1-clustercatalog,resources=clustercatalogs,verbs=create;update,versions=v1,sideEffects=None,timeoutSeconds=10 | |||
|
|||
// +kubebuilder:rbac:groups=olm.operatorframework.io,resources=clustercatalogs,verbs=get;list;watch;patch;update |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason is described in the RFC
It would be amazing if we could keep but the thread off seems make sense 👍
@@ -88,7 +88,7 @@ var prometheuSpec = allowedPolicyDefinition{ | |||
// Ref: https://docs.google.com/document/d/1bHEEWzA65u-kjJFQRUY1iBuMIIM1HbPy4MeDLX4NI3o/edit?usp=sharing | |||
var allowedNetworkPolicies = map[string]allowedPolicyDefinition{ | |||
"catalogd-controller-manager": { | |||
selector: metav1.LabelSelector{MatchLabels: map[string]string{"control-plane": "catalogd-controller-manager"}}, | |||
selector: metav1.LabelSelector{MatchLabels: map[string]string{"app.kubernetes.io/name": "catalogd"}}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All labels that we have now are:
app.kubernetes.io/name: olmv1
app.kubernetes.io/part-of: olm
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/enforce-version: latest
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/audit-version: latest
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/warn-version: latest
# This is the chart version. This version number should be incremented each time you make changes | ||
# to the chart and its templates, including the app version. | ||
# Versions are expected to follow Semantic Versioning (https://semver.org/) | ||
version: 0.1.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should match the version with the release version and have a GitHub action that automatically opens a PR to update it when we do a release for now. In the future, we will need to bump before release. Do we not plan to distribute the chart instead of the shell?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the chart version, which is supposed to be updated with every chart update, it is not necessarily supposed to match the version of the product. Then again, I've also seen cases where no one updates this...
But we are going to continue with a single YAML+shell script installation via goreleaser, and not distribute the chart, yet.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review – Aug 26
I reviewed the diff and everything looks fine overall. 🥇 The only changes that caught my attention were:
- The selector change: https://github.com/operator-framework/operator-controller/pull/2145/files#r2272575728
- The certs name change: https://github.com/operator-framework/operator-controller/pull/2145/files#r2301592489
See: #2145 (comment)
IMO, the only things missing are:
- The Chart version should match the release version (we need to discuss how to keep this updated).
- Add a GitHub Action to run helm lint so that we can validate the Helm chart.
- Optinally: Could we also test out helm install in a GitHub Action for now, just to ensure the chart is deployable? This would be a good stopgap until we start distributing the solution via Helm and use it to run our e2e tests.
I commented on both of these. We moved to a "standard k8s" selector, but we can always put back the original if there are complaints.
The chart version is the version of the chart itself, and is supposed to be incremented whenever the chart is modified. It's not necessarily supposed to match the software release. The field that corresponds the the release (
We don't actually use the field when rendering.
Not sure we need to add a new CI, we can do it as part of
This is a bit out-of-scope now. We are only planning to template/render it. We do use the rendered manifests for e2e testing. |
Signed-off-by: Todd Short <tshort@redhat.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great job 🚀
LGTM
I’ll hold for now, but feel free to unhold whenever you think it’s appropriate.
I believe it’s better to wait until the cut branch is created before merging this one,
so we avoid impacting any changes that should go into version 4.20.
/hold
(until we see that is fine get merged)
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: camilamacedo86 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This does not remove the kustomize config, but instead puts a helm chart into the repo, that should give very close (but not identical) results.
make manifests
to use helm chartdyff
to properly diff the manifestsSignificant changes to the resulting manifests are listed in the RFC.
Assisted-by: Gemini (research)
Assisted-by: Claude (analysis)
Description
Reviewer Checklist