Skip to content

migrate pkg/server/dynamiccertificates to contextual logging #126002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

liyuerich
Copy link
Contributor

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

migrate pkg/server/dynamiccertificates to contextual logging

Which issue(s) this PR fixes:

Ref # kubernetes/enhancements#3077

Special notes for your reviewer:

NONE

Does this PR introduce a user-facing change?

migrate pkg/server/dynamiccertificates to contextual logging

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://github.com/kubernetes/enhancements/tree/42bb993e369d166142b7181e90882066ad6c7651/keps/sig-instrumentation/3077-contextual-logging

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/apiserver sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jul 10, 2024
@k8s-ci-robot k8s-ci-robot requested review from apelisse and ncdc July 10, 2024 11:56
@liyuerich
Copy link
Contributor Author

/release-note-none

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jul 10, 2024
@liyuerich
Copy link
Contributor Author

/retest

@liyuerich
Copy link
Contributor Author

/cc @mengjiao-liu @carlory

@cici37
Copy link
Contributor

cici37 commented Jul 11, 2024

/wg structured-logging
/remove-sig api-machinery

@k8s-ci-robot k8s-ci-robot added wg/structured-logging Categorizes an issue or PR as relevant to WG Structured Logging. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. and removed sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. labels Jul 11, 2024
@liyuerich
Copy link
Contributor Author

there is error as below message, it seems there is no related with PR changes. Do you think I should fix this error in the same PR, or submit a new PR to handle it?

@mengjiao-liu , what is your suggestion?

staging/src/k8s.io/apiserver/pkg/server/dynamiccertificates/tlsconfig.go:214:3: SA1019: newTLSConfigCopy.NameToCertificate has been deprecated since Go 1.14: NameToCertificate only allows associating a single certificate with a given name. Leave this field nil to let the library select the first compatible chain from Certificates. (staticcheck)
ERROR: newTLSConfigCopy.NameToCertificate, err = c.BuildNamedCertificates(ctx, newContent.sniCerts)

@mengjiao-liu
Copy link
Member

there is error as below message, it seems there is no related with PR changes. Do you think I should fix this error in the same PR, or submit a new PR to handle it?

@mengjiao-liu , what is your suggestion?

Because the changes are related to this PR, you can commit two different commits in this PR for the static check and log migration sections. If you feel that the content of the change is too much, you can submit a separate PR.

@liyuerich
Copy link
Contributor Author

there is error as below message, it seems there is no related with PR changes. Do you think I should fix this error in the same PR, or submit a new PR to handle it?

@mengjiao-liu , what is your suggestion?

Because the changes are related to this PR, you can commit two different commits in this PR for the static check and log migration sections. If you feel that the content of the change is too much, you can submit a separate PR.

thanks @mengjiao-liu , will submit a separate PR to handle this error.

@liyuerich
Copy link
Contributor Author

After investigation and check previous PR about deprecated "newTLSConfigCopy.NameToCertificate", it is fundamental function, should be very carefully to remove. and I have no idea to fix it til now.

I have to get the help/consultant/feedback from @liggitt @deads2k and @jackkleeman , who worked on related PR about certificate before.

Do you think it is ok to leave this lint check or I have to remove the deprecated method?

Thanks your feedback and any suggestion.

@liyuerich
Copy link
Contributor Author

liyuerich commented Aug 27, 2024

After investigation and check previous PR about deprecated "newTLSConfigCopy.NameToCertificate", it is fundamental function, should be very carefully to remove. and I have no idea to fix it til now.

I have to get the help/consultant/feedback from @liggitt @deads2k and @jackkleeman , who worked on related PR about certificate before.

Do you think it is ok to leave this lint check or I have to remove the deprecated method?

Thanks your feedback and any suggestion.

add the related PR here:
#83627 Plumb dynamic SNI certificates
#85308 allow an SNI cert to be used to respond for a particular IP

/cc @liggitt @deads2k @jackkleeman for your reference.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 27, 2024
@liyuerich
Copy link
Contributor Author

liyuerich commented Sep 9, 2024

After investigation and check previous PR about deprecated "newTLSConfigCopy.NameToCertificate", it is fundamental function, should be very carefully to remove. and I have no idea to fix it til now.
I have to get the help/consultant/feedback from @liggitt @deads2k and @jackkleeman , who worked on related PR about certificate before.
Do you think it is ok to leave this lint check or I have to remove the deprecated method?
Thanks your feedback and any suggestion.

add the related PR here: #83627 Plumb dynamic SNI certificates #85308 allow an SNI cert to be used to respond for a particular IP

/cc @liggitt @deads2k @jackkleeman for your reference.

just got the feedback from Jack, he might have no idea on my question. thanks Jack.

@liggitt , @deads2k how about your suggestion?

Copy link
Member

@mengjiao-liu mengjiao-liu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 13, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 365855ce18d311fca578f324df9ca27238780460

@dims dims removed the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Dec 16, 2024
@liyuerich
Copy link
Contributor Author

/assign @jpbetz , @smarterclayton , @tallclair please help to approve, thanks.

@liyuerich
Copy link
Contributor Author

/assign @jpbetz , @smarterclayton , @tallclair please help to approve, thanks.

any comment?

@liyuerich
Copy link
Contributor Author

/assign @jpbetz , @smarterclayton , @tallclair please help to approve, thanks.

any comment?

any feedback?

Signed-off-by: liyuerich <yue.li@daocloud.io>
@liyuerich liyuerich force-pushed the dynamiccertificates-support-contextual-logging branch from 01bc2e8 to 8d1993f Compare February 24, 2025 06:15
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 24, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: liyuerich
Once this PR has been reviewed and has the lgtm label, please assign jpbetz, smarterclayton, tallclair for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Feb 24, 2025

@liyuerich: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubernetes-apidiff 45dfcff link false /test pull-kubernetes-apidiff

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@liyuerich
Copy link
Contributor Author

/retest-required

1 similar comment
@liyuerich
Copy link
Contributor Author

/retest-required

@carlory
Copy link
Member

carlory commented Feb 25, 2025

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 25, 2025
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 0fc9f0e76a0cf6026ba2f4d1c7dd30d9affbb05a

@liyuerich
Copy link
Contributor Author

hi @jpbetz , @smarterclayton , @tallclair is it ok to help to approve and merge now? thanks.

@liyuerich
Copy link
Contributor Author

/assign @jpbetz @smarterclayton @tallclair to approve

sorry , have to send this message again.

@liyuerich
Copy link
Contributor Author

/test pull-kubernetes-cmd

@liyuerich
Copy link
Contributor Author

seems all checks are pass, @jpbetz , @smarterclayton , @tallclair is it ok to approve and merge now?

also /cc @mengjiao-liu @pohly , any comment or suggestion?

@mengjiao-liu
Copy link
Member

seems all checks are pass, @jpbetz , @smarterclayton , @tallclair is it ok to approve and merge now?

also /cc @mengjiao-liu @pohly , any comment or suggestion?

LGTM from my side.

@@ -74,7 +74,7 @@ type caBundleAndVerifier struct {
}

// NewDynamicCAContentFromFile returns a CAContentProvider based on a filename that automatically reloads content
func NewDynamicCAContentFromFile(purpose, filename string) (*DynamicFileCAContent, error) {
func NewDynamicCAContentFromFile(ctx context.Context, purpose, filename string) (*DynamicFileCAContent, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see usages for this function and NewDynamicServingContentFromFiles outside just apiserver. (kubelet and cloud-provider). Do we need an additional NewDynamicCAContentFromFileWithContext function to provide backwards compatibility?

Though It's possible this comment applies #126002 (comment) for these controllers as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it is used outside, will add new one.

klog.Warningf("Initial population of dynamic certificates failed: %v", err)
}
go dynamicCertificateController.Run(1, stopCh)
go dynamicCertificateController.Run(ctx, 1, stopCh)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's really confusing to pass ctx and stopCh in, when ctx also cancels when stopCh is closed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will fix soon. thanks

@@ -166,7 +166,7 @@ func Run(ctx context.Context, opts options.CompletedOptions) error {
}

// CreateServerChain creates the apiservers connected via delegation.
func CreateServerChain(config CompletedConfig) (*aggregatorapiserver.APIAggregator, error) {
func CreateServerChain(ctx context.Context, config CompletedConfig) (*aggregatorapiserver.APIAggregator, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to make it clear if the provided context is retained and controls lifecycle of things beyond this function invocation or not (this comment holds for most of the changes)

It's really hard to reason about what context should be passed in here depending on whether it is just used for logging during the function invocation, or if the cancellation of the context after the function returns does something

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by drill down CreateServerChain, and finally in loadCertKeyPair, you could find:

c.certKeyPair.Store(newCertKey)
logger := klog.FromContext(ctx)
logger.V(2).Info("Loaded a new cert/key pair", "name", c.Name())

that is why a ctx is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/apiserver area/cloudprovider area/kubelet area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note-none Denotes a PR that doesn't merit a release note. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. sig/etcd Categorizes an issue or PR as relevant to SIG Etcd. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on. wg/structured-logging Categorizes an issue or PR as relevant to WG Structured Logging.
Projects
Status: Needs Triage
Status: In Review
Archived in project
Status: other-sig (sig-node-approved)
Development

Successfully merging this pull request may close these issues.