Skip to content

WIP: client-go: finish context support #129125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

pohly
Copy link
Contributor

@pohly pohly commented Dec 9, 2024

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

The main purpose is to replace context.TODO with a context provided by the caller. A secondary purpose is to enable contextual logging.

Special notes for your reviewer:

See #129109 for an introduction of the approach taken most of the time when adding alternative context-aware APIs. See commit messages for additional information.

Most likely this will be merged by creating smaller PRs.

Does this PR introduce a user-facing change?

client-go: all context.TODO calls got removed by introducing new APIs where the caller passes in the context.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. labels Dec 9, 2024
@k8s-ci-robot
Copy link
Contributor

Please note that we're already in Test Freeze for the release-1.32 branch. This means every merged PR will be automatically fast-forwarded via the periodic ci-fast-forward job to the release branch of the upcoming v1.32.0 release.

Fast forwards are scheduled to happen every 6 hours, whereas the most recent run was: Mon Dec 9 12:10:29 UTC 2024.

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 9, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/apiserver area/code-generation area/dependency Issues or PRs related to dependency changes area/test sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/testing Categorizes an issue or PR as relevant to SIG Testing. wg/device-management Categorizes an issue or PR as relevant to WG Device Management. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Dec 9, 2024
@k8s-ci-robot k8s-ci-robot requested review from alexzielenski, aojea and a team December 9, 2024 17:10
@pohly pohly force-pushed the log-client-go-tools-apis branch 2 times, most recently from 096f284 to 49e218c Compare December 10, 2024 10:27
@k8s-ci-robot k8s-ci-robot added area/cloudprovider sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. labels Dec 10, 2024
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 7, 2025
I wasn't entirely sure whether this should return a value or a pointer to
satisfy the interface. Both works, so I benchmarked it elsewhere (REST
mapper). Mem allocs are the same (one alloc/call), but returning a value is 10%
slower when calling one method.

What I then benchmarked is whether pointer vs value receiver in the wrapper
makes a difference. Converting from value receiver (what I had before) to
pointer receiver reduced call overhead by 6%. That's because with a value
receiver, Go has to auto-generate a variant with pointer receiver and calls the
value receiver through that.

That can be seen in a debugger (call stack) and when setting breakpoints:

    (dlv) b restMapperWrapper.KindForWithContext
    Command failed: Location "restMapperWrapper.KindForWithContext" ambiguous: k8s.io/apimachinery/pkg/api/meta.restMapperWrapper.KindForWithContext, k8s.io/apimachinery/pkg/api/meta.(*restMapperWrapper).KindForWithContext…

Conventional wisdom is to define types with value receiver because those can be
called also on unmutable instances, making them more flexible.

But for types which will only ever be used via a pointer, I think pointer
receiver is better for the reasons above (small performance difference, easier
to debug).
The main purpose is to replace context.TODO with a context provided by the
caller. A secondary purpose is to enable contextual logging.

Modifying the existing interfaces and APIs would have a big impact on the
ecosystem. This is a no-go. Instead, the following approach was taken:

- All interfaces get duplicated in a *WithContext variant where the methods
  also have a *WithContext suffix and the ctx parameter. All methods are
  treated this way except for obvious local get methods (like RESTClient)
  because it cannot be ruled out entirely that some implementation may
  need a context.

- Implementations of these interfaces implement both method variants
  which is possible because the method names are different.
  The old methods are implemented as thin wrappers around the updated
  code which is now the body of the new methods or shared helpers.
  In some cases there is additional overhead (type checks, potentially
  additional allocations) when using the old methods.

- To*WithContext helpers bridge from the old to the new interfaces. They
  try a type cast first. Because the in-tree implementations implement
  both, they can be used directly. For other implementations wrappers
  are used.

- All old APIs and interfaces are marked as deprecated. There is no
  intent to ever remove them, but consumers should be made aware
  that there are now better alternatives. Implementations also
  get marked this way even if nothing ever calls them directly
  because it shows which code, at least theoretically, could get
  removed.

- Existing unit tests do not get updated to the new APIs. This gives
  us unit test coverage of the old and new API because the old
  APIs call the new ones.

- In-tree consumers will be updated in follow-up PRs. This is likely
  to be a longer process. Because of the deprecation comment,
  `hack/golangci-lint.sh -n` can be used to find code which needs
  to be updated.
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 13, 2025
pohly added 11 commits February 17, 2025 10:29
The only log output is for error messages which should normally not occur. It's
also likely that users expect to see exactly those messages, so it's better to
not touch them.
When debugging, it helps to keep output from different connections
separate. This can be done with contextual logging and using different loggers
for each connection.

Cancellation is handled separately for requests. Therefore the new APIs only
add support for passing a logger instance.
The API for the package already had a context, so all that was missing was to
extract and use the logger from that.
The remaining non-structured, non-contextual log calls hopefully never get
reached, so it's not worth converting them.
This changes the log output of kubectl, which affects some test cases which
check for GET results in that log output.
@pohly pohly force-pushed the log-client-go-tools-apis branch from 9f7af59 to 4f211d5 Compare April 16, 2025 10:42
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 16, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: pohly
Once this PR has been reviewed and has the lgtm label, please assign liggitt for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot
Copy link
Contributor

@pohly: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubernetes-apidiff 1bf1482 link false /test pull-kubernetes-apidiff
pull-kubernetes-linter-hints 4f211d5 link false /test pull-kubernetes-linter-hints
pull-kubernetes-e2e-kind-ipv6 4f211d5 link true /test pull-kubernetes-e2e-kind-ipv6
pull-kubernetes-verify 4f211d5 link true /test pull-kubernetes-verify
pull-kubernetes-e2e-capz-azure-file-vmss 4f211d5 link false /test pull-kubernetes-e2e-capz-azure-file-vmss
pull-kubernetes-e2e-capz-azure-file 4f211d5 link false /test pull-kubernetes-e2e-capz-azure-file
pull-kubernetes-e2e-capz-azure-disk-vmss 4f211d5 link false /test pull-kubernetes-e2e-capz-azure-disk-vmss

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 3, 2025
@k8s-ci-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@dims dims added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 12, 2025
@bart0sh bart0sh moved this from Triage to Work in progress in SIG Node: code and documentation PRs Jul 14, 2025
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle rotten
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Aug 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/apiserver area/cloudprovider area/code-generation area/dependency Issues or PRs related to dependency changes area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/contains-merge-commits Indicates a PR which contains merge commits. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/auth Categorizes an issue or PR as relevant to SIG Auth. sig/cli Categorizes an issue or PR as relevant to SIG CLI. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. sig/instrumentation Categorizes an issue or PR as relevant to SIG Instrumentation. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. wg/device-management Categorizes an issue or PR as relevant to WG Device Management. wg/structured-logging Categorizes an issue or PR as relevant to WG Structured Logging.
Projects
Archived in project
Status: Needs Triage
Status: Needs Triage
Status: Needs Triage
Archived in project
Development

Successfully merging this pull request may close these issues.

7 participants