Skip to content

ktesting: abort entire test suite on SIGINT + /dev/tty #128606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

pohly
Copy link
Contributor

@pohly pohly commented Nov 6, 2024

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

When aborting an integration test with CTRL-C while it runs, the current test fails and etcd exits. But additional tests were still being started and the failed slowly because they couldn't connect to etcd.

It's better to fail additional tests in ktesting.Init when the test run has already been interrupted.

While at it, also make it a bit more obvious that testing was interrupted by logging it and update one comment about this.

Example:

$ go test -v ./test/integration/quota
...
I1106 11:42:48.857162  147325 etcd.go:416] "Not using watch cache" resource="events.events.k8s.io"
I1106 11:42:48.857204  147325 handler.go:286] Adding GroupVersion events.k8s.io v1 to ResourceManager
W1106 11:42:48.857209  147325 genericapiserver.go:765] Skipping API events.k8s.io/v1beta1 because it has no resources.
^C

INFO: canceling context: received interrupt signal

{"level":"warn","ts":"2024-11-06T11:42:48.984676+0100","caller":"embed/serve.go:160","msg":"stopping insecure grpc server due to error","error":"accept tcp 127.0.0.1:44177: use of closed network connection"}
...
I1106 11:42:50.042430  147325 handler.go:142] kube-apiserver: GET "/apis/rbac.authorization.k8s.io/v1/clusterroles" satisfied by gorestful with webservice /apis/rbac.authorization.k8s.io/v1
    test_server.go:241: timed out waiting for the condition
--- FAIL: TestQuota (11.45s)
=== RUN   TestQuotaLimitedResourceDenial
    quota_test.go:292: testing has been interrupted: received interrupt signal
--- FAIL: TestQuotaLimitedResourceDenial (0.00s)
=== RUN   TestQuotaLimitService
    quota_test.go:418: testing has been interrupted: received interrupt signal
--- FAIL: TestQuotaLimitService (0.00s)
FAIL

Does this PR introduce a user-facing change?

NONE

When aborting an integration test with CTRL-C while it runs,
the current test fails and etcd exits. But additional tests were still being
started and the failed slowly because they couldn't connect to etcd.

It's better to fail additional tests in ktesting.Init when the test run has
already been interrupted.

While at it, also make it a bit more obvious that testing was interrupted by
logging it and update one comment about this.

Example:

    $ go test -v ./test/integration/quota
    ...
    I1106 11:42:48.857162  147325 etcd.go:416] "Not using watch cache" resource="events.events.k8s.io"
    I1106 11:42:48.857204  147325 handler.go:286] Adding GroupVersion events.k8s.io v1 to ResourceManager
    W1106 11:42:48.857209  147325 genericapiserver.go:765] Skipping API events.k8s.io/v1beta1 because it has no resources.
    ^C

    INFO: canceling context: received interrupt signal

    {"level":"warn","ts":"2024-11-06T11:42:48.984676+0100","caller":"embed/serve.go:160","msg":"stopping insecure grpc server due to error","error":"accept tcp 127.0.0.1:44177: use of closed network connection"}
    ...
    I1106 11:42:50.042430  147325 handler.go:142] kube-apiserver: GET "/apis/rbac.authorization.k8s.io/v1/clusterroles" satisfied by gorestful with webservice /apis/rbac.authorization.k8s.io/v1
        test_server.go:241: timed out waiting for the condition
    --- FAIL: TestQuota (11.45s)
    === RUN   TestQuotaLimitedResourceDenial
        quota_test.go:292: testing has been interrupted: received interrupt signal
    --- FAIL: TestQuotaLimitedResourceDenial (0.00s)
    === RUN   TestQuotaLimitService
        quota_test.go:418: testing has been interrupted: received interrupt signal
    --- FAIL: TestQuotaLimitService (0.00s)
    FAIL
@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Nov 6, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/test sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Nov 6, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pohly

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 6, 2024
@pohly
Copy link
Contributor Author

pohly commented Nov 6, 2024

/cc @googs1025

Are you perhaps interested in ktesting? This is a small tweak, more might come based on feedback for https://kcsna2024.sched.com/event/1nSjo/unified-framework-for-unit-integration-and-e2e-testing

@googs1025
Copy link
Member

/cc @googs1025

Are you perhaps interested in ktesting? This is a small tweak, more might come based on feedback for https://kcsna2024.sched.com/event/1nSjo/unified-framework-for-unit-integration-and-e2e-testing

Thanks for the invitation, I will follow this topic 😄

The "received interrupt signal" is useful also when running with "go test"
without -v because it shows that the shutdown has started.

But more important is that a progress report gets shown because that feature is
useful in particular when "go test" produces no output while it runs.
@pohly
Copy link
Contributor Author

pohly commented Nov 6, 2024

/assign @googs1025
/priority backlog

@k8s-ci-robot k8s-ci-robot added priority/backlog Higher priority than priority/awaiting-more-evidence. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Nov 6, 2024
@googs1025
Copy link
Member

/retest
to confirm that the CI failed is due to an no related flake, not this change

@aojea
Copy link
Member

aojea commented Nov 6, 2024

can we postpone this for next release to avoid adding functionality to test that may have a large blast radius when we are close to release?

@pohly
Copy link
Contributor Author

pohly commented Nov 6, 2024

Yes, let's hold it for 1.33.

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 6, 2024
@pohly pohly changed the title ktesting: abort entire test suite on SIGINT ktesting: abort entire test suite on SIGINT + /dev/tty Nov 7, 2024
@googs1025
Copy link
Member

/test pull-kubernetes-e2e-gce

maybe we can move to the next step !
/lgtm from my side

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 7, 2025
@googs1025
Copy link
Member

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 7, 2025
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 6, 2025
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle rotten
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Aug 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. priority/backlog Higher priority than priority/awaiting-more-evidence. release-note-none Denotes a PR that doesn't merit a release note. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants