Skip to content

Informer example requires fix: infomer restarted after onerror event doesn't dispatch other events #795

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
faxioman opened this issue Apr 15, 2022 · 11 comments
Closed

Informer example requires fix: infomer restarted after onerror event doesn't dispatch other events #795

faxioman opened this issue Apr 15, 2022 · 11 comments
Labels
lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed.

Comments

@faxioman
Copy link

Describe the bug
The informer example shows a way to deal with errors:

informer.on('error', (err: k8s.V1Pod) => {
  console.error(err);
  // Restart informer after 5sec
  setTimeout(() => {
    informer.start();
  }, 5000);
});

But when an error occurs (eg. the api server pod is restarted) if you restart the informer with informer.start(), all the events (add, delete, update) are never fired again.
I don't know if this is a desired behaviour and the example is wrong or the example is fine and the informer has a bug.

** Client Version **
0.16.3

** Server Version **
Kubernetes v1.23.3

To Reproduce
Execute the informer, restart the api server, observe that informer events are never fired

Expected behavior
After an error event and an informer restart, all add/delete/update event for the informer should be fired again

Environment (please complete the following information):

  • OS: MacOs (minikube v1.25.2)
  • NodeJS Version v16.14.0
  • Cloud runtime minikube v1.25.2
@brendandburns
Copy link
Contributor

That is expected. Events should only be fired for newly added things (since presumably you already received events for the earlier things).

This should be consistent with the way informers work in other languages also.

@faxioman
Copy link
Author

Yes, this behaviour is clear… but after informer.start inside an onerror handler, if I remove a k8s resource and recreate that resource the events delete/add are never fired.

@brendandburns
Copy link
Contributor

@faxioman that's true, in that case you should receive an update event rather than a delete/add event. If you don't receive an update in that situation then there is a bug.

Given the way that the Kubernetes API works, if the watch drops for some reason (forcing a re-list to occur) then I believe it isn't possible to tell the difference between whether an object was deleted and re-added, or that it was simply updated.

Does this behavior cause problems for you? I wouldn't expect that it would cause problems for your operator as it should always be looking at the 'current' state.

As mentioned above, you should get an 'update' event, if you don't get that, then we should figure out why not.

@faxioman
Copy link
Author

@brendandburns ok, I'll check if an update event is raised.
Anyway, I see that if I simply remove a crd (after an informer restart) the event delete is never fired.

@brendandburns
Copy link
Contributor

Hrm, that seems like a bug. Once the informer is restarted, if you delete something, you should see the delete event.

Can you provide more details on the repro (or sample code)

thanks

@brendandburns
Copy link
Contributor

Friendly ping for more details if you have them.

Thanks!

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 13, 2022
@dominykas
Copy link
Contributor

dominykas commented Sep 13, 2022 •
edited
Loading

I have a suspicion that some recent changes affected this to the opposite - I'm now seeing unhandled ECONNRESET in 0.17.0, whereas 0.15.x seems to reconnect OK, while 0.16.x seems to just stay silent.

Will be digging deeper...

@dominykas dominykas mentioned this issue Sep 19, 2022
Closed
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Oct 16, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

@k8s-ci-robot
Copy link
Contributor

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot closed this as not planned Won't fix, can't repro, duplicate, stale Nov 15, 2022
@feloy feloy mentioned this issue Mar 4, 2024
Closed
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@dominykas @faxioman @brendandburns @k8s-ci-robot @k8s-triage-robot