Skip to content

feat: add WorkspaceUpdates tailnet RPC #14847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Nov 1, 2024

Conversation

ethanndickson
Copy link
Member

@ethanndickson ethanndickson commented Sep 27, 2024

Closes #14716
Closes #14717

Adds a new user-scoped tailnet API endpoint (api/v2/tailnet) with a new RPC stream for receiving updates on workspaces owned by a specific user, as defined in #14716.

When a stream is started, the WorkspaceUpdatesProvider will begin listening on the user-scoped pubsub events implemented in #14964. When a relevant event type is seen (such as a workspace state transition), the provider will query the DB for all the workspaces (and agents) owned by the user. This gets compared against the result of the previous query to produce a set of workspace updates.

Workspace updates can be requested for any user ID, however only workspaces the authorised user is permitted to ActionRead will have their updates streamed.
Opening a tunnel to an agent requires that the user can perform ActionSSH against the workspace containing it.

Copy link
Member Author

ethanndickson commented Sep 27, 2024

@ethanndickson ethanndickson changed the title feat: add WorkspaceUpdates rpc feat: add WorkspaceUpdates tailnet RPC Sep 27, 2024
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from 22985f7 to db04bcf Compare September 27, 2024 08:42
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from ddc493e to d2d165e Compare September 27, 2024 08:42
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from db04bcf to 50f874c Compare September 30, 2024 09:35
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch 2 times, most recently from d57f3e6 to 8102c71 Compare September 30, 2024 12:12
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from 50f874c to fe1e8b5 Compare October 1, 2024 04:14
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 8102c71 to 4f57562 Compare October 1, 2024 04:14
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from fe1e8b5 to aaf8e86 Compare October 1, 2024 12:02
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch 2 times, most recently from ab7f678 to fb84465 Compare October 1, 2024 12:16
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from be7fa9e to cb5adf4 Compare October 1, 2024 12:22
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from fb84465 to 1a8392d Compare October 1, 2024 12:36
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from cb5adf4 to 2bc6e69 Compare October 2, 2024 05:09
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 1a8392d to 51cd615 Compare October 2, 2024 05:09
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from 2bc6e69 to 54cbfaf Compare October 3, 2024 15:07
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 51cd615 to de1435d Compare October 3, 2024 15:07
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from 54cbfaf to aad36cf Compare October 4, 2024 03:45
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from de1435d to 398fa2d Compare October 4, 2024 03:45
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from aad36cf to b5f7529 Compare October 4, 2024 05:51
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 398fa2d to 077608b Compare October 4, 2024 05:51
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from b5f7529 to fabe2d1 Compare October 4, 2024 11:45
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 077608b to c762ae8 Compare October 4, 2024 11:45
@github-actions github-actions bot added the stale This issue is like stale bread. label Oct 12, 2024
@matifali matifali removed the stale This issue is like stale bread. label Oct 12, 2024
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from fabe2d1 to d89d373 Compare October 17, 2024 06:55
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 84730b8 to f3b74b7 Compare October 29, 2024 06:34
auth rbac.Authorizer

ctx context.Context
cancelFn func()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ctx isn't used anywhere, so it and cancelFn are superfluous, which I think makes Close() on this superfluous.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intention was for sub to store a context who's parent is updatesProvider's context, so that calling Close on the updates provider closes all the existing subscriptions - will fix.

@@ -133,7 +133,7 @@ var errDisconnect = xerrors.New("graceful disconnect")

func (c *connIO) handleRequest(req *proto.CoordinateRequest) error {
c.logger.Debug(c.peerCtx, "got request")
err := c.auth.Authorize(req)
err := c.auth.Authorize(c.coordCtx, req)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be the c.peerCtx --- coordCtx is the context of the Coordinator's lifetime.

You should add a test that adds a subject to a context you use to call Coordinate() and verify the subject is there on the Authorize call. Same test for the AGPL implementation.

@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from e05b286 to a45a43c Compare October 29, 2024 09:53
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from f3b74b7 to d1fac01 Compare October 29, 2024 09:53
_ = testutil.RequireRecvCtx(ctx, t, ch)
// If we don't cancel the context, the coordinator close will wait until the
// peer request loop finishes, which will be after the timeout
peerCtxCancel()
Copy link
Member Author

@ethanndickson ethanndickson Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was actually passing without, but it would take the entire 10 seconds. Interestingly, the PGCoord doesn't have this behaviour

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, PGCoord is more aggressive with closing things down, forcibly booting anything still connected because it has contexts plumbed thru for background tasks like heartbeats. The AGPL coordinator is considerably simpler and has no coordinator-scoped context.

@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch 2 times, most recently from 964bfe9 to 494afa9 Compare October 29, 2024 10:23
Copy link
Contributor

@spikecurtis spikecurtis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits inline, but I don't need to review again

_ = testutil.RequireRecvCtx(ctx, t, ch)
// If we don't cancel the context, the coordinator close will wait until the
// peer request loop finishes, which will be after the timeout
peerCtxCancel()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, PGCoord is more aggressive with closing things down, forcibly booting anything still connected because it has contexts plumbed thru for background tasks like heartbeats. The AGPL coordinator is considerably simpler and has no coordinator-scoped context.

@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 494afa9 to cdb4235 Compare October 29, 2024 13:35
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch from a45a43c to 744633f Compare November 1, 2024 03:02
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from cdb4235 to 7d0a2b6 Compare November 1, 2024 03:02
@ethanndickson ethanndickson force-pushed the 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents branch 2 times, most recently from 54ceea2 to 8be52ed Compare November 1, 2024 03:18
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 7d0a2b6 to 33d6628 Compare November 1, 2024 03:18
@ethanndickson ethanndickson changed the base branch from 09-25-chore_add_db_query_to_retrieve_workspaces_their_agents to graphite-base/14847 November 1, 2024 03:36
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 33d6628 to 7be63f5 Compare November 1, 2024 03:36
@ethanndickson ethanndickson changed the base branch from graphite-base/14847 to main November 1, 2024 03:37
@ethanndickson ethanndickson force-pushed the 09-27-feat_add_workspaceupdates_rpc branch from 7be63f5 to b509099 Compare November 1, 2024 03:37
@ethanndickson ethanndickson merged commit b1298a3 into main Nov 1, 2024
28 checks passed
@ethanndickson ethanndickson deleted the 09-27-feat_add_workspaceupdates_rpc branch November 1, 2024 03:53
@github-actions github-actions bot locked and limited conversation to collaborators Nov 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants