Skip to content

feat(coderd/database/dbpurge): retain most recent agent build logs #14460

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Aug 30, 2024
Merged

feat(coderd/database/dbpurge): retain most recent agent build logs #14460

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
9451962
chore: improve coverage of dbpurge tests for DeleteOldWorkspaceAgentLogs
johnstcn Aug 29, 2024
0c14a3e
feat(dbpurge): update DeleteOldWorkspaceAgentLogs query to keep lates…
johnstcn Aug 29, 2024
0af051e
remove unnecessary columns from CTE
johnstcn Aug 29, 2024
dd97358
fix test
johnstcn Aug 29, 2024
2017434
implement dbmem
johnstcn Aug 29, 2024
725eeb0
ensure that ticks from quartz are dbtimey
johnstcn Aug 29, 2024
591dd91
extract awaitDoTick() test helper
johnstcn Aug 30, 2024
0f8ab25
address race condition in initial tick
johnstcn Aug 30, 2024
5c77b5e
fix double-tick startup race
johnstcn Aug 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat(dbpurge): update DeleteOldWorkspaceAgentLogs query to keep lates… 
…t builds and delete never-connected workspaces
  • Loading branch information
@johnstcn
johnstcn committed Aug 30, 2024
commit 0c14a3ec492dfc9ac9f1f1b0143d64da4ba46840
1 change: 1 addition & 0 deletions coderd/database/querier.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

44 changes: 41 additions & 3 deletions coderd/database/queries.sql.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

44 changes: 41 additions & 3 deletions coderd/database/queries/workspaceagents.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,11 +188,49 @@ INSERT INTO
SELECT * FROM workspace_agent_log_sources WHERE workspace_agent_id = ANY(@ids :: uuid [ ]);

-- If an agent hasn't connected in the last 7 days, we purge it's logs.
-- Exception: if the logs are related to the latest build, we keep those around.
-- Logs can take up a lot of space, so it's important we clean up frequently.
-- name: DeleteOldWorkspaceAgentLogs :exec
DELETE FROM workspace_agent_logs WHERE agent_id IN
(SELECT id FROM workspace_agents WHERE last_connected_at IS NOT NULL
AND last_connected_at < @threshold :: timestamptz);
WITH
latest_builds AS (
SELECT
workspace_id, max(build_number) AS max_build_number
FROM
workspace_builds
GROUP BY
workspace_id
),
old_agents AS (
SELECT
wa.id, wa.last_connected_at, wb.build_number, wb.workspace_id
FROM
workspace_agents AS wa
JOIN
workspace_resources AS wr
ON
wa.resource_id = wr.id
JOIN
workspace_builds AS wb
ON
wb.job_id = wr.job_id
LEFT JOIN
latest_builds
ON
latest_builds.workspace_id = wb.workspace_id
AND
latest_builds.max_build_number = wb.build_number
WHERE
-- Filter out the latest builds for each workspace.
latest_builds.workspace_id IS NULL
AND CASE
-- If the last time the agent connected was before @threshold
WHEN wa.last_connected_at IS NOT NULL THEN
wa.last_connected_at < @threshold :: timestamptz
-- The agent never connected, and was created before @threshold
ELSE wa.created_at < @threshold :: timestamptz
END
)
DELETE FROM workspace_agent_logs WHERE agent_id IN (SELECT id FROM old_agents);

-- name: GetWorkspaceAgentsInLatestBuildByWorkspaceID :many
SELECT
Expand Down