Skip to content

chore!: delete old connection events from audit log #18735

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 15, 2025

Conversation

ethanndickson
Copy link
Member

@ethanndickson ethanndickson commented Jul 3, 2025

Breaking change (changelog note):

With new connection events appearing in the Connection Log, connection events older than 90 days will now be deleted from the Audit Log. If you require this legacy data, we recommend querying it from the REST API or making a backup of the database/these events before upgrading your Coder deployment. Please see the PR for details on what exactly will be deleted.
Of note is that there are currently no plans to delete connection events from the Connection Log.

Context

This is the fifth PR for moving connection events out of the audit log.

In previous PRs:

  • New connection logs have been routed to the connection_logs table. They will not appear in the audit log.
  • These new connection logs are served from the new /api/v2/connectionlog endpoint.

In this PR:

  • We'll now clean existing connection events out of the audit log, if they are older than 90 days, We do this in batches of 1000, every 10 minutes.

The criteria for deletion is simple:

WHERE
(
     action = 'connect'
     OR action = 'disconnect'
     OR action = 'open'
     OR action = 'close'
)
AND "time" < @before_time::timestamp with time zone

where @before_time is currently configured to 90 days in the past.

Future PRs:

  • Write documentation for the endpoint / feature

Copilot

This comment was marked as outdated.

@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from 5db1b73 to 4be35f4 Compare July 3, 2025 07:36
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from da6d50a to 0386f5b Compare July 3, 2025 07:36
@ethanndickson ethanndickson marked this pull request as ready for review July 3, 2025 08:24
@github-actions github-actions bot added the release/breaking This label is applied to PRs to detect breaking changes as part of the release process label Jul 3, 2025
@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from 4be35f4 to 9239251 Compare July 3, 2025 08:29
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from 0386f5b to ab296c6 Compare July 3, 2025 08:30
@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from 9239251 to 7f2dccd Compare July 10, 2025 05:58
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from ab296c6 to 8034bbf Compare July 10, 2025 05:58
@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from 7f2dccd to d6ec519 Compare July 10, 2025 10:29
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from 8034bbf to 143bf38 Compare July 10, 2025 10:29
@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from d6ec519 to 15aac3f Compare July 10, 2025 10:37
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from 143bf38 to fd367a7 Compare July 10, 2025 10:37
@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from 15aac3f to c505c02 Compare July 10, 2025 10:50
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from fd367a7 to a7f5137 Compare July 10, 2025 10:50
@ethanndickson ethanndickson requested a review from Copilot July 10, 2025 10:51
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds a new cleanup routine to remove old “connect”, “disconnect”, “open”, and “close” events from the audit_logs table (older than 90 days) in 1,000-row batches every 10 minutes.

  • Introduces a SQL command and Go wrapper for DeleteOldAuditLogConnectionEvents.
  • Hooks the deletion into the existing dbpurge ticker loop, with tests, metrics, mocks, and authz rules.
  • Updates interfaces, mocks, and authorization policies to support the new operation.

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
coderd/database/queries/auditlogs.sql New DELETE FROM audit_logs query definition
coderd/database/queries.sql.go SQL constant and executor method for the delete query
coderd/database/querier.go Interface addition for the new deletion method
coderd/database/dbpurge/dbpurge.go Integrated deletion into the purge workflow
coderd/database/dbpurge/dbpurge_test.go Tests verifying deletion logic and batch limits
coderd/database/dbmock/dbmock.go Mock implementations for the new method
coderd/database/dbmetrics/querymetrics.go Metrics wrapper around the delete operation
coderd/database/dbauthz/dbauthz.go Authorization guard for the deletion call
coderd/database/dbauthz/dbauthz_test.go Authz test for the new deletion endpoint
Comments suppressed due to low confidence (1)

coderd/database/querier.go:99

  • [nitpick] Add a doc comment above DeleteOldAuditLogConnectionEvents in the sqlcQuerier interface to describe its behavior, matching the style of other interface methods.
	DeleteOldAuditLogConnectionEvents(ctx context.Context, arg DeleteOldAuditLogConnectionEventsParams) error

@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from c505c02 to 9238f25 Compare July 14, 2025 02:04
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from a7f5137 to d370914 Compare July 14, 2025 02:04
@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from 9238f25 to 823dc41 Compare July 14, 2025 06:13
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from d370914 to 20a639f Compare July 14, 2025 06:13
@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from 823dc41 to e364d00 Compare July 15, 2025 03:33
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from 20a639f to 8a1d30b Compare July 15, 2025 03:33
@ethanndickson ethanndickson force-pushed the ethan/fe-connection-log branch from e364d00 to cf9592b Compare July 15, 2025 04:00
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from 8a1d30b to cdf23a2 Compare July 15, 2025 04:00
@ethanndickson ethanndickson changed the base branch from ethan/fe-connection-log to graphite-base/18735 July 15, 2025 05:04
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from cdf23a2 to 325ab3e Compare July 15, 2025 05:12
@graphite-app graphite-app bot changed the base branch from graphite-base/18735 to main July 15, 2025 05:12
@ethanndickson ethanndickson force-pushed the ethan/delete-old-connection-events branch from 325ab3e to ba228c9 Compare July 15, 2025 05:12
@ethanndickson ethanndickson merged commit f42de9f into main Jul 15, 2025
55 of 56 checks passed
@ethanndickson ethanndickson deleted the ethan/delete-old-connection-events branch July 15, 2025 05:45
@github-actions github-actions bot locked and limited conversation to collaborators Jul 15, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
release/breaking This label is applied to PRs to detect breaking changes as part of the release process
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants