Skip to content

feat: add migrations and queries to support prebuilds #16891

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 77 commits into from
Apr 3, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
300e80f
add prebuilds system user database changes and associated changes
SasSwart Mar 12, 2025
b788237
optionally prevent system users from counting to user count
dannykopping Mar 13, 2025
8122595
appease the linter
dannykopping Mar 13, 2025
bfb7c28
add unit test for system user behaviour
dannykopping Mar 13, 2025
6639167
reverting RBAC changes; not relevant here
dannykopping Mar 13, 2025
769ae1d
removing unnecessary changes
dannykopping Mar 13, 2025
e7e9c27
exclude system user db tests from non-linux OSs
dannykopping Mar 13, 2025
3936047
Rename prebuild system user reference
SasSwart Mar 17, 2025
8bdcafb
ensure that users.IsSystem is not nullable
SasSwart Mar 17, 2025
412d198
feat: add migrations and queries to support prebuilds
SasSwart Mar 12, 2025
51773ec
Simplify workspace_latest_build view
dannykopping Mar 14, 2025
23773c2
Revert test change
dannykopping Mar 17, 2025
bc3ff44
make gen
dannykopping Mar 17, 2025
baa3076
refactor: add comments to SQL queries
evgeniy-scherbina Mar 19, 2025
ed14fb3
test: added get-presets-backoff test
evgeniy-scherbina Mar 20, 2025
3cc74fb
refactor: add comment to SQL query
evgeniy-scherbina Mar 20, 2025
fc32154
refactor: add comments + improve tests
evgeniy-scherbina Mar 21, 2025
d7b4ec4
fix: bug in SQL
evgeniy-scherbina Mar 21, 2025
e8b53f7
test: minor changes to the test
evgeniy-scherbina Mar 21, 2025
9df6554
refactor: remove job_status from SQL query
evgeniy-scherbina Mar 21, 2025
ccc309e
refactor: embed preset_prebuilds table into presets table
evgeniy-scherbina Mar 21, 2025
ee1f16a
refactor: rename sql table
evgeniy-scherbina Mar 21, 2025
d040ddd
refactor: remove unnecessary JOIN
evgeniy-scherbina Mar 23, 2025
83a6722
refactor: remove unnecessary JOIN
evgeniy-scherbina Mar 23, 2025
cd70710
refactor: use INNER JOIN for consistency
evgeniy-scherbina Mar 23, 2025
97cc4ff
refactor: simplify GetPresetsBackoff SQL Query
evgeniy-scherbina Mar 24, 2025
4d59039
Revert "refactor: simplify GetPresetsBackoff SQL Query"
evgeniy-scherbina Mar 24, 2025
205d6af
refactor: improve GetPresetsBackoff query
evgeniy-scherbina Mar 24, 2025
e489e1b
Merge remote-tracking branch 'origin/main' into prebuilds-db
evgeniy-scherbina Mar 25, 2025
1b29686
Merge remote-tracking branch 'origin/main' into prebuilds-db
evgeniy-scherbina Mar 25, 2025
20470e4
fix: bump migration numbers
evgeniy-scherbina Mar 25, 2025
7b9c8ce
test: remove deprecated test
evgeniy-scherbina Mar 25, 2025
e189a0b
fix: fix linter
evgeniy-scherbina Mar 25, 2025
692c0e5
fix: fix 000310_prebuilds.down migration
evgeniy-scherbina Mar 25, 2025
f747db0
fix: fix fixture migration
evgeniy-scherbina Mar 25, 2025
3166a42
fix: fix get-presets-backoff test
evgeniy-scherbina Mar 25, 2025
aa6b490
fix: fix linter
evgeniy-scherbina Mar 25, 2025
bc4e7d2
fix: fix linter
evgeniy-scherbina Mar 25, 2025
f167b92
correctly select for the latest built with a preset in latest_prebuil…
SasSwart Mar 26, 2025
8fd34ab
Merge remote-tracking branch 'origin/main' into prebuilds-db
SasSwart Mar 26, 2025
7a8ec49
Properly label and filter metrics for prebuilds
SasSwart Mar 26, 2025
a64d661
test: fix db tests
evgeniy-scherbina Mar 27, 2025
c787cd2
test: added tests for workspaces with multiple agents
evgeniy-scherbina Mar 27, 2025
bd38603
refactor: avoid code duplication
evgeniy-scherbina Mar 27, 2025
097f9c3
clarify query clause
SasSwart Mar 27, 2025
4cfdd6f
tidy up dbauthz_test.go
SasSwart Mar 27, 2025
4a34d52
refactor: remove * usage from prebuilds.sql queries
evgeniy-scherbina Mar 27, 2025
8d9cd45
refactor: remove * usage from prebuilds views
evgeniy-scherbina Mar 27, 2025
f870d7e
refactor: join wlb with pj
evgeniy-scherbina Mar 27, 2025
18ad931
refactor: Rename SQL query
evgeniy-scherbina Mar 27, 2025
4667171
Added comments for SQL query
evgeniy-scherbina Mar 27, 2025
a26c094
refactor: fix down migration
evgeniy-scherbina Mar 27, 2025
6ed4121
Merge remote-tracking branch 'origin/main' into prebuilds-db
SasSwart Mar 28, 2025
2312f41
renumber migrations
SasSwart Mar 28, 2025
5150a5c
refactor: clarify comment for SQL query
evgeniy-scherbina Mar 28, 2025
bff34ea
refactor: fix indentations
evgeniy-scherbina Mar 28, 2025
ef462b6
refactor: rename helper func in test package
evgeniy-scherbina Mar 28, 2025
dc45165
refactor: database level tests
evgeniy-scherbina Mar 28, 2025
9c8a352
refactor: database level tests
evgeniy-scherbina Mar 28, 2025
eb80919
refactor: helper funcs in db-level tests
evgeniy-scherbina Mar 28, 2025
0b2bbee
refactor: minor improvement in SQL query
evgeniy-scherbina Mar 28, 2025
3a97bf6
refactor: rename SQL queries
evgeniy-scherbina Mar 28, 2025
2eeb884
refactor: rename SQL queries
evgeniy-scherbina Mar 28, 2025
73f99e8
refactor: rename fields in SQL query
evgeniy-scherbina Mar 31, 2025
113e12b
fix: improve rbac policies
evgeniy-scherbina Apr 1, 2025
217cae0
fix: minor fix in filtered_builds CTE
evgeniy-scherbina Apr 1, 2025
b0bf220
fix: formatting
evgeniy-scherbina Apr 2, 2025
ff72a00
refactor: minor refactoring
evgeniy-scherbina Apr 2, 2025
a007d4a
fix: handle presets with the same tv.id and name
evgeniy-scherbina Apr 2, 2025
ea9c53b
fix: redefine RBAC permissions for prebuilds
evgeniy-scherbina Apr 2, 2025
66d44ed
Merge remote-tracking branch 'origin/main' into prebuilds-db
evgeniy-scherbina Apr 2, 2025
29e121f
fix: fix migration numbers
evgeniy-scherbina Apr 2, 2025
f64754f
test: fix dbmem tests
evgeniy-scherbina Apr 2, 2025
7e02397
test: fix dbmem tests
evgeniy-scherbina Apr 2, 2025
55d9827
fix: linter
evgeniy-scherbina Apr 2, 2025
866454b
fix: linter
evgeniy-scherbina Apr 2, 2025
4fa959a
fix: linter
evgeniy-scherbina Apr 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: formatting
  • Loading branch information
evgeniy-scherbina committed Apr 2, 2025
commit b0bf2202fc4f093721c79fae6732cbbc21f95f4a
40 changes: 20 additions & 20 deletions coderd/database/migrations/000313_prebuilds.up.sql
Original file line number Diff line number Diff line change
Expand Up @@ -17,44 +17,44 @@ ORDER BY wb.workspace_id, wb.build_number DESC;
-- (including lifecycle_state which indicates is agent ready or not) and corresponding preset_id for prebuild
CREATE VIEW workspace_prebuilds AS
WITH
-- All workspaces owned by the "prebuilds" user.
all_prebuilds AS (
-- All workspaces owned by the "prebuilds" user.
all_prebuilds AS (
SELECT w.id, w.name, w.template_id, w.created_at
FROM workspaces w
WHERE w.owner_id = 'c42fdf75-3097-471c-8c33-fb52454d81c0' -- The system user responsible for prebuilds.
),
-- We can't rely on the template_version_preset_id in the workspace_builds table because this value is only set on the
-- initial workspace creation. Subsequent stop/start transitions will not have a value for template_version_preset_id,
-- and therefore we can't rely on (say) the latest build's chosen template_version_preset_id.
--
-- See https://github.com/coder/internal/issues/398
workspaces_with_latest_presets AS (
SELECT DISTINCT ON (workspace_id) workspace_id, template_version_preset_id
FROM workspace_builds
WHERE template_version_preset_id IS NOT NULL
ORDER BY workspace_id, build_number DESC
),
-- We can't rely on the template_version_preset_id in the workspace_builds table because this value is only set on the
-- initial workspace creation. Subsequent stop/start transitions will not have a value for template_version_preset_id,
-- and therefore we can't rely on (say) the latest build's chosen template_version_preset_id.
--
-- See https://github.com/coder/internal/issues/398
workspaces_with_latest_presets AS (
SELECT DISTINCT ON (workspace_id) workspace_id, template_version_preset_id
FROM workspace_builds
WHERE template_version_preset_id IS NOT NULL
ORDER BY workspace_id, build_number DESC
),
-- workspaces_with_agents_status contains workspaces owned by the "prebuilds" user,
-- along with the readiness status of their agents.
-- A workspace is marked as 'ready' only if ALL of its agents are ready.
workspaces_with_agents_status AS (
SELECT w.id AS workspace_id,
BOOL_AND(wa.lifecycle_state = 'ready'::workspace_agent_lifecycle_state) AS ready
BOOL_AND(wa.lifecycle_state = 'ready'::workspace_agent_lifecycle_state) AS ready
FROM workspaces w
INNER JOIN workspace_latest_builds wlb ON wlb.workspace_id = w.id
INNER JOIN workspace_resources wr ON wr.job_id = wlb.job_id
INNER JOIN workspace_agents wa ON wa.resource_id = wr.id
WHERE w.owner_id = 'c42fdf75-3097-471c-8c33-fb52454d81c0' -- The system user responsible for prebuilds.
GROUP BY w.id
),
current_presets AS (SELECT w.id AS prebuild_id, wlp.template_version_preset_id
FROM workspaces w
INNER JOIN workspaces_with_latest_presets wlp ON wlp.workspace_id = w.id
WHERE w.owner_id = 'c42fdf75-3097-471c-8c33-fb52454d81c0') -- The system user responsible for prebuilds.
current_presets AS (SELECT w.id AS prebuild_id, wlp.template_version_preset_id
FROM workspaces w
INNER JOIN workspaces_with_latest_presets wlp ON wlp.workspace_id = w.id
WHERE w.owner_id = 'c42fdf75-3097-471c-8c33-fb52454d81c0') -- The system user responsible for prebuilds.
SELECT p.id, p.name, p.template_id, p.created_at, COALESCE(a.ready, false) AS ready, cp.template_version_preset_id AS current_preset_id
FROM all_prebuilds p
LEFT JOIN workspaces_with_agents_status a ON a.workspace_id = p.id
INNER JOIN current_presets cp ON cp.prebuild_id = p.id;
LEFT JOIN workspaces_with_agents_status a ON a.workspace_id = p.id
INNER JOIN current_presets cp ON cp.prebuild_id = p.id;

CREATE VIEW workspace_prebuild_builds AS
SELECT id, workspace_id, template_version_id, transition, job_id, template_version_preset_id, build_number
Expand Down
4 changes: 2 additions & 2 deletions coderd/database/migrations/000314_preset_prebuilds.up.sql
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
ALTER TABLE template_version_presets
ADD COLUMN desired_instances INT NULL,
ADD COLUMN invalidate_after_secs INT NULL DEFAULT 0;
ADD COLUMN desired_instances INT NULL,
ADD COLUMN invalidate_after_secs INT NULL DEFAULT 0;

-- We should not be able to have presets with the same name for a particular template version.
CREATE UNIQUE INDEX idx_unique_preset_name ON template_version_presets (name, template_version_id);
98 changes: 50 additions & 48 deletions coderd/database/queries.sql.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

96 changes: 49 additions & 47 deletions coderd/database/queries/prebuilds.sql
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,23 @@ SELECT
t.deleted,
t.deprecated != '' AS deprecated
FROM templates t
INNER JOIN template_versions tv ON tv.template_id = t.id
INNER JOIN template_version_presets tvp ON tvp.template_version_id = tv.id
INNER JOIN organizations o ON o.id = t.organization_id
INNER JOIN template_versions tv ON tv.template_id = t.id
INNER JOIN template_version_presets tvp ON tvp.template_version_id = tv.id
INNER JOIN organizations o ON o.id = t.organization_id
WHERE tvp.desired_instances IS NOT NULL -- Consider only presets that have a prebuild configuration.
AND (t.id = sqlc.narg('template_id')::uuid OR sqlc.narg('template_id') IS NULL);
AND (t.id = sqlc.narg('template_id')::uuid OR sqlc.narg('template_id') IS NULL);

-- name: GetRunningPrebuiltWorkspaces :many
SELECT p.id,
p.name,
p.template_id,
b.template_version_id,
p.current_preset_id AS current_preset_id,
p.ready,
p.created_at
SELECT
p.id,
p.name,
p.template_id,
b.template_version_id,
p.current_preset_id AS current_preset_id,
p.ready,
p.created_at
FROM workspace_prebuilds p
INNER JOIN workspace_latest_builds b ON b.workspace_id = p.id
INNER JOIN workspace_latest_builds b ON b.workspace_id = p.id
WHERE (b.transition = 'start'::workspace_transition
AND b.job_status = 'succeeded'::provisioner_job_status);

Expand All @@ -39,14 +40,14 @@ WHERE (b.transition = 'start'::workspace_transition
-- Prebuild considered in-progress if it's in the "starting", "stopping", or "deleting" state.
SELECT t.id AS template_id, wpb.template_version_id, wpb.transition, COUNT(wpb.transition)::int AS count
FROM workspace_latest_builds wlb
INNER JOIN workspace_prebuild_builds wpb ON wpb.id = wlb.id
-- We only need these counts for active template versions.
-- It doesn't influence whether we create or delete prebuilds
-- for inactive template versions. This is because we never create
-- prebuilds for inactive template versions, we always delete
-- running prebuilds for inactive template versions, and we ignore
-- prebuilds that are still building.
INNER JOIN templates t ON t.active_version_id = wlb.template_version_id
INNER JOIN workspace_prebuild_builds wpb ON wpb.id = wlb.id
-- We only need these counts for active template versions.
-- It doesn't influence whether we create or delete prebuilds
-- for inactive template versions. This is because we never create
-- prebuilds for inactive template versions, we always delete
-- running prebuilds for inactive template versions, and we ignore
-- prebuilds that are still building.
INNER JOIN templates t ON t.active_version_id = wlb.template_version_id
WHERE wlb.job_status IN ('pending'::provisioner_job_status, 'running'::provisioner_job_status)
GROUP BY t.id, wpb.template_version_id, wpb.transition;

Expand All @@ -69,37 +70,38 @@ WITH filtered_builds AS (
-- Only select builds which are for prebuild creations
SELECT wlb.template_version_id, wlb.created_at, tvp.id AS preset_id, wlb.job_status, tvp.desired_instances
FROM template_version_presets tvp
INNER JOIN workspace_latest_builds wlb ON wlb.template_version_preset_id = tvp.id
INNER JOIN workspaces w ON wlb.workspace_id = w.id
INNER JOIN template_versions tv ON wlb.template_version_id = tv.id
INNER JOIN templates t ON tv.template_id = t.id AND t.active_version_id = tv.id
INNER JOIN workspace_latest_builds wlb ON wlb.template_version_preset_id = tvp.id
INNER JOIN workspaces w ON wlb.workspace_id = w.id
INNER JOIN template_versions tv ON wlb.template_version_id = tv.id
INNER JOIN templates t ON tv.template_id = t.id AND t.active_version_id = tv.id
WHERE tvp.desired_instances IS NOT NULL -- Consider only presets that have a prebuild configuration.
AND wlb.transition = 'start'::workspace_transition
AND w.owner_id = 'c42fdf75-3097-471c-8c33-fb52454d81c0'
AND wlb.transition = 'start'::workspace_transition
AND w.owner_id = 'c42fdf75-3097-471c-8c33-fb52454d81c0'
),
time_sorted_builds AS (
-- Group builds by preset, then sort each group by created_at.
-- Group builds by preset, then sort each group by created_at.
SELECT fb.template_version_id, fb.created_at, fb.preset_id, fb.job_status, fb.desired_instances,
ROW_NUMBER() OVER (PARTITION BY fb.preset_id ORDER BY fb.created_at DESC) as rn
ROW_NUMBER() OVER (PARTITION BY fb.preset_id ORDER BY fb.created_at DESC) as rn
FROM filtered_builds fb
),
failed_count AS (
-- Count failed builds per preset in the given period
-- Count failed builds per preset in the given period
SELECT preset_id, COUNT(*) AS num_failed
FROM filtered_builds
WHERE job_status = 'failed'::provisioner_job_status
AND created_at >= @lookback::timestamptz
GROUP BY preset_id
)
SELECT tsb.template_version_id,
tsb.preset_id,
COALESCE(fc.num_failed, 0)::int AS num_failed,
MAX(tsb.created_at)::timestamptz AS last_build_at
SELECT
tsb.template_version_id,
tsb.preset_id,
COALESCE(fc.num_failed, 0)::int AS num_failed,
MAX(tsb.created_at)::timestamptz AS last_build_at
FROM time_sorted_builds tsb
LEFT JOIN failed_count fc ON fc.preset_id = tsb.preset_id
LEFT JOIN failed_count fc ON fc.preset_id = tsb.preset_id
WHERE tsb.rn <= tsb.desired_instances -- Fetch the last N builds, where N is the number of desired instances; if any fail, we backoff
AND tsb.job_status = 'failed'::provisioner_job_status
AND created_at >= @lookback::timestamptz
AND tsb.job_status = 'failed'::provisioner_job_status
AND created_at >= @lookback::timestamptz
GROUP BY tsb.template_version_id, tsb.preset_id, fc.num_failed;

-- name: ClaimPrebuiltWorkspace :one
Expand All @@ -114,24 +116,24 @@ WHERE w.id IN (
INNER JOIN templates t ON p.template_id = t.id
WHERE (b.transition = 'start'::workspace_transition
AND b.job_status IN ('succeeded'::provisioner_job_status))
-- The prebuilds system should never try to claim a prebuild for an inactive template version.
-- Nevertheless, this filter is here as a defensive measure:
AND b.template_version_id = t.active_version_id
AND p.current_preset_id = @preset_id::uuid
AND p.ready
-- The prebuilds system should never try to claim a prebuild for an inactive template version.
-- Nevertheless, this filter is here as a defensive measure:
AND b.template_version_id = t.active_version_id
AND p.current_preset_id = @preset_id::uuid
AND p.ready
LIMIT 1 FOR UPDATE OF p SKIP LOCKED -- Ensure that a concurrent request will not select the same prebuild.
)
RETURNING w.id, w.name;

-- name: GetPrebuildMetrics :many
SELECT
t.name as template_name,
tvp.name as preset_name,
t.name as template_name,
tvp.name as preset_name,
o.name as organization_name,
COUNT(*) as created_count,
COUNT(*) FILTER (WHERE pj.job_status = 'failed'::provisioner_job_status) as failed_count,
COUNT(*) FILTER (
WHERE w.owner_id != 'c42fdf75-3097-471c-8c33-fb52454d81c0'::uuid -- The system user responsible for prebuilds.
COUNT(*) as created_count,
COUNT(*) FILTER (WHERE pj.job_status = 'failed'::provisioner_job_status) as failed_count,
COUNT(*) FILTER (
WHERE w.owner_id != 'c42fdf75-3097-471c-8c33-fb52454d81c0'::uuid -- The system user responsible for prebuilds.
) as claimed_count
FROM workspaces w
INNER JOIN workspace_prebuild_builds wpb ON wpb.workspace_id = w.id
Expand Down
2 changes: 1 addition & 1 deletion coderd/database/queries/presets.sql
Original file line number Diff line number Diff line change
Expand Up @@ -52,5 +52,5 @@ WHERE
-- name: GetPresetByID :one
SELECT tvp.*, tv.template_id, tv.organization_id FROM
template_version_presets tvp
INNER JOIN template_versions tv ON tvp.template_version_id = tv.id
INNER JOIN template_versions tv ON tvp.template_version_id = tv.id
WHERE tvp.id = @preset_id;