feat: implement autoscaling mechanism for prebuilds #18126

evgeniy-scherbina · 2025-05-30T12:24:16Z

Closes coder/internal#312
Depends on coder/terraform-provider-coder#408

This PR adds support for defining an autoscaling block for prebuilds, allowing number of desired instances to scale dynamically based on a schedule.

Example usage:

data "coder_workspace_preset" "us-nix" {
  ...
  
  prebuilds = {
    instances = 0                  # default to 0 instances
    
    autoscaling = {
      timezone = "UTC"             # a single timezone is used for simplicity
      
      # Scale to 3 instances during the work week
      schedule {
        cron = "* 8-18 * * 1-5"    # from 8AM–6:59PM, Mon–Fri, UTC
        instances = 3              # scale to 3 instances
      }
      
      # Scale to 1 instance on Saturdays for urgent support queries
      schedule {
        cron = "* 8-14 * * 6"      # from 8AM–2:59PM, Sat, UTC
        instances = 1              # scale to 1 instance
      }
    }
  }
}

Behavior

Multiple schedule blocks per prebuilds block are supported.
If the current time matches any defined autoscaling schedule, the corresponding number of instances is used.
If no schedule matches, the default instance count (prebuilds.instances) is used as a fallback.

Why

This feature allows prebuild instance capacity to adapt to predictable usage patterns, such as:

Scaling up during business hours or high-demand periods
Reducing capacity during off-hours to save resources

Cron specification

The cron specification is interpreted as a continuous time range.

For example, the expression:

* 9-18 * * 1-5

is intended to represent a continuous range from 09:00 to 18:59, Monday through Friday.

However, due to minor implementation imprecision, it is currently interpreted as a range from 08:59:00 to 18:58:59, Monday through Friday.

This slight discrepancy arises because the evaluation is based on whether a specific point in time falls within the range, using the github.com/coder/coder/v2/coderd/schedule/cron library, which performs per-minute matching rather than strict range evaluation.

dannykopping · 2025-06-13T08:55:53Z

coderd/database/dbgen/dbgen.go

 	})
 	require.NoError(t, err, "insert preset")
 	return preset
 }

+func PresetPrebuildSchedule(t testing.TB, db database.Store, seed database.InsertPresetPrebuildScheduleParams) database.TemplateVersionPresetPrebuildSchedule {


Typically dbgen help funcs define default values when seed is a zero value. Can you add those please?

dannykopping · 2025-06-13T08:56:01Z

coderd/database/dbgen/dbgen.go

+		CronExpression: seed.CronExpression,
+		Instances:      seed.Instances,
+	})
+	require.NoError(t, err, "insert preset")


Suggested change

require.NoError(t, err, "insert preset")

require.NoError(t, err, "insert preset prebuild schedule")

dannykopping · 2025-06-13T08:58:35Z

coderd/database/migrations/000335_add_autoscaling_to_presets.up.sql

@@ -0,0 +1,12 @@
+-- Add autoscaling_timezone column to template_version_presets table
+ALTER TABLE template_version_presets
+ADD COLUMN autoscaling_timezone TEXT DEFAULT 'UTC' NOT NULL;


Why do we need a default here, if the provider is not defining a default?

dannykopping · 2025-06-13T08:59:34Z

coderd/database/migrations/000335_add_autoscaling_to_presets.up.sql

+    id UUID PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
+    preset_id UUID NOT NULL,
+    cron_expression TEXT NOT NULL,
+    instances INTEGER NOT NULL,


Nit: to match template_version_presets' definition.

Suggested change

instances INTEGER NOT NULL,

desired_instances INTEGER NOT NULL,

dannykopping · 2025-06-13T09:03:27Z

coderd/database/queries/presets.sql

@@ -69,3 +83,6 @@ SELECT tvp.*, tv.template_id, tv.organization_id FROM
 	template_version_presets tvp
 	INNER JOIN template_versions tv ON tvp.template_version_id = tv.id
 WHERE tvp.id = @preset_id;
+
+-- name: GetPresetPrebuildSchedules :many
+SELECT * FROM template_version_preset_prebuild_schedules;


Don't we want to constrain this to schedules that are only associated to active template versions?

dannykopping · 2025-06-13T09:20:43Z

coderd/prebuilds/preset_snapshot.go

+	}
+
+	// Look for a schedule whose cron expression matches the provided time
+	for _, schedule := range p.PrebuildSchedules {


This is my main reason for blocking this review. We have to deal with conflicting schedules here, explicitly.

If we take this current approach, and conflicting schedules are evaluated in different orders on each reconciliation loop, then that will cause a lot of churn by spinning up and down prebuilds if those schedules' instance counts differ.

dannykopping · 2025-06-13T09:26:31Z

coderd/prebuilds/preset_snapshot_test.go

+			spec:            "* 9-18 * * 1-5",
+			at:              mustParseTime(t, time.RFC1123, "Mon, 02 Jun 2025 8:59:00 UTC"),


This would violate my expectations as an operator. If the autoscaling schedule is going to kick in 1m before my given cron expression, that would be quite surprising and unintuitive.

I think we should try address this shortcoming if we can.
Edit: I looked into the implementation and made some suggestions.

This test could also be more explicit about why 8:59:00 matches 9-18.

coderd/schedule/cron/cron.go

dannykopping · 2025-06-13T09:59:54Z

go.mod

+
+//replace github.com/coder/terraform-provider-coder/v2 => /home/coder/terraform-provider-coder


Suggested change

//replace github.com/coder/terraform-provider-coder/v2 => /home/coder/terraform-provider-coder

dannykopping · 2025-06-13T09:59:59Z

go.mod

@@ -101,7 +101,8 @@ require (
 	github.com/coder/quartz v0.2.1
 	github.com/coder/retry v1.5.1
 	github.com/coder/serpent v0.10.0
-	github.com/coder/terraform-provider-coder/v2 v2.5.3
+	// TODO: release new provider version and set it here
+	github.com/coder/terraform-provider-coder/v2 v2.5.4-0.20250611180058-d61894db8492


Remember TODO.

spikecurtis · 2025-06-13T11:40:55Z

The "continuous time" interpretation of the crontab makes it, I guess, relatively simple to specify spans that start and end on the hour, but consider attempting to program a span like 7:45am - 9:10am. You'd need 3 spans

45-59 7 * * 1-5
* 8 * * 1-5
0-10 9 * * 1-5

Are we just kind of assuming that operators likely only want hourly precision?

Co-authored-by: Danny Kopping <danny@coder.com>

…toscaling-mechanism

github-actions bot assigned evgeniy-scherbina May 30, 2025

evgeniy-scherbina force-pushed the yevhenii/prebuilds-autoscaling-mechanism branch from 606894f to ff0e813 Compare May 30, 2025 12:25

evgeniy-scherbina changed the title ~~Implement autoscaling mechanism for prebuilds~~ feat: implement autoscaling mechanism for prebuilds May 30, 2025

evgeniy-scherbina force-pushed the yevhenii/prebuilds-autoscaling-mechanism branch 5 times, most recently from 9af5e02 to bcfbb04 Compare June 6, 2025 19:53

evgeniy-scherbina force-pushed the yevhenii/prebuilds-autoscaling-mechanism branch 4 times, most recently from 3a25178 to e0d1de7 Compare June 11, 2025 19:05

feat: implement autoscaling mechanism for prebuilds

47cb6fc

evgeniy-scherbina force-pushed the yevhenii/prebuilds-autoscaling-mechanism branch from 80e52ee to 47cb6fc Compare June 12, 2025 19:08

evgeniy-scherbina marked this pull request as ready for review June 12, 2025 19:45

evgeniy-scherbina requested review from spikecurtis and johnstcn as code owners June 12, 2025 19:45

evgeniy-scherbina mentioned this pull request Jun 12, 2025

feat: add autoscaling configuration for prebuilds coder/terraform-provider-coder#408

Open

dannykopping requested changes Jun 13, 2025

View reviewed changes

evgeniy-scherbina and others added 10 commits June 13, 2025 10:22

fix: precision issue with is-within-range function

0f4d521

Co-authored-by: Danny Kopping <danny@coder.com>

test: fix is-within-range test

b957eb6

test: minor fixes in is-within-range test

de920b8

test: fix tests failed due to is-within-range function

5d31b43

fix: migration numbers

b3130a8

feat: define functions for validation of schedules

0565fb7

Merge remote-tracking branch 'origin/main' into yevhenii/prebuilds-au…

99d76e5

…toscaling-mechanism

regenerate protofiles

bdfcd9e

test: improve test coverage for schedules-overlap function

ea7766f

test: improve test coverage for validate-schedules function

1e78317

evgeniy-scherbina added 7 commits June 16, 2025 13:26

refactor: add logger to preset-snapshot

5d99ef1

refactor: fallback to default on error in calc-desired-instances

158b92e

refactor: change signature of calc-desired-instances

396d080

fix: improve schedule validation

12dba6c

fix: bug related to DOM and DOW interpretation

3fa98b2

fix: add schedules validation

8a063ff

fix: use TimeRange function instead of Weekly

12d3d88

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: implement autoscaling mechanism for prebuilds #18126

feat: implement autoscaling mechanism for prebuilds #18126

evgeniy-scherbina commented May 30, 2025 •

edited

Loading

Uh oh!

dannykopping Jun 13, 2025

Uh oh!

dannykopping Jun 13, 2025

Uh oh!

dannykopping Jun 13, 2025

Uh oh!

dannykopping Jun 13, 2025

Uh oh!

dannykopping Jun 13, 2025

Uh oh!

dannykopping Jun 13, 2025

Uh oh!

dannykopping Jun 13, 2025 •

edited

Loading

Uh oh!

Uh oh!

dannykopping Jun 13, 2025

Uh oh!

dannykopping Jun 13, 2025

Uh oh!

spikecurtis commented Jun 13, 2025

Uh oh!

Uh oh!

	require.NoError(t, err, "insert preset")
	require.NoError(t, err, "insert preset prebuild schedule")

	instances INTEGER NOT NULL,
	desired_instances INTEGER NOT NULL,

		spec: "* 9-18 * * 1-5",
		at: mustParseTime(t, time.RFC1123, "Mon, 02 Jun 2025 8:59:00 UTC"),


		//replace github.com/coder/terraform-provider-coder/v2 => /home/coder/terraform-provider-coder

feat: implement autoscaling mechanism for prebuilds #18126

Are you sure you want to change the base?

feat: implement autoscaling mechanism for prebuilds #18126

Conversation

evgeniy-scherbina commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Behavior

Why

Cron specification

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dannykopping Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

spikecurtis commented Jun 13, 2025

Uh oh!

Uh oh!

evgeniy-scherbina commented May 30, 2025 •

edited

Loading

dannykopping Jun 13, 2025 •

edited

Loading