Skip to content

feat: add support for coder_script #9584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 68 commits into from
Sep 25, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
7199651
Add basic migrations
kylecarbs Sep 3, 2023
51b0079
Merge branch 'main' into execscripts
kylecarbs Sep 5, 2023
c18a401
Improve schema
kylecarbs Sep 5, 2023
9ae6e62
Merge branch 'main' into execscripts
kylecarbs Sep 5, 2023
70ebaf3
Refactor agent scripts into it's own package
kylecarbs Sep 7, 2023
89c7af1
Support legacy start and stop script format
kylecarbs Sep 7, 2023
d5df133
Pipe the scripts!
kylecarbs Sep 7, 2023
58964c9
Finish the piping
kylecarbs Sep 7, 2023
00a4e73
Fix context usage
kylecarbs Sep 7, 2023
942fde6
It works!
kylecarbs Sep 7, 2023
92dedad
Fix sql query
kylecarbs Sep 7, 2023
7cf6f0c
Fix SQL query
kylecarbs Sep 7, 2023
5b6f264
Rename `LogSourceID` -> `SourceID`
kylecarbs Sep 7, 2023
e2c9f91
Fix the FE
kylecarbs Sep 7, 2023
6fab755
Merge branch 'main' into execscripts
kylecarbs Sep 7, 2023
51e08f4
fmt
kylecarbs Sep 7, 2023
9a38131
Rename migrations
kylecarbs Sep 7, 2023
c0fac6b
Fix log tests
kylecarbs Sep 8, 2023
f7f1c7a
Fix lint err
kylecarbs Sep 8, 2023
f0a8f53
Fix gen
kylecarbs Sep 8, 2023
66f9185
Fix story type
kylecarbs Sep 8, 2023
78f01d1
Rename source to script
kylecarbs Sep 13, 2023
75388f7
Fix schema jank
kylecarbs Sep 13, 2023
8810326
Uncomment test
kylecarbs Sep 13, 2023
45b395e
Rename proto to TimeoutSeconds
kylecarbs Sep 13, 2023
40bcd9d
Fix comments
kylecarbs Sep 17, 2023
b744c9f
Fix comments
kylecarbs Sep 17, 2023
c844462
Fix legacy endpoint without specified log_source
kylecarbs Sep 18, 2023
139bbf9
Merge branch 'main' into execscripts
kylecarbs Sep 18, 2023
a21f085
Fix non-blocking by default in agent
kylecarbs Sep 18, 2023
49808b1
Fix resources tests
kylecarbs Sep 18, 2023
a47fe28
Fix dbfake
kylecarbs Sep 18, 2023
da40c79
Fix resources
kylecarbs Sep 18, 2023
3631cfa
Fix linting I think
kylecarbs Sep 18, 2023
77bc6e1
Add fixtures
kylecarbs Sep 18, 2023
5b2bd86
fmt
kylecarbs Sep 18, 2023
9d1a4fe
Fix startup script behavior
kylecarbs Sep 18, 2023
ad0d678
Fix comments
kylecarbs Sep 19, 2023
39fb9d3
Fix context
kylecarbs Sep 21, 2023
444decb
Merge branch 'main' into execscripts
kylecarbs Sep 21, 2023
c4d3cb8
Fix cancel
kylecarbs Sep 21, 2023
4c17a8b
Fix SQL tests
kylecarbs Sep 21, 2023
9591e34
Fix e2e tests
kylecarbs Sep 21, 2023
a4a0270
Interrupt on Windows
kylecarbs Sep 21, 2023
35c1c11
Merge branch 'main' into execscripts
matifali Sep 22, 2023
dd5abdf
Fix agent leaking script process
kylecarbs Sep 24, 2023
9e85d7b
Fix migrations
kylecarbs Sep 24, 2023
c26a01b
Merge branch 'main' into execscripts
kylecarbs Sep 24, 2023
9513acf
Fix stories
kylecarbs Sep 24, 2023
2e3611b
Merge branch 'main' into execscripts
matifali Sep 25, 2023
e8b1e43
Fix duplicate logs appearing
kylecarbs Sep 25, 2023
ee1fe11
Merge branch 'execscripts' of https://github.com/coder/coder into exe…
kylecarbs Sep 25, 2023
b837aac
Gen
kylecarbs Sep 25, 2023
f1ff5cc
Fix log location
kylecarbs Sep 25, 2023
4ec3a87
Fix tests
kylecarbs Sep 25, 2023
d36ab53
Fix tests
kylecarbs Sep 25, 2023
eeddb52
Fix log output
kylecarbs Sep 25, 2023
aa5540b
Show display name in output
kylecarbs Sep 25, 2023
f866a92
Fix print
kylecarbs Sep 25, 2023
a99b6dd
Return timeout on start context
kylecarbs Sep 25, 2023
1865590
Gen
kylecarbs Sep 25, 2023
3b26aa0
Fix fixture
kylecarbs Sep 25, 2023
f2f69bb
Fix the agent status
kylecarbs Sep 25, 2023
aa68796
Fix startup timeout msg
kylecarbs Sep 25, 2023
73a7a78
Fix command using shared context
kylecarbs Sep 25, 2023
d1f4963
Fix timeout draining
kylecarbs Sep 25, 2023
784e616
Change signal type
kylecarbs Sep 25, 2023
7ac782b
Add deterministic colors to startup script logs
kylecarbs Sep 25, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix schema jank
  • Loading branch information
kylecarbs committed Sep 13, 2023
commit 75388f701fb0804e3be45950f877eedefd48c2b3
36 changes: 18 additions & 18 deletions agent/agent_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1209,9 +1209,9 @@ func TestAgent_Lifecycle(t *testing.T) {

_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Scripts: []codersdk.WorkspaceAgentScript{{
Script: "sleep 3",
Timeout: time.Nanosecond,
RunOnStart: true,
Script: "sleep 3",
TimeoutSeconds: time.Nanosecond,
RunOnStart: true,
}},
}, 0)

Expand All @@ -1234,9 +1234,9 @@ func TestAgent_Lifecycle(t *testing.T) {

_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Scripts: []codersdk.WorkspaceAgentScript{{
Script: "false",
Timeout: 30 * time.Second,
RunOnStart: true,
Script: "false",
TimeoutSeconds: 30 * time.Second,
RunOnStart: true,
}},
}, 0)

Expand All @@ -1259,9 +1259,9 @@ func TestAgent_Lifecycle(t *testing.T) {

_, client, _, _, _ := setupAgent(t, agentsdk.Manifest{
Scripts: []codersdk.WorkspaceAgentScript{{
Script: "true",
Timeout: 30 * time.Second,
RunOnStart: true,
Script: "true",
TimeoutSeconds: 30 * time.Second,
RunOnStart: true,
}},
}, 0)

Expand All @@ -1284,9 +1284,9 @@ func TestAgent_Lifecycle(t *testing.T) {

_, client, _, _, closer := setupAgent(t, agentsdk.Manifest{
Scripts: []codersdk.WorkspaceAgentScript{{
Script: "sleep 3",
Timeout: 30 * time.Second,
RunOnStop: true,
Script: "sleep 3",
TimeoutSeconds: 30 * time.Second,
RunOnStop: true,
}},
}, 0)

Expand Down Expand Up @@ -1325,9 +1325,9 @@ func TestAgent_Lifecycle(t *testing.T) {

_, client, _, _, closer := setupAgent(t, agentsdk.Manifest{
Scripts: []codersdk.WorkspaceAgentScript{{
Script: "sleep 3",
Timeout: time.Nanosecond,
RunOnStop: true,
Script: "sleep 3",
TimeoutSeconds: time.Nanosecond,
RunOnStop: true,
}},
}, 0)

Expand Down Expand Up @@ -1367,9 +1367,9 @@ func TestAgent_Lifecycle(t *testing.T) {

_, client, _, _, closer := setupAgent(t, agentsdk.Manifest{
Scripts: []codersdk.WorkspaceAgentScript{{
Script: "false",
Timeout: 30 * time.Second,
RunOnStop: true,
Script: "false",
TimeoutSeconds: 30 * time.Second,
RunOnStop: true,
}},
}, 0)

Expand Down
9 changes: 5 additions & 4 deletions agent/agentscripts/agentscripts.go
Original file line number Diff line number Diff line change
Expand Up @@ -141,10 +141,10 @@ func (r *Runner) run(script codersdk.WorkspaceAgentScript) error {
}()

var cmd *exec.Cmd
if script.Timeout > 0 {
if script.TimeoutSeconds > 0 {
var cancel context.CancelFunc
// Add a buffer to forcefully kill with the context.
ctx, cancel = context.WithTimeout(ctx, script.Timeout+(3*time.Second))
ctx, cancel = context.WithTimeout(ctx, script.TimeoutSeconds+(3*time.Second))
defer cancel()
}

Expand Down Expand Up @@ -196,9 +196,9 @@ func (r *Runner) run(script codersdk.WorkspaceAgentScript) error {

// timeout stores whether the process timed out then was gracefully killed.
var timeout chan struct{}
if script.Timeout > 0 {
if script.TimeoutSeconds > 0 {
timeout = make(chan struct{})
timer := time.AfterFunc(script.Timeout, func() {
timer := time.AfterFunc(script.TimeoutSeconds, func() {
close(timeout)
err := cmd.Process.Signal(os.Interrupt)
if err != nil {
Expand Down Expand Up @@ -232,6 +232,7 @@ func (r *Runner) Close() error {
return nil
}
close(r.closed)
r.cron.Stop()
r.cmdCloseWait.Wait()
return nil
}
Expand Down
4 changes: 2 additions & 2 deletions agent/agentscripts/agentscripts_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ func TestTimeout(t *testing.T) {
runner := setup(t, nil)
defer runner.Close()
err := runner.Init([]codersdk.WorkspaceAgentScript{{
Script: "sleep 3",
Timeout: time.Nanosecond,
Script: "sleep 3",
TimeoutSeconds: time.Nanosecond,
}})
require.NoError(t, err)
require.ErrorIs(t, runner.Execute(nil), agentscripts.ErrTimeout)
Expand Down
8 changes: 4 additions & 4 deletions coderd/database/dbfake/dbfake.go
Original file line number Diff line number Diff line change
Expand Up @@ -4557,7 +4557,7 @@ func (q *FakeQuerier) InsertWorkspaceAgentLogSources(_ context.Context, arg data
logSource := database.WorkspaceAgentLogSource{
ID: source,
WorkspaceAgentID: arg.WorkspaceAgentID,
CreatedAt: arg.CreatedAt[index],
CreatedAt: arg.CreatedAt,
DisplayName: arg.DisplayName[index],
Icon: arg.Icon[index],
}
Expand Down Expand Up @@ -4586,7 +4586,7 @@ func (q *FakeQuerier) InsertWorkspaceAgentLogs(_ context.Context, arg database.I
logs = append(logs, database.WorkspaceAgentLog{
ID: id,
AgentID: arg.AgentID,
CreatedAt: arg.CreatedAt[index],
CreatedAt: arg.CreatedAt,
Level: arg.Level[index],
LogSourceID: arg.LogSourceID,
Output: output,
Expand Down Expand Up @@ -4650,8 +4650,8 @@ func (q *FakeQuerier) InsertWorkspaceAgentScripts(_ context.Context, arg databas
StartBlocksLogin: arg.StartBlocksLogin[index],
RunOnStart: arg.RunOnStart[index],
RunOnStop: arg.RunOnStop[index],
Timeout: arg.Timeout[index],
CreatedAt: arg.CreatedAt[index],
TimeoutSeconds: arg.TimeoutSeconds[index],
CreatedAt: arg.CreatedAt,
}
scripts = append(scripts, script)
}
Expand Down
2 changes: 1 addition & 1 deletion coderd/database/dbgen/dbgen.go
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ func WorkspaceAgentLogSource(t testing.TB, db database.Store, orig database.Work
sources, err := db.InsertWorkspaceAgentLogSources(genCtx, database.InsertWorkspaceAgentLogSourcesParams{
WorkspaceAgentID: takeFirst(orig.WorkspaceAgentID, uuid.New()),
ID: []uuid.UUID{takeFirst(orig.ID, uuid.New())},
CreatedAt: []time.Time{takeFirst(orig.CreatedAt, dbtime.Now())},
CreatedAt: takeFirst(orig.CreatedAt, dbtime.Now()),
DisplayName: []string{takeFirst(orig.DisplayName, namesgenerator.GetRandomName(1))},
Icon: []string{takeFirst(orig.Icon, namesgenerator.GetRandomName(1))},
})
Expand Down
8 changes: 7 additions & 1 deletion coderd/database/dump.sql

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
BEGIN;
CREATE TABLE workspace_agent_log_sources (
workspace_agent_id uuid NOT NULL,
id uuid NOT NULL,
id uuid NOT NULL UNIQUE,
created_at timestamptz NOT NULL,
display_name varchar(127) NOT NULL,
icon text NOT NULL,
Expand All @@ -10,15 +10,15 @@ CREATE TABLE workspace_agent_log_sources (

CREATE TABLE workspace_agent_scripts (
workspace_agent_id uuid NOT NULL,
log_source_id uuid NOT NULL,
log_source_id uuid NOT NULL REFERENCES workspace_agent_log_sources(id) ON DELETE CASCADE,
log_path text NOT NULL,
created_at timestamptz NOT NULL,
script text NOT NULL,
cron text NOT NULL,
start_blocks_login boolean NOT NULL,
run_on_start boolean NOT NULL,
run_on_stop boolean NOT NULL,
timeout integer NOT NULL
timeout_seconds integer NOT NULL
);

ALTER TABLE workspace_agent_logs ADD COLUMN log_source_id uuid NOT NULL;
Expand Down
2 changes: 1 addition & 1 deletion coderd/database/models.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

50 changes: 25 additions & 25 deletions coderd/database/queries.sql.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions coderd/database/queries/workspaceagents.sql
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ INSERT INTO
workspace_agent_logs (agent_id, created_at, output, level, log_source_id)
SELECT
@agent_id :: uuid AS agent_id,
unnest(@created_at :: timestamptz [ ]) AS created_at,
@created_at :: timestamptz AS created_at,
unnest(@output :: VARCHAR(1024) [ ]) AS output,
unnest(@level :: log_level [ ]) AS level,
@log_source_id :: uuid AS log_source_id
Expand All @@ -165,8 +165,8 @@ INSERT INTO
workspace_agent_log_sources (workspace_agent_id, id, created_at, display_name, icon)
SELECT
@workspace_agent_id :: uuid AS workspace_agent_id,
@created_at :: timestamptz AS created_at,
unnest(@id :: uuid [ ]) AS id,
unnest(@created_at :: timestamptz [ ]) AS created_at,
unnest(@display_name :: VARCHAR(127) [ ]) AS display_name,
unnest(@icon :: text [ ]) AS icon
RETURNING workspace_agent_log_sources.*;
Expand Down
Loading