Skip to content

ci: Run tests using PostgreSQL database and mock #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jan 22, 2022
Merged

ci: Run tests using PostgreSQL database and mock #49

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e45b11e
ci: Run tests using PostgreSQL database and mock
kylecarbs Jan 22, 2022
0a0da98
Only run PostgreSQL tests once for speed
kylecarbs Jan 22, 2022
3a29464
Merge branch 'main' into cidb
kylecarbs Jan 22, 2022
53412a2
Fix race condition of log after close
kylecarbs Jan 22, 2022
bceaeeb
Fix comment
kylecarbs Jan 22, 2022
05ded12
Merge branch 'main' into cidb
kylecarbs Jan 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/workflows/coder.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,11 +149,19 @@ jobs:

- run: go install gotest.tools/gotestsum@latest

- run:
- name: Test with Mock Database
run:
gotestsum --jsonfile="gotests.json" --packages="./..." --
-covermode=atomic -coverprofile="gotests.coverage" -timeout=3m
-count=3 -race -parallel=2

- name: Test with PostgreSQL Database
if: runner.os == 'Linux'
run:
DB=true gotestsum --jsonfile="gotests.json" --packages="./..." --
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! It's neat that it's so easy to test both ways.

I like that we get the best of both worlds here - the mock DB seems ideal for quick iteration because it's so fast, but testing against a real PostgreSL driver is important too, but doesn't have to slow down test authoring.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed!

-covermode=atomic -coverprofile="gotests.coverage" -timeout=3m
-count=1 -race -parallel=2

- uses: codecov/codecov-action@v2
with:
token: ${{ secrets.CODECOV_TOKEN }}
Expand Down
18 changes: 18 additions & 0 deletions coderd/coderdtest/coderdtest.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,20 @@ package coderdtest

import (
"context"
"database/sql"
"net/http/httptest"
"net/url"
"os"
"testing"

"github.com/stretchr/testify/require"

"cdr.dev/slog/sloggers/slogtest"
"github.com/coder/coder/coderd"
"github.com/coder/coder/codersdk"
"github.com/coder/coder/database"
"github.com/coder/coder/database/databasefake"
"github.com/coder/coder/database/postgres"
)

// Server represents a test instance of coderd.
Expand All @@ -27,6 +31,20 @@ type Server struct {
func New(t *testing.T) Server {
// This can be hotswapped for a live database instance.
db := databasefake.New()
if os.Getenv("DB") != "" {
connectionURL, close, err := postgres.Open()
require.NoError(t, err)
t.Cleanup(close)
sqlDB, err := sql.Open("postgres", connectionURL)
require.NoError(t, err)
t.Cleanup(func() {
_ = sqlDB.Close()
})
err = database.Migrate(sqlDB)
require.NoError(t, err)
db = database.New(sqlDB)
}

handler := coderd.New(&coderd.Options{
Logger: slogtest.Make(t, nil),
Database: db,
Expand Down
3 changes: 2 additions & 1 deletion database/query.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,13 +86,14 @@ INSERT INTO
email,
name,
login_type,
revoked,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem related to the ability to switch between DB / mock - but maybe needed to be updated?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, this was actually a bug the DB implementation caught!

hashed_password,
created_at,
updated_at,
username
)
VALUES
($1, $2, $3, $4, $5, $6, $7, $8) RETURNING *;
($1, $2, $3, $4, false, $5, $6, $7, $8) RETURNING *;

-- name: UpdateAPIKeyByID :exec
UPDATE
Expand Down
3 changes: 2 additions & 1 deletion database/query.sql.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions peer/channel.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ func (c *Channel) init() {

c.conn.dcDisconnectListeners.Add(1)
c.conn.dcFailedListeners.Add(1)
c.conn.dcClosedWaitGroup.Add(1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch on these! Looks like a candidate for a backport to cdr/m too - seems like we're missing this wait logic there too

go func() {
var err error
// A DataChannel can disconnect multiple times, so this needs to loop.
Expand Down Expand Up @@ -274,6 +275,7 @@ func (c *Channel) closeWithError(err error) error {
close(c.sendMore)
c.conn.dcDisconnectListeners.Sub(1)
c.conn.dcFailedListeners.Sub(1)
c.conn.dcClosedWaitGroup.Done()

if c.rwc != nil {
_ = c.rwc.Close()
Expand Down
15 changes: 10 additions & 5 deletions peer/conn.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ type Conn struct {
dcDisconnectListeners atomic.Uint32
dcFailedChannel chan struct{}
dcFailedListeners atomic.Uint32
dcClosedWaitGroup sync.WaitGroup

localCandidateChannel chan webrtc.ICECandidateInit
localSessionDescriptionChannel chan webrtc.SessionDescription
Expand All @@ -125,11 +126,10 @@ type Conn struct {
pingEchoChan *Channel
pingEchoOnce sync.Once
pingEchoError error

pingMutex sync.Mutex
pingOnce sync.Once
pingChan *Channel
pingError error
pingMutex sync.Mutex
pingOnce sync.Once
pingChan *Channel
pingError error
}

func (c *Conn) init() error {
Expand Down Expand Up @@ -502,5 +502,10 @@ func (c *Conn) CloseWithError(err error) error {
// this call will return an error that isn't typed. We don't check the error because
// closing an already closed connection isn't an issue for us.
_ = c.rtc.Close()

// Waits for all DataChannels to exit before officially labeling as closed.
// All logging, goroutines, and async functionality is cleaned up after this.
c.dcClosedWaitGroup.Wait()

return err
}