Skip to content

fix: Refactor tailnet conn AwaitReachable to allow for pings >1s RTT #5096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 15, 2022
Merged

fix: Refactor tailnet conn AwaitReachable to allow for pings >1s RTT #5096

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b0d789b
fix: Refactor tailnet conn AwaitReachable to allow for pings >1s RTT
mafredri Nov 15, 2022
b7256c5
fix: Add exponential backoff
mafredri Nov 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 27 additions & 7 deletions tailnet/conn.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"sync"
"time"

"github.com/cenkalti/backoff/v4"
"github.com/google/uuid"
"go4.org/netipx"
"golang.org/x/xerrors"
Expand Down Expand Up @@ -424,24 +425,43 @@ func (c *Conn) Ping(ctx context.Context, ip netip.Addr) (time.Duration, error) {
// address is reachable. It's the callers responsibility to provide
// a timeout, otherwise this function will block forever.
func (c *Conn) AwaitReachable(ctx context.Context, ip netip.Addr) bool {
ticker := time.NewTicker(time.Millisecond * 100)
defer ticker.Stop()
completedCtx, completed := context.WithCancel(ctx)
ctx, cancel := context.WithCancel(ctx)
defer cancel() // Cancel all pending pings on exit.

completedCtx, completed := context.WithCancel(context.Background())
defer completed()

run := func() {
ctx, cancelFunc := context.WithTimeout(completedCtx, time.Second)
defer cancelFunc()
// Safety timeout, initially we'll have around 10-20 goroutines
// running in parallel. The exponential backoff will converge
// around ~1 ping / 30s, this means we'll have around 10-20
// goroutines pending towards the end as well.
ctx, cancel := context.WithTimeout(ctx, 5*time.Minute)
defer cancel()

_, err := c.Ping(ctx, ip)
if err == nil {
completed()
}
}

eb := backoff.NewExponentialBackOff()
eb.MaxElapsedTime = 0
eb.InitialInterval = 50 * time.Millisecond
eb.MaxInterval = 30 * time.Second
// Consume the first interval since
// we'll fire off a ping immediately.
_ = eb.NextBackOff()

t := backoff.NewTicker(eb)
defer t.Stop()

go run()
defer completed()
for {
select {
case <-completedCtx.Done():
return true
case <-ticker.C:
case <-t.C:
// Pings can take a while, so we can run multiple
// in parallel to return ASAP.
go run()
Expand Down