Skip to content

More retry #236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 2, 2023
Merged

More retry #236

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
4bacfc4
Add log when deploying before listing editors
code-asher Apr 27, 2023
73f9b19
Fix timer not canceling
code-asher Apr 28, 2023
405fb43
Retry direct connection
code-asher Apr 28, 2023
4e5206e
Provide better error when dd times out
code-asher Apr 28, 2023
88c3297
Throwable does not need to be nullable
code-asher May 1, 2023
f417542
Break out humanizeDuration
code-asher May 1, 2023
1a67157
Munge error at point of display
code-asher May 1, 2023
d1d3e91
Refactor retry
code-asher May 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Provide better error when dd times out
  • Loading branch information
@code-asher
code-asher committed Apr 28, 2023
commit 4e5206ee6453ffa4f01d654e5e617a67fddadf24
16 changes: 13 additions & 3 deletions src/main/kotlin/com/coder/gateway/sdk/Retry.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package com.coder.gateway.sdk
import com.intellij.openapi.diagnostic.Logger
import com.intellij.openapi.progress.ProcessCanceledException
import com.intellij.ssh.SshException
import com.jetbrains.gateway.ssh.deploy.DeployException
import kotlinx.coroutines.delay
import java.util.Random
import java.util.concurrent.TimeUnit
Expand All @@ -19,8 +20,9 @@ fun unwrap(ex: Exception): Throwable? {

/**
* Similar to Intellij's except it gives you the next delay, logs differently,
* updates periodically (for counting down), runs forever, and takes a
* predicate for determining whether we should retry.
* updates periodically (for counting down), runs forever, takes a predicate for
* determining whether we should retry, and has some special handling for
* exceptions to provide the true cause or better messages.
*
* The update will have a boolean to indicate whether it is the first update (so
* things like duplicate logs can be avoided). If remaining is null then no
Expand Down Expand Up @@ -67,7 +69,15 @@ suspend fun <T> suspendingRetryWithExponentialBackOff(
while (remainingMs > 0) {
val remainingS = TimeUnit.MILLISECONDS.toSeconds(remainingMs)
val remaining = if (remainingS < 1) "now" else "in $remainingS second${if (remainingS > 1) "s" else ""}"
update(attempt, unwrappedEx, remaining)
// When the worker upload times out Gateway just says it failed.
// Even the root cause (IllegalStateException) is useless. The
// error also includes a very long useless tmp path. With all
// that in mind, provide a better error.
val mungedEx =
if (unwrappedEx is DeployException && unwrappedEx.message.contains("Worker binary deploy failed"))
DeployException("Failed to upload worker binary...it may have timed out", unwrappedEx)
else unwrappedEx
update(attempt, mungedEx, remaining)
val next = min(remainingMs, TimeUnit.SECONDS.toMillis(1))
remainingMs -= next
delay(next)
Expand Down