Skip to content

fix: provisionerd: add more context to logs emitted, fix log level #6508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 8, 2023

Conversation

johnstcn
Copy link
Member

@johnstcn johnstcn commented Mar 8, 2023

  • Previously, we were logging all provision response logs at level INFO, regardless of the log level of the log streamed from the provisioner. We now log these at the original level (ERROR or WARN, defaulting to INFO).
  • Now logging "provision failed" message at level ERROR WARN and including the error field in the message.

@johnstcn johnstcn self-assigned this Mar 8, 2023
@johnstcn johnstcn marked this pull request as ready for review March 8, 2023 14:27
@johnstcn johnstcn requested review from mafredri, coadler and mtojek March 8, 2023 14:27
@@ -895,8 +895,9 @@ func (r *Runner) buildWorkspace(ctx context.Context, stage string, req *sdkproto
})
case *sdkproto.Provision_Response_Complete:
if msgType.Complete.Error != "" {
r.logger.Info(context.Background(), "provision failed; updating state",
r.logger.Warn(context.Background(), "provision failed; updating state",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: logging at level ERROR breaks existing unit tests, and I don't want to set slogtest.IgnoreErrors.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understandable. It might be the "correct" approach though, if we expect errors to be logged, then setting IgnoreErrors seems like the right course of action.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mafredri Would you agree that anything "ERROR" and above should be mainly for events useful to a system administrator?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be relevant for e.g. template authors, but generally I doubt coder/provisioner logs (via logger) would be surfaced to anyone but systems admins.

case sdkproto.LogLevel_ERROR:
r.logger.Error(ctx, msg, fields...)
default: // should never happen, but we should not explode either.
r.logger.Info(ctx, msg, fields...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, it could be possible that a customer misconfigures the Coder deployment, INFO will be selected as default logging level, and it can blow up the cluster if there are thousands of logs, but that's just theory.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

INFO is the status quo, and the only way for someone to select a different level is to set CODER_VERBOSE or --verbose, both of which need to be set explicitly.

@johnstcn johnstcn merged commit 26a725f into main Mar 8, 2023
@johnstcn johnstcn deleted the cj/provisionerd-log-improvements branch March 8, 2023 15:12
@github-actions github-actions bot locked and limited conversation to collaborators Mar 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants