Skip to content

Prometheus metrics endpoint returns 500 instead of metrics #11451

Closed
@janLo

Description

@janLo

Our instances stopped to respond to metric requests:

❯ curl http://coder-service2.xxx:2112/
An error has occurred while serving metrics:

132 error(s) occurred:
* collected metric "agent_sessions_errors_total" { label:{name:"agent_name"  value:"main"}  label:{name:"error_type"  value:"output_io_copy"}  label:{name:"magic_type"  value:"ssh"}  label:{name:"pty"  value:"yes"}  label:{name:"template_name"  value:"debian-like-buildenv"}  label:{name:"username"  value:"xxx"}  label:{name:"workspace_name"  value:"muts"}  counter:{value:11}} was collected before with the same name and label values
* collected metric "agent_sessions_total" { label:{name:"agent_name"  value:"main"}  label:{name:"magic_type"  value:"ssh"}  label:{name:"pty"  value:"no"}  label:{name:"template_name"  value:"debian-like-buildenv"}  label:{name:"username"  value:"xxx"}  label:{name:"workspace_name"  value:"muts"}  counter:{value:6}} was collected before with the same name and label values
* collected metric "agent_sessions_total" { label:{name:"agent_name"  value:"main"}  label:{name:"magic_type"  value:"ssh"}  label:{name:"pty"  value:"yes"}  label:{name:"template_name"  value:"debian-like-buildenv"}  label:{name:"username"  value:"xxx"}  label:{name:"workspace_name"  value:"muts"}  counter:{value:18}} was collected before with the same name and label values
[...]

It seem to have happened with the upgrade to 2.6.0.

Metadata

Metadata

Assignees

Labels

s1Bugs that break core workflows. Only humans may set this.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions