Skip to content

feat: expose agent metrics via Prometheus endpoint #7011

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Apr 7, 2023
Merged

feat: expose agent metrics via Prometheus endpoint #7011

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
8d4e67d
WIP
mtojek Apr 3, 2023
da729e6
Merge branch 'main' into 6724-metrics
mtojek Apr 4, 2023
9ad09b2
WIP
mtojek Apr 4, 2023
440657c
WIP
mtojek Apr 5, 2023
8764f89
Agents
mtojek Apr 5, 2023
663b5d5
fix
mtojek Apr 5, 2023
63aff5e
1min
mtojek Apr 5, 2023
3905481
fix
mtojek Apr 5, 2023
f8d6f46
WIP
mtojek Apr 5, 2023
d487a77
Test
mtojek Apr 5, 2023
7acbaf0
docs
mtojek Apr 5, 2023
7418779
fmt
mtojek Apr 5, 2023
3a8e4e6
Add timer to measure the metrics collection
mtojek Apr 6, 2023
b5d0581
Use CachedGaugeVec
mtojek Apr 6, 2023
e4d708b
Unit tests
mtojek Apr 6, 2023
e0669f0
Address PR comments
mtojek Apr 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Address PR comments
  • Loading branch information
@mtojek
mtojek committed Apr 7, 2023
commit e0669f013cc947f8719ddd53b49e18086b8eb54d
25 changes: 20 additions & 5 deletions coderd/prometheusmetrics/collector.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@ import (
"github.com/prometheus/client_golang/prometheus"
)

// CachedGaugeVec is a wrapper for the prometheus.GaugeVec which allows
// for staging changes in the metrics vector. Calling "WithLabelValues(...)"
// will update the internal gauge value, but it will not be returned by
// "Collect(...)" until the "Commit()" method is called. The "Commit()" method
// resets the internal gauge and applies all staged changes to it.
//
// The Use of CachedGaugeVec is recommended for use cases when there is a risk
// that the Prometheus collector receives incomplete metrics, collected
// in the middle of metrics recalculation, between "Reset()" and the last
// "WithLabelValues()" call.
type CachedGaugeVec struct {
m sync.Mutex

Expand Down Expand Up @@ -35,9 +45,6 @@ func NewCachedGaugeVec(gaugeVec *prometheus.GaugeVec) *CachedGaugeVec {
}

func (v *CachedGaugeVec) Describe(desc chan<- *prometheus.Desc) {
v.m.Lock()
defer v.m.Unlock()

v.gaugeVec.Describe(desc)
}

Expand All @@ -49,6 +56,13 @@ func (v *CachedGaugeVec) Collect(ch chan<- prometheus.Metric) {
}

func (v *CachedGaugeVec) WithLabelValues(operation VectorOperation, value float64, labelValues ...string) {
switch operation {
case VectorOperationAdd:
case VectorOperationSet:
default:
Comment on lines +59 to +62
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually prefer this. But it does not matter.

switch operation {
	case VectorOperationAdd, VectorOperationSet:
	default:
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#7085

panic("unsupported vector operation")
}

v.m.Lock()
defer v.m.Unlock()

Expand All @@ -59,6 +73,9 @@ func (v *CachedGaugeVec) WithLabelValues(operation VectorOperation, value float6
})
}

// Commit will set the internal value as the cached value to return from "Collect()".
// The internal metric value is completely reset, so the caller should expect
// the gauge to be empty for the next 'WithLabelValues' values.
func (v *CachedGaugeVec) Commit() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Commit will set the internal value as the cached value to return from 'Collect'.
// The internal metric value is completely reset, so the caller should expect
// the gauge to be empty for the next 'WithLabelValues' values.

Suggested change
func (v *CachedGaugeVec) Commit() {
func (v *CachedGaugeVec) Commit() {

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment added.

v.m.Lock()
defer v.m.Unlock()
Expand All @@ -71,8 +88,6 @@ func (v *CachedGaugeVec) Commit() {
g.Add(record.value)
case VectorOperationSet:
g.Set(record.value)
default:
panic("unsupported vector operation")
}
}

Expand Down