-
Notifications
You must be signed in to change notification settings - Fork 899
feat: make agent stats' cardinality configurable #12535
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: make agent stats' cardinality configurable #12535
Conversation
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
…ebug works Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
Signed-off-by: Danny Kopping <danny@coder.com>
"github.com/coder/coder/v2/coderd/agentmetrics" | ||
|
||
"github.com/coder/coder/v2/coderd/batchstats" | ||
"github.com/coder/coder/v2/coderd/database/dbtestutil" | ||
"github.com/coder/coder/v2/coderd/database/dbtime" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: sorting?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hhmm, I ran make fmt
and it didn't correct it so I think we have some things to fix in our CI. Will manually sort though 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
goimports
only ensures that imports are grouped, but doesn't handle merging multiple import groups from the same source. I recall that https://github.com/indeedeng/go-groups was built in anger to address this, but it has not been updated in a while.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh nice. I just ran it and it... changes a lot.
I'll raise a separate PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is extremely opinionated
codersdk/deployment.go
Outdated
"github.com/coder/coder/v2/coderd/agentmetrics" | ||
|
||
"github.com/coreos/go-oidc/v3/oidc" | ||
|
||
"github.com/coder/coder/v2/buildinfo" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: here too
func filterInvalidLabels(labels []string) []string { | ||
var out []string | ||
|
||
for _, label := range labels { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe optimize it this way?
validLabels := make([]string, 0, len(labels))
for _, label := range labels {
if label != agentmetrics.LabelTemplateName {
validLabels = append(validLabels, label)
}
}
// filterInvalidLabels handles a slightly messy situation whereby `prometheus-aggregate-agent-stats-by` can control on | ||
// which labels agent stats are aggregated, but for these specific metrics in this file there is no `template` label value, | ||
// and therefore we have to exclude it from the list of acceptable labels. | ||
func filterInvalidLabels(labels []string) []string { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we add an internal_test.go
to verify this function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only a couple of nits from me. Tested locally with a small scaletest.
"golang.org/x/xerrors" | ||
|
||
"github.com/coder/coder/v2/coderd/agentmetrics" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: sorting
…/coder into dk/configurable-cardinality
Signed-off-by: Danny Kopping <danny@coder.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
Part 2 of #12468
We initially had to revert the above PR because of a panic from Prometheus.
This was caused by the 4 default labels being passed in to
AgentStats()
, however these metrics do not contain a value for thetemplate_name
label. This was masked in a test by passing innil
for the aggregation labels which defaulted them to the 3 correct ones, but I subsequently changed the default value fromnil
in the actual code to all 4 labels for various reasons.I implemented the fix in 3c1d016. I can't think of a cleaner approach than this; if you come up with a better approach, please share!
Note to reviewers: the code is otherwise identical to #12468, so simply reviewing 3c1d016 should suffice.