Description
Follow-up of #12762
We expose a few metrics which are effectively being "exported" from our database. We should use the idiomatic approach of writing exporters as described in https://prometheus.io/docs/instrumenting/writing_exporters/#collectors. In other words, run the queries in response to scrape requests; currently we run these queries in coderd/prometheusmetrics/prometheusmetrics.go
every minute and update the metrics.
Our current approach leads to stale metrics between scrapes (the underlying data may have changed, but subsequent scrapes do not pull the "correct" data) and we leave ourselves open to a possible data races by scraping in the middle of updating metrics.
See https://github.com/prometheus-community/postgres_exporter/blob/master/collector/pg_locks.go for an example of how we should be approaching this.