For Docker, Kubernetes this can be done by utilizing cgroups

check

cat /sys/fs/cgroup/cpu.stat
cat /sys/fs/cgroup/memory.current
cat /sys/fs/cgroup/memory.max

Note cgroups paths can be different on different Linux kernel version [see attached screenshot]
https://stackoverflow.com/a/51251148/9183518

@deansheather, would you like to make Pittsburgh and Helsinki identical?

If this differs by kernel version then we probably should use some other way to determine the memory usage instead for compatibility

I will install the same kernel on Pittsburgh though, but I can't restart it during work hours

Nope I can't install the same kernel version because it doesn't install on fossa due to uninstallable dependencies, and it's not worth upgrading the entire server unless we have to IMO.

I could try compiling the kernel myself at some point though

The image is jammy but the hosts run different OSes. Only the Pittsburgh region runs the old kernel and 20.04 release.

I added metadata but most of the devs in us so it doesn't work for them. 😞

As I now have a way to show correct memory and cpu usage using cgroups. Do we need to implement coder stat

They are available on dev.coder.com dogfood template in all regions except the Pittsburgh (it's still on kernel 5.4 where the cgroups paths are different)

cc: @ammario @bpmct

@sreya will look into this. I'm a bit concerned that, without a coder stat, it's difficult for a user to know which command to use, for which kernel, and the info won't be accurate. If we can provide good docs + a command that doesn't require uncommon dependencies + its easy for users to use, I'm fine with that.

@johnstcn expressed interest in doing this so re-assigning to him

https://pkg.go.dev/github.com/containerd/cgroups#readme-get-stats-on-the-cgroup

coder stat is nice because I assume it can work in any containerized environment (e.g. docker, ECS, GKE autopilot) and does not require metrics-server. However, we may find cases where it is better to get metrics from Kubernetes for accuracy, performance, or availability reasons.

An alternative (or additional enhancement) could be to extend https://github.com/coder/coder-logstream-kube to query the Kubernetes metrics-server and send metadata to the workspace.

Status update: roughly 90% complete.