Sometimes agent does not connect due to websockets not being forwarded correctly while running behind a reverse proxy. We should mention that in the docs here,
https://coder.com/docs/v2/latest/templates#agent-connection-issues

cc: @bpmct