Provide an unprotected healthcheck endpoint for K8S #4168
Comments
|
It's also worth considering what goes into a health report. I think simply running a web server is not enough. IMO, it should account for the overall health. Maybe resource usage is not important for K8S, but not all systems run on K8S. In my opinion, a healthy service can deliver on its promise. In the case of Browserless, this means providing a browser instance that can perform browser actions promptly (defined by SLA = config values). Thus, any deviation from that would qualify the instance as "not healthy". |
|
@moltar it'd be interesting to see how that's instrumented in K8s. What you're describing sounds like a passive health check and not a discreet health-check API? Just making sure I understand correctly. |
|
The idea of a health endpoint is simple, IMO, and it works the same way anywhere: A pre-defined, well-known endpoint (conventionally named
What is considered healthy and not healthy is entirely application-specific and can be anything! Typically, a database connection check is done, and some other resource checks, such as the amount of RAM left, etc. The main idea behind health checks is that it is designed to take unhealthy servers (aka instances, pods, tasks) out of rotation and replace them with healthy ones, or at least not serve them more requests (in the hope it will recover). This is also often used for service upgrade rollouts, where you roll out gradually, and once the service passes all health checks, it is swapped out (blue/green). So, in the case of Browserless, the health endpoint should return an unhealthy status when it cannot deliver on its promise, i.e., it cannot fulfill requests in time, which is mainly about the queue size and/or RAM/CPU. |
|
There is a From the logs (startup): browserless.io:limiter:info Concurrency: 3 queue: 10 timeout: 120000ms +0ms
browserless.io:index:info
---------------------------------------------------------
| browserless.io
| To read documentation and more, load in your browser:
|
| OpenAPI: http://0.0.0.0:3000/docs
| Full Documentation: https://docs.browserless.io/
| Debbuger: http://0.0.0.0:3000/debugger/?token=xxx
---------------------------------------------------------
█▓▒
████▒
████▒
████▒ ▒██▓▒
████▒ ▒████
████▒ ▒████
████▒ ▒████
████▒ ▒████
████▒ ▒████
████▒ ▒██████▓▒
████▒ ▒██████████▒
████▒ ▒██████▓████
████▒ ▒█▓▓▒ ▒████
████▒ ▒████
████▒ ▒▓██████
████▒ ▒▓████████▓▒
████▓▓████████▓▒
██████████▓▒
▓███▓▒
+0ms
browserless.io:index:info Running as user "blessuser" +1ms
browserless.io:index:info Starting import of HTTP Routes +0ms
browserless.io:index:info Starting import of WebSocket Routes +120ms
browserless.io:router:trace Registering HTTP POST /content?(/),/chromium/content?(/) +0ms
browserless.io:router:trace Registering HTTP POST /download?(/),/chromium/download?(/) +0ms
browserless.io:router:trace Registering HTTP POST /function?(/),/chromium/function?(/) +0ms
browserless.io:router:trace Registering HTTP GET /json/list?(/) +0ms
browserless.io:router:trace Registering HTTP PUT /json/new?(/) +1ms
browserless.io:router:trace Registering HTTP GET /json/protocol?(/) +0ms
browserless.io:router:trace Registering HTTP GET /json/version?(/) +0ms
browserless.io:router:trace Registering HTTP POST /pdf?(/),/chromium/pdf?(/) +0ms
browserless.io:router:trace Registering HTTP POST /performance?(/),/chromium/performance?(/) +0ms
browserless.io:router:trace Registering HTTP POST /scrape?(/),/chromium/scrape?(/) +0ms
browserless.io:router:trace Registering HTTP POST /screenshot?(/),/chromium/screenshot?(/) +0ms
browserless.io:router:trace Registering HTTP GET /active?(/) +1ms
browserless.io:router:trace Registering HTTP GET /config?(/) +0ms
browserless.io:router:trace Registering HTTP GET /metrics/total?(/) +0ms
browserless.io:router:trace Registering HTTP GET /metrics?(/) +0ms
browserless.io:router:trace Registering HTTP GET /pressure?(/) +0ms
browserless.io:router:trace Registering HTTP GET /sessions?(/) +0ms
browserless.io:router:trace Registering HTTP GET / +1ms
browserless.io:router:trace Registering WebSocket "/devtools/browser/*" +0ms
browserless.io:router:trace Registering WebSocket "?(/),/chromium?(/)" +0ms
browserless.io:router:trace Registering WebSocket "/devtools/page/*" +0ms
browserless.io:router:trace Registering WebSocket "/playwright/chromium?(/),/chromium/playwright?(/)" +0ms
browserless.io:index:info Imported and validated all route files, starting up server. +18ms
browserless.io:server:info Server instantiated with host "0.0.0.0" on port "3000 +0ms
browserless.io:server:info HTTP Server is starting +2ms
browserless.io:server:info HTTP Server is listening on http://0.0.0.0:3000
Use http://0.0.0.0:3000 for API and connect calls +3ms
browserless.io:index:info Starting metrics collection. +7msfrom http://localhost:3000/docs#tag/Management-REST-APIs/paths/~1active/get : --> livenessProbe:
httpGet:
path: /active
port: 3000
readinessProbe:
httpGet:
path: /active
port: 3000 |
|
@kaerbr I think the route is fine, I believe that @winston0410 's issue with it is that a token is required because the route requires authentication. This is usually not ideal for healthcheck routes that are called by infrastructure services like Docker, load balancers, K8s, etc. because there is usually no way to obfuscate the token in the configuration. |
|
Ah I see. |
Is your feature request related to a problem? Please describe.
At the moment there is no unprotected healthcheck endpoint provided by Browserless. Even though a healthcheck mechanism is provided by browserless, it is not K8S native and not fault-tolerant.
For example, if we use this health check feature and create two browserless instance, and one is overloaded(instance A), since the healthcheck result is only known to browserless, K8S might still send traffic to instance A. Whereas if there is an unprotected healthcheck endpoint, and we can use that on
readinessProbe, K8S will know instance A is not ready, and route the request to instance B.Describe the solution you'd like
An unprotected healthcheck endpoint,
/healththat would return 200 if the headless browser is up. Otherwise return 503. Do not check for CPU/Memory resource inside the route, as those requirement will be handled natively else where(For K8S it will be resources limit. Similar feature is provided in docker-compose and other orchestration platform)Describe alternatives you've considered
That is the most K8S native way
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: