Prometheus-compatible metrics are made available on the default port, on the /metrics endpoint.
Below is a list of the metrics that are exposed:
| Metric Name | Type |
|---|---|
lorax_request_count |
Counter |
lorax_request_success |
Counter |
lorax_request_failure |
Counter |
lorax_request_duration |
Histogram |
lorax_request_queue_duration |
Histogram |
lorax_request_validation_duration |
Histogram |
lorax_request_inference_duration |
Histogram |
lorax_request_mean_time_per_token_duration |
Histogram |
lorax_request_generated_tokens |
Histogram |
lorax_request_input_length |
Histogram |
For all histograms, there are metrics that are autogenerated which are the metric name + _sum and _count, which are the sum of all values for that histogram, and the count of all instances of that histogram respectively.