Monitoring vLLM in Production: Metrics, PromQL, Alerts, and Runbooks
A production-oriented guide to monitoring vLLM 0.23.x with Prometheus and Grafana: latency, queueing, preemption, KV-cache pressure, throughput, alerting, and incident diagnosis.
Andrey Krisanov
Production LLM inference, AI infrastructure, and distributed systems