How to Plan LLM Inference Capacity for a Shared Platform
Turn product scenarios, workload traces, latency SLOs, and failure requirements into an evidence-based GPU capacity plan.
#performance
Monitoring vLLM in Production: Metrics, PromQL, Alerts, and Runbooks
A production guide to monitoring vLLM 0.23.x with Prometheus and Grafana, including key metrics, PromQL, alerts, and runbooks.
Why vLLM Scales: Paging the KV-Cache for Faster LLM Inference
How vLLM improves LLM serving efficiency with PagedAttention, better KV-cache utilization, higher throughput, and steadier latency.