Hidden Costs of LiteLLM's Health Checks
How LiteLLM model health checks generated unexpected external-model spend, and how we kept checks for local models only.
#llm-inference
How to Plan LLM Inference Capacity for a Shared Platform
Turn product scenarios, workload traces, latency SLOs, and failure requirements into an evidence-based GPU capacity plan.
Kubernetes Model Serving in 2026: What Changed Since 2024
An update on OCI model volumes, inference-aware routing, KServe, llm-d, multi-node inference, GPU scheduling, and the remaining gaps.
Monitoring vLLM in Production: Metrics, PromQL, Alerts, and Runbooks
A production guide to monitoring vLLM 0.23.x with Prometheus and Grafana, including key metrics, PromQL, alerts, and runbooks.
Why vLLM Scales: Paging the KV-Cache for Faster LLM Inference
How vLLM improves LLM serving efficiency with PagedAttention, better KV-cache utilization, higher throughput, and steadier latency.