Hidden Costs of LiteLLM's Health Checks
How LiteLLM model health checks generated unexpected external-model spend, and how we kept checks for local models only.
Hi, I'm Andrey. This is where I write about software, systems, and things I learn while building them.
Selected writing
How to Plan LLM Inference Capacity for a Shared Platform
Turn product scenarios, workload traces, latency SLOs, and failure requirements into an evidence-based GPU capacity plan.
Kubernetes Model Serving in 2026: What Changed Since 2024
An update on OCI model volumes, inference-aware routing, KServe, llm-d, multi-node inference, GPU scheduling, and the remaining gaps.
Monitoring vLLM in Production: Metrics, PromQL, Alerts, and Runbooks
A production guide to monitoring vLLM 0.23.x with Prometheus and Grafana, including key metrics, PromQL, alerts, and runbooks.
Latest writing
My Modern C++ Setup on macOS and Apple Silicon
Set up a modern C++ toolchain on Apple Silicon macOS with LLVM, CMake, Ninja, VS Code, Helix, and mise.
Running Kubernetes with a GPU Inside WSL2 on My RTX Laptop
Run Kubernetes with NVIDIA GPU support inside WSL2 on a laptop, from k3s setup and validation to the limits of local GPU workloads.
Helix for Python Development
Set up Helix for Python development with LSP, ty type checking, Ruff formatting and linting, plus a few editor refinements.
Why vLLM Scales: Paging the KV-Cache for Faster LLM Inference
How vLLM improves LLM serving efficiency with PagedAttention, better KV-cache utilization, higher throughput, and steadier latency.
uv: Managing Python Versions, Dependencies, Tools, and Scripts
A concise uv cheat sheet for managing Python versions, environments, dependencies, tools, and scripts in one workflow.