How to Plan LLM Inference Capacity for a Shared Platform
Turn product scenarios, workload traces, latency SLOs, and failure requirements into an evidence-based GPU capacity plan.
I lead the architecture of Severstal’s shared platform for serving open-weight models on Kubernetes, vLLM, and NVIDIA GPUs.
Focus Control planes · Performance · Reliability · Observability · GPU capacity
Selected writing
Kubernetes Model Serving in 2026: What Changed Since 2024
An update on OCI model volumes, inference-aware routing, KServe, llm-d, multi-node inference, GPU scheduling, and the remaining gaps.
Monitoring vLLM in Production: Metrics, PromQL, Alerts, and Runbooks
A production guide to monitoring vLLM 0.23.x with Prometheus and Grafana, including key metrics, PromQL, alerts, and runbooks.
Latest writing
My Modern C++ Setup on macOS and Apple Silicon
Set up a modern C++ toolchain on Apple Silicon macOS with LLVM, CMake, Ninja, VS Code, Helix, and mise.
Running Kubernetes with a GPU Inside WSL2 on My RTX Laptop
Run Kubernetes with NVIDIA GPU support inside WSL2 on a laptop, from k3s setup and validation to the limits of local GPU workloads.
Helix for Python Development
Set up Helix for Python development with LSP, ty type checking, Ruff formatting and linting, plus a few editor refinements.
Why vLLM Scales: Paging the KV-Cache for Faster LLM Inference
How vLLM improves LLM serving efficiency with PagedAttention, better KV-cache utilization, higher throughput, and steadier latency.
uv: A Cargo-Like Python Tool That Replaces pipx, pyenv, and More
A concise uv cheat sheet for managing Python versions, environments, dependencies, tools, and scripts in one workflow.
Identifying Vulnerable Dependencies in .NET Projects
Scan .NET dependencies for known NuGet vulnerabilities and automate checks in GitLab CI to improve supply chain security.