▲ ALGORITHM
Attention mechanisms, optimization, quantization, and the mathematical foundations of modern ML.
Context Window Management for Multi-Agentic Platforms
Eight strategies ordered by leverage-per-complexity — from prompt caching to bandit-scored tool selection — with the economics, code, and latency budget for each.
Flash Attention: A Complete Architectural Autopsy
From O(N²) HBM catastrophe to Hopper-native warp-specialized pipelines: every trick, every number, every tradeoff decoded. Standard attention is not bottlenecked by arithmetic — it is bottlenecked by memory bandwidth.
Positional Encoding in Transformers: A First-Principles Engineering Treatise
Self-attention is blind to token order by design. From sinusoidal foundations through RoPE, ALiBi, and YaRN, every positional encoding makes a different tradeoff between in-context position and out-of-distribution length generalization.
Attention's Memory Problem Has Three Solutions: MQA, GQA, and MLA
At 32K context and batch 32 in FP16, Llama-3-70B's KV cache exceeds 137 GB — more than an H100's HBM. MQA, GQA, and MLA compress that cache architecturally, each making a different quality-memory tradeoff.
Speculative Decoding: The Physics of Stolen Time in Autoregressive Inference
A large model verifies tokens in roughly the same wall-clock time it generates one. Speculative decoding weaponizes this asymmetry: measurable, provable speedup without changing the probability distribution the target model assigns.
Beyond Vanilla Speculative Decoding: Medusa, Eagle, and Lookahead
Vanilla speculative decoding requires a separately trained and versioned draft model — one that must be realigned every time the target is fine-tuned. Medusa, Eagle, and Lookahead each eliminate that operational burden differently.
Mixture-of-Experts Internals: A Systems Engineer's Field Manual
MoE decouples total parameters from per-token compute — DeepSeek-V3 uses 37B of 671B parameters per token, an 18:1 ratio dense models cannot match. The router, AllToAll dispatch, and aux-loss design are where that leverage is won or lost.
The Inference Engineer's Definitive Guide to Quantization
LLM decode is memory-bandwidth-bound: loading 140 GB of FP16 weights takes 42ms at H100 bandwidth regardless of batch size. Cutting to INT4 quadruples your throughput ceiling and eliminates multi-GPU tensor parallelism on 70B models.
The LoRA Family: A Mathematically Precise Guide to Parameter-Efficient Fine-Tuning
Fine-tuning weight updates concentrate in a low-rank subspace of the d×d parameter matrix. LoRA exploits this to reduce trainable parameters by 10,000× per layer, from the foundational algebra through QLoRA and DoRA.
The Modern Alignment Landscape: A Sharp Comparative Analysis for ML Engineers
RLHF requires four models in memory simultaneously and is notoriously hyperparameter-sensitive. DPO eliminates the reward model algebraically. ORPO eliminates the reference model. KTO works with binary feedback alone.
Engineering the Infinite Context Window: A Systems-Level Guide to 128K+ Token Models
Extending to 1M tokens hits three physical walls: quadratic attention compute, linear KV cache memory (327 GB for Llama-3-70B at 1M tokens), and positional encoding extrapolation. Each demands a different class of solution.
