My Gradient Clipping Was Never Clipping. It Was Secretly Multiplying My Weight Decay
DEV · 04.11.2026 · 23:45
Yesterday's experiment ended with a clean verdict: LR warmup is insurance — free at the tuned learning rate, a full payout at an overheated one. Gradient clipping has the exact same…