Радар на почту

My Gradient Clipping Was Never Clipping. It Was Secretly Multiplying My Weight Decay

DEV · 04.11.2026 · 23:45

Yesterday's experiment ended with a clean verdict: LR warmup is insurance — free at the tuned learning rate, a full payout at an overheated one. Gradient clipping has the exact same…

Читать в источнике