Sign-Based Optimizers

1 paper touches this idea.

Papers

Nobody Has Proven the World's Favorite AI Optimizer Actually Works

Related concepts

Ideas that show up alongside Sign-Based Optimizers.

AdamW Convergence Analysis Adaptive Gradient Methods Convergence Lower Bounds Large Language Model Training Optimization Under Heavy-Tailed Noise Stochastic Optimization Theory