📈 Learning Rate Scaling Calculator
Scale a learning rate when changing effective batch size using linear or square-root rules, warmup, gradient accumulation, and world size.
Calculate Learning Rate Scaling
What this learning rate scaling calculator calculates
Suggest a starting learning rate after changing effective batch size. This is a heuristic planning tool, not an automated learning-rate range test.
Use the result as an experiment starting point. Optimizer, architecture, data quality, warmup, precision, and loss scale can change the stable range.
Learning Rate Scaling Calculator formula
Assumptions and limitations
The result is an engineering estimate based on the values entered. AI models, tokenizers, runtimes, accelerators, cloud services, and provider billing rules differ. Validate important decisions with measured data from the exact model, hardware, framework, and pricing plan you intend to use.
Worked example
Moving from effective batch 256 to 1,024 scales a 0.0001 learning rate to 0.0004 with the linear rule or 0.0002 with the square-root rule.
How to use the result
Use the result as an experiment starting point. Optimizer, architecture, data quality, warmup, precision, and loss scale can change the stable range.
- Start with representative production assumptions rather than best-case demos.
- Run a low, expected, and high scenario to understand the range.
- Record model version, pricing date, hardware, precision, and workload details.
- Replace assumptions with observed p50 and p95 measurements after testing.
Common mistakes to avoid
- Calling a heuristic formula a true learning-rate finder.
- Using microbatch size instead of effective global batch size.
- Scaling aggressively without warmup, monitoring loss, or testing stability.
Frequently asked questions
What is effective batch size?
Microbatch per device × number of devices × gradient accumulation steps.
When is linear scaling used?
It is a common starting heuristic when increasing global batch size, especially in large-batch training.
Why use square-root scaling?
It is more conservative and may be safer when linear scaling destabilizes optimization.
Related AI calculators
Methodology and privacy
This educational tool uses the formula and assumptions displayed on the page. Calculations run locally in your browser, and the page does not transmit the values you enter. Results are estimates rather than provider quotes, benchmark guarantees, financial advice, or capacity guarantees. Last methodology review: August 2, 2026.
Formula Explorer connections
Interpretation: This relationship turns dataset size, batch structure or parallel capacity into a training and evaluation planning quantity. Assumption: Real results depend on data quality, hardware utilization, communication overhead, convergence behavior and the exact experimental design.