Ask AIERA

Your friendly guide to this blog. Ask a question in your own words and I’ll find the best articles and terms for you.

I search AIERA-TS articles and the glossary, right here in your browser.

Concept of the week

The learning rate decides whether training ever settles

Gradient descent moves a parameter a little way downhill on every step. The learning rate sets how far. Here the loss is a simple bowl, so the best value of x is 0.

x ← x − η · dL/dx    where L(x) = x²

Try a tiny rate, then something near 0.5, then push past 1.0. On this bowl every step multiplies x by (1 − 2η), so any rate above 1.0 makes each step overshoot further than the last.

Read the full article on gradient descent
Slow

Learning paths

Read these in order. Each step builds on the one before it.