AIERALearn how AI models actually work, one layer at a time.
AIERA is a weekly blog for people who want more than headlines about AI. Each piece explains one idea in machine learning with real numbers, small code you can run, and diagrams you can explore.
Attention, step by step: how a transformer decides what to look at
Queries, keys and values sound abstract until you follow a few tokens through the math by hand.
Latest articles
Gradient descent, felt rather than memorized
Build intuition for slopes, step sizes and why the loss sometimes explodes, before touching any framework.
Why your validation loss went up while training loss kept falling
Overfitting, explained with one small dataset and three fixes you can try today.
What the first layer of a CNN learns to see
Visualize trained filters and you find edge and colour detectors, much like the early stages of the visual cortex.
Tokenizers are where your model's quirks start
Byte-pair encoding explains why language models struggle to count letters and do arithmetic on long numbers.
Reward is not the goal: notes on reward hacking
When an agent maximizes the number you gave it rather than the behaviour you meant.
The learning rate decides whether training ever settles
Gradient descent moves a parameter a little way downhill on every step. The learning rate sets how far. Here the loss is a simple bowl, so the best value of x is 0.
Try a tiny rate, then something near 0.5, then push past 1.0. On this bowl every step multiplies x by (1 − 2η), so any rate above 1.0 makes each step overshoot further than the last.
Read the full article on gradient descentLearning paths
Read these in order. Each step builds on the one before it.
Zero to neural networks
For readers comfortable with high-school algebra.
- Gradient descent, felt rather than memorized
- Linear regression as a one-neuron network (coming soon)
- Why your validation loss went up while training loss kept falling
- Backpropagation with a pencil (coming soon)
- What the first layer of a CNN learns to see
Understanding language models
From text to tokens to attention to evaluation.
- Tokenizers are where your model's quirks start
- Embeddings as coordinates for meaning (coming soon)
- Attention, step by step: how a transformer decides what to look at
- Pretraining versus fine-tuning (coming soon)
- Evaluating LLM outputs without fooling yourself
Machine learning in production
What changes once real users depend on your model.
- Versioning data alongside code (coming soon)
- Serving a model behind an API (coming soon)
- Shipping a model is the easy part: watching for drift
- Reward is not the goal: notes on reward hacking