You will be able to
- Calculate a derivative of a quadratic.
- Predict a small loss change using its local slope.
A local sensitivity
A derivative measures how a function changes per unit change of its input near a point. A secant uses two distinct points; a tangent is their limiting local slope. For several parameters, the gradient collects the partial derivatives, each with the other parameters held fixed.
Walk downhill carefully
For L(w)=(w−3)² the derivative is 2(w−3). At w=0 it is negative, so increasing w reduces loss locally. The negative gradient points downhill, but an oversized step can jump past a minimum. Later lessons will apply the same idea to many model weights.
Worked example
At w=1, L=4 and slope=−4. Increasing w by 0.01 predicts ΔL≈−0.04; the actual change is (−1.99)²−4=−0.0399.
A common trap
A derivative is local information, not a guarantee that an arbitrarily large move improves the function.
Predict, then experiment
Starting at zero, should the first step toward lower loss increase or decrease w? Before touching the controls, say or write your prediction.
Try it yourself
Watch gradient descent
When does a downhill step overshoot?
Exact iterations of L(w)=(w−3)², gradient 2(w−3), for 16 full-batch steps. Solid curve is parameter w; dashed is the optimum 3. This quadratic is not a neural-training benchmark.
Can you make it happen?
Start at 0 or below and choose a learning rate that gives loss below 0.001 after 16 updates.
Predict a setting before you try it. This optional challenge checks the current parameters, not lesson completion.
A hint
For this quadratic, the error is multiplied by 1−2η each step. Very small steps are slow; steps above 1 are unstable.
One solution & why it works
Start 0 and learning rate 0.5 reach w=3 in one step. Many other rates between 0 and 1 also meet the 16-step target.
Your experiment
Use start 0 and learning rate 0.1. Inspect the parameter sequence. Try 1.2 and explain why a locally correct direction can still diverge.
Change one parameter at a time. Compare what you observed with your prediction and record an explanation in your notes.
Check your understanding
Try from memory first. Hints and feedback are part of learning, not a penalty. These checks are for self-study, not a graded qualification.
Reference answers (after your attempt)
- toward larger w — Correct: the derivative is −6.
- 4 — 2(5−3)=4.
Apply it: Check a derivative numerically
Use paper, your notes, or a local notebook/editor. Code examples are Python 3 starting points to run yourself; this page does not execute code or automatically assess the practical task.
- Evaluate L at w=1 and w=1.001.
- Divide the difference by 0.001.
- Compare with the analytical derivative and explain the small error.
Reference code / starting point
def loss(w):
return (w - 3)**2
w, eps = 1.0, 0.001
print((loss(w + eps) - loss(w)) / eps)Reference solution & review criteria
The finite-difference slope is −3.999, versus derivative −4. The finite interval includes quadratic curvature.
Your notebook
Notes and progress stay in this browser. They are not sent to the server; ordinary web-server request logs still operate. Export a backup from the course page.
Completion requires both correct checks and your practical-task acknowledgement.
Go deeper
Sources support further study; the worked examples and questions here are original. Browser experiments are small, disclosed models—not evidence of production-model quality or hardware performance.