Signals to learning / 6. Learn from data

Lesson 21 of 40 · Intermediate · About 30 minutes

Features and linear regression

What exactly does a model learn from examples?

You will be able to

  • Distinguish features, targets and learned parameters.
  • Calculate predictions, residuals and mean squared error.

Represent the task

A supervised example pairs input features x with a target y. The model uses parameters to map features to a prediction. For one feature, ŷ=wx+b: slope w and intercept b are learned parameters, not extra measurements. Feature design should reflect what is available at prediction time.

Turn error into an objective

A residual is ŷ−y. Squared error penalises larger residuals more strongly, and mean squared error averages over examples. Minimising training loss fits this dataset; it does not establish predictive accuracy on a new population. Outliers, feature scaling and an unsuitable linear model can all limit the fit.

ŷᵢ = wxᵢ+b; MSE = (1/N)Σᵢ(ŷᵢ−yᵢ)²

Worked example

For x=[0,1], y=[1,3] and w=1,b=1, predictions are [1,2], residuals [0,−1] and MSE=0.5. Setting w=2,b=1 fits these two examples exactly.

A common trap

A learned coefficient is not automatically a causal effect; observational correlations and confounding can produce convincing predictive fits.

Predict, then experiment

Does changing intercept rotate the line or shift it vertically? Before touching the controls, say or write your prediction.

Try it yourself

Fit a straight-line model

Which slope and intercept reduce training error?

Five fixed synthetic training examples; circles are targets, solid is wx+b. No test set is shown. Units are arbitrary; MSE measures fit to these five points only.

Can you make it happen?

Find a line with training MSE below 0.15 on these five points.

Predict a setting before you try it. This optional challenge checks the current parameters, not lesson completion.

A hint

Adjust the slope first, then shift the intercept. Read the error rather than judging only by the plot.

One solution & why it works

Slope 2 and intercept 1 give MSE 0.084. This is a training-fit result, not a claim about unseen examples.

Your experiment

Start with w=1,b=0. Change only b, then only w. Try to reduce the five-point training MSE; write why the best straight line may still have nonzero error.

Change one parameter at a time. Compare what you observed with your prediction and record an explanation in your notes.

Check your understanding

Try from memory first. Hints and feedback are part of learning, not a penalty. These checks are for self-study, not a graded qualification.

1. In ŷ=wx+b, which values are normally learned?
Give me a hint

Distinguish data from adjustable parameters.

2. For residuals [1,−1,2], what is MSE?

Accepted absolute tolerance: ±0.001 squared target units.

Give me a hint

Square each residual, then average.

Reference answers (after your attempt)
  1. w and b — Correct: those are model parameters.
  2. 2squared target units — (1+1+4)/3=2.

Apply it: Build a baseline before a neural network

Use paper, your notes, or a local notebook/editor. Code examples are Python 3 starting points to run yourself; this page does not execute code or automatically assess the practical task.

  1. Choose a numeric target and features available at prediction time.
  2. Define a constant baseline and a linear baseline.
  3. State the held-out metric and one non-causal interpretation risk.
Reference solution & review criteria

For duration prediction, a training-mean constant predictor is a baseline; linear regression may use known acquisition features. Compare held-out MSE, with grouping if relevant. Coefficients alone do not show interventions cause outcomes.

Your notebook

Notes and progress stay in this browser. They are not sent to the server; ordinary web-server request logs still operate. Export a backup from the course page.

Review list

Completion requires both correct checks and your practical-task acknowledgement.

Go deeper

Sources support further study; the worked examples and questions here are original. Browser experiments are small, disclosed models—not evidence of production-model quality or hardware performance.