You will be able to
- Construct a group-aware feature/classification pipeline.
- Report decision metrics without tuning on the final test.
A manageable synthetic task
Treat each recording as one example of a low-frequency or high-frequency tone. Extract energy in a low and high frequency band using the DFT skills from module 4. Introduce variation in amplitude, phase and interference. Keep any windows derived from the same recording together. Start with a transparent band-energy score before considering a trained neural model.
Fit the process, not the test
Fix recording groups into train, validation and test before fitting scalers or choosing bands/thresholds. A logistic-regression extension fits weights using training cross-entropy; a simpler score threshold can be selected on validation. Freeze the entire feature and threshold policy before final test. If you make a new choice after seeing the test, declare that set used for selection and obtain another assessment set.
Audit failure and deployment
Report TP,FP,FN,TN, precision, recall, class support and the baseline. Test amplitude change and a new noise spectrum. Synthetic tones do not establish performance on speech, music or real sensors. Document the data generator and known limitations. The reference code supplies a complete synthetic feature and logistic-regression pipeline. Extend it with failure tests; the browser probability lab does not train that model.
Worked example
For TP=8,FP=2,FN=4,TN=10, precision=0.8, recall≈0.667 and accuracy=18/24=0.75. The correct metric depends on the application, not which one is numerically largest.
A common trap
Overlapping windows from the same signal are not independent recordings. Randomly splitting them can inflate scores.
Predict, then experiment
Will a threshold chosen for balanced classes necessarily be appropriate for a rare-positive deployment? Before touching the controls, say or write your prediction.
Try it yourself
From a logit to probability
How does confidence affect cross-entropy?
Curve is sigmoid(z). The circle marks the adjustable logit; target is 0 or 1. Binary cross-entropy uses natural logarithms and a numerically stable logit formula.
Your experiment
Compare logits −2,0,+2. Write decisions at probability thresholds 0.3,0.5,0.7 and connect each to precision/recall trade-offs; the lab does not generate your final evaluation.
Change one parameter at a time. Compare what you observed with your prediction and record an explanation in your notes.
Check your understanding
Try from memory first. Hints and feedback are part of learning, not a penalty. These checks are for self-study, not a graded qualification.
Reference answers (after your attempt)
- validation data — Correct: hold the final test out of selection.
- 0.75 — (8+10)/(8+2+4+10)=18/24=0.75.
Apply it: Deliver a dataset and evaluation card
Use paper, your notes, or a local notebook/editor. Code examples are Python 3 starting points to run yourself; this page does not execute code or automatically assess the practical task.
- Generate at least 30 independent synthetic recordings per class with documented variation and fixed reproducibility. Split by recording; keep derived windows grouped.
- Implement DFT band-energy features and a simple baseline; select thresholds on validation only. Optionally compare training-only logistic regression.
- Freeze the pipeline, report final confusion counts and failure cases, and explain why synthetic success does not establish real-world performance.
Reference code / starting point
from math import sin, cos, pi, log, exp, sqrt
from random import Random
rng = Random(7)
N, fs = 64, 64
def band_energy(x, bins):
return sum(
(sum(v*cos(2*pi*k*n/N) for n, v in enumerate(x)))**2
+ (sum(-v*sin(2*pi*k*n/N) for n, v in enumerate(x)))**2
for k in bins) / N**2
def make_record(label, recording_id):
frequency = (3 if label == 0 else 13) + rng.uniform(-0.4, 0.4)
amplitude, phase = rng.uniform(0.2, 1.8), rng.uniform(-pi, pi)
x = [amplitude*sin(2*pi*frequency*n/fs+phase)
+ 0.2*sin(2*pi*19*n/fs) + rng.uniform(-0.15, 0.15)
for n in range(N)]
low, high = band_energy(x, range(1, 8)), band_energy(x, range(9, 18))
return (recording_id, log((high+1e-8)/(low+1e-8)), label)
train, validation, test = [], [], []
for label in (0, 1):
records = [make_record(label, str(label)+"-"+str(i)) for i in range(30)]
rng.shuffle(records)
train += records[:18]
validation += records[18:24]
test += records[24:]
# Every recording contributes one example, and IDs are disjoint.
assert not (set(r[0] for r in train) & set(r[0] for r in test))
mean = sum(r[1] for r in train)/len(train)
scale = sqrt(sum((r[1]-mean)**2 for r in train)/len(train)) or 1
def feature(record):
return (record[1]-mean)/scale
def probability(record, w, b):
z = w*feature(record)+b
return 1/(1+exp(-z)) if z >= 0 else exp(z)/(1+exp(z))
w, b = 0.0, 0.0
for _ in range(200):
errors = [(r, probability(r, w, b)-r[2]) for r in train]
dw = sum(error*feature(r) for r, error in errors)/len(train)
db = sum(error for _, error in errors)/len(train)
w, b = w-0.1*dw, b-0.1*db
def accuracy(records, threshold):
return sum((probability(r,w,b) >= threshold) == bool(r[2])
for r in records)/len(records)
# All selection uses validation, not the final test.
threshold = max((0.2, 0.35, 0.5, 0.65, 0.8),
key=lambda t: accuracy(validation, t))
predictions = [(r[2], int(probability(r,w,b) >= threshold)) for r in test]
tp = sum(y == 1 and p == 1 for y,p in predictions)
fp = sum(y == 0 and p == 1 for y,p in predictions)
fn = sum(y == 1 and p == 0 for y,p in predictions)
tn = sum(y == 0 and p == 0 for y,p in predictions)
print("train/validation/test", len(train), len(validation), len(test))
print("frozen threshold", threshold, "TP FP FN TN", tp, fp, fn, tn)
print("accuracy", (tp+tn)/len(test), "always-negative baseline", 0.5)
print("precision", tp/(tp+fp) if tp+fp else None)
print("recall", tp/(tp+fn) if tp+fn else None)
print("Synthetic tones only; this is not evidence about real audio.")
Reference solution & review criteria
Rubric: explicit generator/seed and group separation; training-only preprocessing; transparent baseline; validation-only tuning; untouched final test; correct supported metrics; amplitude/noise shift check; limitations. No target accuracy is imposed because the generator and task difficulty must be disclosed.
Your notebook
Notes and progress stay in this browser. They are not sent to the server; ordinary web-server request logs still operate. Export a backup from the course page.
Completion requires both correct checks and your practical-task acknowledgement.
Go deeper
Sources support further study; the worked examples and questions here are original. Browser experiments are small, disclosed models—not evidence of production-model quality or hardware performance.