July 2026
githubI enjoyed this project a lot more than the first machine learning one I did (Linear Regression), maybe because of the increased algorithmic complexity. It got me excited about digging deeper on this subject.
A multilayer perceptron trained to classify breast tumor as malignant or benign from the Wisconsin Breast Cancer dataset. No machine learning framework — for ward propagation, backpropagation, and gradient descent are all implemented directly, with Eigen providing only the linear algebra primitives.
The project runs as a small CLI pipeline: split the raw dataset, train a network with configurable depth and hyperparameters, then evaluate it on unseen data — with an interactive comparison mode to overlay multiple training runs.
Feedforward. Each layer computes a weighted sum of its inputs, adds a bias, and applies an activation — ReLU in the hidden layers, softmax on the output layer to turn the final logits into a probability distribution over the two classes.
Backpropagation. The error between prediction and ground truth is propagated backward through the network via the chain rule: each layer works out how much its own weight contributed to the error, then hands the gradient back to the layer before it. Gradients are accumulated over a mini-batch before any weight update happens.
Gradient descent. Weights move by a small step in the direction that reduces the loss, with the learning rate controlling the step size.
| Choice | Rationale |
|---|---|
| ReLU in hidden layers | Faster convergence than sigmoid |
| He Uniform weight initialization | Variance scaled to input size to avoid initialization at zero which does not work well with ReLU |
| Z-score normalization, fit on train only | Puts all features on the same scale without leaking validation statistics into training |
| Cobined softmax + cross-entropy derivative | Simplifies to ŷ - y directly, avoiding the full softmax Jacobian in the backward pass |
binary cross-entropy loss: 0.015279
accuracy: 98.00%
precision: 100.00%
recall: 96.77%
f1 score: 98.36%In this medical context, recall and precision matter more than raw accuracy: a false negative (a missed malignant tumor) is far more costly than a false alarm. This is also why thje network reports precision, recall, and F1 rather than accuracy alone. Adding metrics was a subject requirements for bonus points, but I found I enjoyed iy as it suddenly made the project quite real
An Adam optimizer was implemented alongside mini-batch gradient descent, maintaining par-weight moment estimates to adapt the effective learning rate during training. It converges noticeably faster in the early epochs, though on this small dataset it reaches similar final performance to plain gradient descent.