Neural Networks
Multi-Layer Perceptron
Add or remove hidden layers/neurons and watch the decision boundary go from a straight line to a complex curved region in real time.
How hidden layers and non-linear activations warp the feature space to separate complex data that a single line cannot.
Stage 1 of 4: Data & Architecture
- Positive Class
- Negative Class
- Decision Region
The dataset mapped out in feature space, and the network architecture.
Multi-Layer Perceptron
A single neuron can only draw a straight line. To separate complex, curving data, you need multiple neurons working together in hidden layers.
But just adding layers isn't enough. If every neuron only does linear math, combining them just creates another straight line. To bend the feature space, we need an activation function — a non-linear spark that lets the network warp its decision boundary to fit the data.
The Architecture
- Hidden Layers: Depth gives the network capacity to combine simple boundaries into complex ones.
- Neurons per Layer: Width gives each layer more components to work with.
- Activation: The non-linearity (like ReLU or Tanh) that bends the straight lines.
Play with the architecture. Watch how changing the number of neurons and layers affects the shape of the decision boundary. Then, try setting the activation to linear and watch the network lose its power, proving that depth alone without non-linearity achieves nothing.
Break it on purpose
Setting activation to linear on non-linear data. The boundary stays a straight line no matter how many layers are added, proving that deep linear networks collapse to a single linear transformation.