IV.08 / AI
Kernel methods
When does a similarity function let a classifier recover a boundary missed in the original coordinates?
Start with the question.
Kernel methods allow a learning procedure to use similarities corresponding to a richer feature space without always constructing that space explicitly. Cortes and Vapnik's support-vector work gives a concrete historical setting for studying margins, errors, and generalization. The useful comparison holds the data split fixed while varying the representation and the control on model flexibility.
Make a nonlinear boundary visible.
Place four points at (−1,−1), (−1,1), (1,−1), and (1,1) and give diagonally opposite corners the same class. A straight line in the original plane cannot separate the two classes. Add the feature x·y; its sign separates the pattern. The example shows how a representation can change a classification problem before any claim about a particular kernel's performance.
What would you inspect?
- Feature scaling, kernel choice, regularization, and support-vector count.
- Performance on an untouched test set and sensitivity to outliers.
Take the next step.
Compare a linear model and one kernel model on a small documented dataset, using the same preprocessing and validation split.
Read the supporting material.
- Support-vector networks
Corinna Cortes and Vladimir Vapnik · Support-vector classification and benchmark comparison.
