A Review of Hands-On Machine Learning with Scikit-Learn, Keras & Tensorflow by Aurélien Géron
Summary
Support Vector Machines (SVMs) are a type of Machine Learning Model that use linear separability to create decision boundaries. SVMs do this by attempting to create a “street” between the different categories of data. The above image illustrates a “street” created by the Linear SVM in the above image that separates out the two classes represented (the blue and green dots).
SVMs are not just a powerful method for linear classification but can also be used for non-linear classification as well as regression. This can be done by changing the kernel that sklearn uses upon the creation of the model.
Major Takeaways
SVMs are sensitive to feature scales so it is essential to use some form of scaling to improve the powers of the model we create.
Different forms of SVMs have different time complexities ranging from O(m x n) to O(m³ x n) time. Due to this variance in time complexity we have to be very careful with what algorithm we use. LinearSVC and SGDClassifier both run in O(m x n) time while SVC is significantly slower running in O(m² x n) to O(m³ x n) time.
SVMs can be tuned with the hyperparameter C and depending on the kernel gamma (γ). C is used as a regularization term which means that it dictates the amount that each individual data point influences the decision boundary. Gamma is the hyperparameter when a SVM uses a Gaussian (non-linear) kernel. Gamma, like C, also focuses on how much each individual point effects the decision boundary in terms of how linear the model is. A low γ creates a smoother more regular Decision Boundary while a high γ created an irregular decision boundary.
My Thoughts
Support Vector Machines are a powerful and diverse method for machine learning. Having relatively little experience building SVMs I chose to skim over the “Under the Hood” portion of the chapter but I am excited to learn more about the mathematics that support SVMs as I’m not a fan of just assuming that things work without understanding why they work.
Working through the exercises for this chapter refreshed my understanding of SVMs and reminded me how strong the hyperparameters are in over/underfitting. Having done some further research I saw that it was recommended to use a gridsearch to attempt to attain the hyperparameters for the specific problem.
Thanks for reading!
If you have any questions or feedback please reach out to me on twitter @wtothdev or leave a comment!
Additionally, I wanted to give a huge thanks to Aurélien Géron for writing such an excellent book. You can purchase said book here (non-affiliate).
Disclaimer: I don’t make any money from any of the services referenced and chose to read and review this book under my own free will.
