In this article, we will be learning various things about the SVM. We will look at code samples to understand the algorithm. Also, we will be studying about libraries with which we can design an SVM and many more things.<\/p>\n
So let us begin.<\/p>\n
What is SVM in Machine Learning?<\/h3>\n
An SVM is a classification based method or algorithm. There are some cases where we can use it for regression. However, there are rare cases of use in unsupervised learning as well.<\/p>\n
SVM in clustering is under research for the unsupervised learning aspect. Here, we use unlabeled data for SVM. Since the topic is under research, we will only look at what it means.<\/p>\n
In regression, we call the concept SVR or support vector regression<\/strong>. It is quite similar to SVM with only a few changes. However, it is more complicated than SVM.<\/p>\n Now, we come to SVM. It is a strong data classifier.<\/p>\n The support vector machine uses two or more labelled classes of data. It separates two different classes of data by a hyperplane. The data points based on their position according to the hyperplane will be put in separate classes.<\/p>\n In addition, an important thing to note is that SVM in Machine Learning always uses graphs to plot the data. Therefore, we will be seeing some graphs in the article.<\/p>\n Now, let\u2019s learn some more stuff.<\/p>\n To understand SVM mathematically, we have to keep in mind a few important terms. These terms will always come whenever you use the SVM algorithm. So let\u2019s start looking at them one by one.<\/p>\n Support vectors are special data points in the dataset. They are responsible for the construction of the hyperplane and are the closest points to the hyperplane. If these points were removed, the position of the hyperplane would be altered. The hyperplane has decision boundaries around it.<\/p>\n The support vectors help in decreasing and increasing the size of the boundaries. They are the main components in making an SVM.<\/p>\n We can see the picture for this. The yellow and green points here are the support vectors. Red and blue dots are separate classes.<\/p>\n The middle dark line is the hyperplane in 2-D and the two lines alongside the hyperplane are the decision boundaries. They collectively form the decision surface.<\/p>\n According to the SVM algorithm we discover the points closest to the road from both the classes. These points are called support vectors. Decision boundaries in SVM are the two lines that we see alongside the hyperplane.<\/p>\n The distance between the two light-toned lines is called the margin. An optimal or best hyperplane form when the margin size is maximum. The SVM algorithm adjusts the hyperplane and its margins according to the support vectors.<\/p>\n The hyperplane is the central line in the diagram above. In this case, the hyperplane is a line because the dimension is 2-D. If we had a 3-D plane, the hyperplane would have been a 2-D plane itself. There is a lot of mathematics involved in studying the hyperplane.<\/p>\n We will be looking at that. But, to understand a hyperplane we need to imagine it first.<\/p>\n Imagine there is a feature space (a blank piece of paper)<\/strong>. Now, imagine a line cutting through it from the center. That is the hyperplane.<\/p>\n The math equation for the hyperplane is a linear equation.<\/p>\n a0<\/sub> + a1<\/sub>x1<\/sub> + a2<\/sub>x2<\/sub> + \u2026\u2026. + an<\/sub>xn<\/sub><\/strong><\/p>\n This is the equation.<\/p>\n Here a0 is the intercept of the hyperplane. Also, a1 and a2 define the first and second axes respectively. X1 and X2 are for two dimensions.<\/p>\n Let us assume that the equation is equal to E. So if the data points lie beneath the hyperplane then E<0. If they are above it, the E>=0. This is how we classify data using a hyperplane.<\/p>\n In any ML method, we would have the training and testing data. So here we have n*p matrix which has n observations and p dimensions.<\/p>\n We have a variable Y, which decides in which class the points would lie. So, we have two values 1 and -1. Y can only be these two values in any case.<\/p>\n If Y is 1 then data is in class 1. If Y is -1 then data is in class -1.<\/p>\n Based on all this information, we have a small code here. This code will perfectly explain the conditions above.<\/p>\n We will use only basic libraries for now, as it is only an example. Here we are using NumPy for math operations on the array of data.<\/p>\n Also, we are using matplotlib for plotting the graph.<\/p>\n Here we have \u2018a\u2019 variable that holds the array of data. This data is the coordinates of the points on the dataset. Here, \u2018b\u2019 is the data specifying in which class the points lie.<\/p>\n The program plots the data as \u2018*\u2019 for one class and \u2018o\u2019 for another. Based on the \u2018if\u2019 condition it places the datapoints above or below the hyperplane.<\/p>\n The sample data helps the algorithm to plot the data. \u2018s\u2019 is the length of the hyperplane. Linewidth is the thickness of the hyperplane.<\/p>\n All of this data runs in a \u2018for\u2019 loop which takes in data from \u2018a\u2019.<\/p>\n The last line of code plots the hyperplane. The coordinates of the plot function are the coordinates of the two end-points of the hyperplane.<\/p>\n The result of this code in graphical<\/strong> format is shown below.<\/p>\n The hyperplane has many concepts like the maximum margin classifier<\/strong> that is used in setting the maximum margin of the plane. The maximum margin classifier helps to adjust the hyperplane and the decision boundaries.<\/p>\n Still, there can be cases where data can be indistinguishable and hence, where we cannot draw a hyperplane.<\/p>\n Here, we could use quadratic or cubic equations to define the problem. The concept used here is the support vector classifier. Here, we try to solve things by increasing the size of feature space by using kernel tricks and all that.<\/p>\n Kernel trick<\/strong> is the inner product of two support vectors that increases the size of the feature space.<\/p>\n We increase the feature space for displaying the non-linear equation results, which can be curves and all. This is all for data which is inseparable and to which hyperplane is not possible.<\/p>\n This was all about the hyperplane<\/strong>.<\/p>\n At last, we can discuss where we can use SVM in Machine Learning. It can be used in places like image classification. We can use it in biometrics. It can be used to design security codes. We can use it for protein classification and there are many more uses to it.<\/p>\n For this, let us compare the working of SVM and other classifiers for better understanding. Let us talk about SVMs and perceptrons.<\/p>\n Perceptrons<\/strong> and other classifiers focus on all the points that are present in the data. For these classifiers, the focus is more on just separating the complex points and adjusting the dividing line.<\/p>\n Perceptrons are made by taking one point at a time and fixing the dividing line accordingly. When all the complex points are separated, the perceptron algorithm stops.<\/p>\n This is the end of the process for these classifiers, as they do not improve the position of the dividing line. So, finding the optimal dividing line is not what the perceptron does.<\/p>\n Whereas, the case with SVM is a little different. In this, the algorithm only focuses on the points that are complex to separate and it ignores the rest of the points.<\/p>\n The algorithm finds the points that are closest to each other. It then draws a line<\/strong> between them. The dividing line would be optimal if it is perpendicular to the line connecting the points. The best part is that two different classes are formed on either side of the line. Whatever new point enters the dataset, it won\u2019t be affecting the hyperplane.<\/p>\n The only points that would affect the hyperplane are the support vectors. The hyperplane won\u2019t allow the data from both classes to mix in most cases.<\/p>\n Also, the hyperplane can adjust itself by maximizing the size of its margin. The margin is the space between the hyperplane and the decision boundaries.<\/p>\n This is how the SVM in Machine Learning works.<\/p>\n Machine Learning as we know can be programmed with various languages and Python<\/strong> is one of them. We generally prefer Python as it is relatively easier to code with than other languages like Java. Now let\u2019s look at how it is implemented in Python.<\/p>\n SVM in Machine Learning can be programmed using specific libraries like Scikit-learn. We can also use simpler libraries like pandas, NumPy, and matplotlib. We can understand this with some codes.<\/p>\n Note:<\/strong> If you are doing this on Google colab, you need to first upload the dataset from your drive to Google colab. This is shown in the link below.<\/p>\n Dataset: Implementation of SVM in Python<\/strong><\/p>\n 1.<\/strong> First, we import the libraries.<\/p>\n 2.<\/strong> Now, we import datasets.<\/p>\n 3.<\/strong> After importing the data, we can view the data by applying some basic operations.<\/p>\n In this step, we explore the data and analyze it.<\/p>\n This is the output of data analysis. 4.<\/strong> The next step is a very crucial step in SVM in Machine Learning. Here, we perform Data Preprocessing. This step divides the data into attributes and labels data.<\/p>\n 5.<\/strong> In preprocessing, we divide the attributes and labels into two variables. Here, we take a particular column from the data. The column is dropped from the first variable. The second variable now holds the dropped column.<\/p>\n 6.<\/strong> Now, after the data gets separated into attributes and labels, we arrive at the final preprocessing step. Now, we use the Scikit-learn library for dividing data into training and testing data.<\/p>\n 7.<\/strong> The above step shows that the train_test_split<\/strong> method is a part of the model_selection<\/strong> library in Scikit-learn. Using this, we will divide the data. When we run this command, the data gets divided.<\/p>\n 8.<\/strong> Now, the next step is training your algorithm.<\/p>\n 9.<\/strong> The training of data is done by using the SVM library. This library has built-in functions and classes for various SVM algorithms. We also use a library for classification. This library is SVC or support vector classifier<\/strong> class. We have to specify the type of kernel we are using in this class.<\/p>\n The code snippet is mentioned above, whereas the image of the output of this snippet is given below. 10.<\/strong> The fit method of the SVC class used above trains the algorithm.<\/p>\n 11.<\/strong> After training, we use the predict method to give predictions.<\/p>\n 12.<\/strong> The last step is evaluating the algorithm and finding the result for the SVM.<\/p>\n This is the output that is given in image format.Parts of SVM in Machine Learning
![\"parts](\"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/sites\/2\/2020\/03\/parts-of-SVM-in-machine-learning.jpg\")
<\/a><\/h3>\n1. Support Vectors<\/h4>\n
<\/a><\/p>\n2. Decision Boundaries<\/h4>\n
3. Hyperplane<\/h4>\n
import numpy as np\nfrom matplotlib import pyplot as plt<\/pre>\n
a = np.array([[0,3,-1],\n [2,2,-1],\n [2, 5, -1],\n [2, 4, -1],\n [4, 5, -1]\n ])\n\nb = np.array([-1,-1,1,1,1])\n\nfor i, sample in enumerate(a):\n if i < 2:\n plt.scatter(sample[0], sample[1], s=150, marker='*', linewidths=3)\nelse:\n plt.scatter(sample[0], sample[1], s=150, marker='o', linewidths=3)\n\nplt.plot([-2,6],[5,1.5])\n<\/pre>\n
<\/a><\/p>\nHow Does an SVM Work?<\/h3>\n
Implementation of SVM in Python<\/h3>\n
import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt<\/pre>\n
data = pd.read_csv('creditcard.csv')<\/pre>\ndata.head()<\/pre>\n
<\/a><\/p>\nX = data.drop('Class', axis=1)\ny = data['Class']<\/pre>\nfrom sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20)<\/pre>\n
from sklearn.svm import SVC\nsvclassifier = SVC(kernel='linear')\nsvclassifier.fit(X_train, y_train)<\/pre>\n
<\/a><\/p>\ny_pred = svclassifier.predict(X_test)<\/pre>\n
from sklearn.metrics import classification_report, confusion_matrix\nprint(confusion_matrix(y_test,y_pred))\nprint(classification_report(y_test,y_pred))<\/pre>\n
<\/a><\/p>\n