![\"R](\"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/sites\/2\/2020\/05\/Random-Forest-in-R.jpg\")
<\/a><\/p>\nR Random Forest<\/h2>\n
Random forest is an ensemble learning technique<\/strong> that means that it works by running a collection of learning algorithms to increase the preciseness and accuracy of the results. Random forest works by creating multiple decision trees for a dataset and then aggregating the results. Before we go study random forest in detail, let\u2019s learn about ensemble methods and ensemble theory.<\/p>\n In machine learning, ensemble learning methods are learning algorithms that use multiple learning techniques and algorithms to achieve better results. Ensemble learning methods take more computation than single algorithms. They trade extra computation for improved predictive performance.<\/p>\n Ensemble methods are supervised learning algorithms as they train on data and then make predictions. They tend to be more efficient and accurate when diversity in the component algorithms is more i.e. a collection of random decision trees is better than a collection of entropy-reducing decision trees.<\/p>\n Ensembles are much more flexible in terms of they can represent than singular learning algorithms. This is due to their function being a combination of their component singular algorithms.<\/p>\n A model in machine learning is said to be overfitted when it is overly dependent on data. Such a model corresponds too closely to a particular dataset and therefore will be incorrect when tested on any other data.<\/p>\n Underfitting is the opposite of overfitting. A model is said to be underfitted when it cannot reliably account for the majority of variance in the data. Such a model cannot make reliable predictions about the data it has been trained on.<\/p>\n Overfitting and underfitting are also called overtraining and undertraining.<\/p>\n Bagging is an abbreviation for Bootstrapping Aggregating. It is a technique that improves the stability and accuracy of regression and classification algorithms. The biggest advantages and reasons to use bagging are that it reduces variance and helps in avoiding overfitting. Bagging is most commonly used for decision tree algorithms.<\/p>\n Given a training set X=x1, x2, x3, . . . , xNwith responses Y=y1, y2, y3 . . . , yN. We take random samples from the training set multiple times and fit trees to them. Let f(xi,yi)be the decision tree for the variables xiand yi. Finally, to predict the results for x’ we can average the result of all the trees fi corresponding to x'( in case of continuous ). If one or more features are very strong predictors of the response variable, these features will be in many of the trees which will make these trees correlated. The random forest method employs a technique called feature bagging. This involves selecting a random subset of the features at each candidate split in the learning process. This ensures that the correlation is lower.<\/p>\n Let us now implement the random forest method in R. We will be using the randomForest and the caTools packages for this. The dataset that we are going to use is a heart disease dataset from the UCI Machine Learning Repository and can be found here<\/a>.<\/p>\n 1. Firstly let us begin with loading the packages and the dataset.<\/p>\n Output<\/strong><\/p>\n 2. Let us now take a look at the data and get familiar with it.<\/p>\n Output<\/strong><\/p>\n 3. As the dataset does not have names to the columns, let\u2019s add some names manually.<\/p>\n Output<\/strong><\/p>\n 4. For simplicity, let\u2019s turn the degree of heart disease into a binary. We will leave the value 0 which indicates an absence of heart disease and replace any value above 1 with 1 which signifies the presence of heart disease. We can also check the class of each column to verify that the categorical variables are stored as such.<\/p>\n Output<\/strong><\/p>\n 5. As we can see above, the sex variable has a numeric class when it should be a factor. Let us change the class of all the columns to what they should be and also check for missing values. As R expects NA for missing values, it puts a \u2018?\u2019 as a placeholder while importing the data. We can use that to find the number of missing values in each column.<\/p>\n 6. Thai has 2 missing values and ca have 4. We know that both thai and ca are factors. For thai, 3 = normal, 6 = fixed defect, and 7 = reversible defect and ca is the number of major vessels (0-3). So let\u2019s replace the NA\u2019s in thai with 3 and we can remove the rows with NA\u2019s in the ca column. We will then cast the columns as factors to remove the “?\u201d as a level from them.<\/p>\n Output<\/strong><\/p>\n 7. Let us now split our data and set some aside for testing.<\/p>\n Output<\/strong><\/p>\n 8. Now let\u2019s create our random forest using the randomForest() function.<\/p>\n Output<\/strong><\/p>\nEnsemble Learning Methods<\/h2>\n
Overfitting and Underfitting<\/h2>\n
Bagging<\/h2>\n
Algorithm for R Random Forest Method<\/h2>\n
<\/a>In the case of categorical output, the majority output can be chosen. We can calculate an estimation of the uncertainty of the prediction as such:
\n<\/a>Generally, the number of trees in a random forest can vary from a few hundred to several thousand depending on the training set. The optimal number of trees can be found by using the cross-validation method.<\/p>\nPractical Implementation of Random Forest in R<\/h2>\n
library(randomForest)\nlibrary(caTools)\ndata <- read.csv(\"processed.cleveland.data\", header=FALSE)\n<\/pre>\n
<\/a><\/p>\nstr(data)<\/pre>\n
<\/a><\/p>\nnames(data) <- c(\"age\", \"sex\", \"cp\", \"trestbps\", \"choi\", \"fbs\", \"restecg\", \"thalach\", \"exang\", \"oldpeak\", \"slope\", \"ca\", \"thai\", \"num\")\nhead(data)\n<\/pre>\n
<\/a><\/p>\ndata$num[data$num > 1] <- 1\nsapply(data,class)\n<\/pre>\n
<\/a><\/p>\ndata <- transform(data,\n age=as.integer(age),\n sex=as.factor(sex),\n cp=as.factor(cp),\n trestbps=as.integer(trestbps),\n choi=as.integer(choi),\n fbs=as.factor(fbs),\n restecg=as.factor(restecg),\n thalach=as.integer(thalach),\n exang=as.factor(exang),\n oldpeak=as.numeric(oldpeak),\n slope=as.factor(slope),\n ca=as.factor(ca),\n thai=as.factor(thai),\n num=as.factor(num)\n )\ndata[data == \"?\"] <- NA\ncolSums(is.na(data))\n<\/pre>\n
<\/a><\/p>\ndata$thai[is.na(data$thai)] <- as.factor(\"3.0\")\ndata <- data[!(data$ca %in% c(NA)), ]\ndata$thai <- as.factor(data$thai)\ndata$ca <- as.factor(data$ca)\n<\/pre>\n
<\/a><\/p>\nsample = sample.split(data$num, SplitRatio = .75)\ndata.train = subset(data, sample == TRUE)\ndata.test = subset(data, sample == FALSE)\ndim(data.train)\ndim(data.test)\n<\/pre>\n
<\/a><\/p>\ndata.rf <- randomForest(num ~ ., data=data.train)\nprint(data.rf)\n<\/pre>\n