{"id":75852,"date":"2020-01-27T14:59:41","date_gmt":"2020-01-27T09:29:41","guid":{"rendered":"https:\/\/techvidvan.com\/tutorials\/?p=75852"},"modified":"2020-01-27T14:59:41","modified_gmt":"2020-01-27T09:29:41","slug":"pca-and-factor-analysis-in-r","status":"publish","type":"post","link":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/","title":{"rendered":"PCA and Factor Analysis in R – Methods, Functions, Datasets"},"content":{"rendered":"

In this TechVidvan<\/strong> tutorial, we will learn about principal component analysis (PCA) and factor analysis in R. We will learn what these techniques are and where they are used. Finally, we will implement them in R on a sample dataset.<\/p>\n

Introduction to PCA and Factor Analysis<\/h2>\n

Principal component analysis(PCA) and factor analysis in R are statistical analysis techniques also known as multivariate analysis techniques<\/strong>. These techniques are most useful in R when the available data has too many variables to be feasibly analyzed. They reduce the number of variables that need to be processed without compromising the information conveyed by them.<\/p>\n

What is Principal-Component Analysis?<\/h3>\n

Principal-component analysis (PCA<\/strong>) is a multivariate analysis technique<\/strong>. The basic idea behind this technique is to find variables with strong correlations between them and extract a single variable that can then represent them at the same time.<\/p>\n

What are Principal-Components?<\/h3>\n

In the PCA, we find the correlation between all the available variables. We do this by making the covariance matrix<\/strong> of the dataset. Then we extract new variables that can depict the original information more efficiently than the older variables. We call these new variables as principal-components<\/strong>.<\/p>\n

Our goal is to keep the number of principal-components less than the number of original variables. Thus, reducing the number of variables, we need to process.<\/p>\n

Principal components are normalized linear combinations<\/strong> of the original variables. Let\u2019s say that we have variable x1<\/sup>, x2<\/sup>, \u2026,xn<\/sup>. Then our first principal component will be:<\/p>\n

Z1<\/sup>=\u03a611<\/sup>X1<\/sup>+\u03a621<\/sup>X2<\/sup>+\u03a631<\/sup>X3<\/sup>+\u2026. +\u03a6n1<\/sup>Xn<\/sup><\/span><\/strong><\/p>\n

Where Z1<\/sup><\/span>\u00a0is the first principal component,<\/p>\n

\u03a6p1<\/sup> is the loading vector of the first principal component. The loadings\u2019 sum of squares is equal to one,<\/p>\n

And X1<\/sup>, X2<\/sup>,…, Xn<\/sup> are normalized variables.<\/p>\n

When do we use Principal-Component Analysis?<\/h3>\n

Imagine we need to predict the annual profits of a company in the year 2019. Assuming that we have the profits in the first quarter of 2019, the number of staff working in the company, their salaries, and the details of other expenditure by the company.<\/p>\n

We can use other information such as tax reports, costs of ongoing ventures, and their expected returns. We also have the annual profits of the years 2018, 2017, 2016, and so on. It might look like we have a lot of variables to go on and make a prediction as accurate as possible. But that is where the problem lies.<\/p>\n

We have a lot of variables.<\/strong><\/em><\/p>\n

PCA becomes useful in such cases. By finding variables with high correlation and grouping them, principal-component analysis reduces the number of variables we need to process without compromising the information they convey.<\/p>\n

How does Principal-Component Analysis Work?<\/h3>\n

Here is a step-by-step overview of the process involved in principal-component analysis:<\/p>\n

    \n
  1. Subtract the mean of every variable from each instance of them. Suppose we have m instances of n variables named A, B, and C, we would have a m x n matrix. Let\u2019s call this matrix M.<\/li>\n
  2. Find the covariance matrix of M. Let\u2019s call this as CM.<\/li>\n
  3. Compute the eigenvectors and eigenvalues of CM.<\/li>\n
  4. Arrange the eigenvectors according to their eigenvalues.<\/li>\n
  5. Select the eigenvectors with the highest eigenvalues. If we select k eigenvectors, we will get an n x k matrix. Let\u2019s call this matrix EM.<\/li>\n
  6. Obtain the transformed data. We can get the transformed data by using the following formula:<\/li>\n<\/ol>\n

    \u00a0 \u00a0 \u00a0 \u00a0 M’ = M*EM<\/strong><\/p>\n

    Methods for Principal Component Analysis<\/h3>\n

    There are two methods for principal component analysis. These are:<\/p>\n

      \n
    1. Spectral decomposition<\/li>\n
    2. Singular value decomposition<\/li>\n<\/ol>\n

      1. Spectral Decomposition<\/h4>\n

      The spectral decomposition of a square m x m matrix A is defined as<\/p>\n

      A = U\u0394UT<\/sup><\/strong><\/p>\n

      Where U is an orthogonal matrix that is UUT<\/sup>=I, the columns of U are the eigenvectors of A,<\/p>\n

      And \u0394 is a diagonal matrix with A\u2019s eigenvalues at it\u2019s diagonal.<\/p>\n

      We can find a matrix\u2019s principal components by performing spectral decomposition on its covariance matrix.<\/p>\n

      Don’t know much about R matrix?<\/strong> Learn to create, modify, and access R matrix<\/strong><\/a> components.<\/p>\n

      2. Singular Value Decomposition<\/h4>\n

      The single value decomposition of an n x m matrix B, where n \u2265 m, is defined as<\/p>\n

      B=U\u0393VT<\/sup><\/span><\/strong><\/p>\n

      Where U is an n x m orthogonal matrix, \u0393 is an m x m orthogonal matrix, and V is an n x n diagonal matrix. Here, the columns of \u0393 are the eigenvectors of B, and the square of the elements of the diagonal matrix V are the eigenvalues of U.<\/p>\n

      The singular value decomposition of the covariance matrix is another way of finding the principal components of a matrix.<\/p>\n

      Implementation of PCA in R<\/h3>\n

      Now we will perform principal-component analysis on a dataset in the R programming language.<\/p>\n

      Performing PCA on a dataset<\/h4>\n

      We will use the built-in dataset mtcars<\/strong>. The dataset has 32 instances for 11 variables. It gives 11 features like \u2018miles per gallon\u2019, \u2018number of cylinders\u2019, \u2018horsepower\u2019, etc. of 32 different models of cars. In the dataset, there are two categorical variables.<\/p>\n

      First is \u2018vs\u2019 that shows whether the car\u2019s engine is \u2018v\u2019 shaped (1) or not (0).<\/p>\n

      The second one is \u2018am\u2019 that shows whether the car has an automatic transmission (1) or manual (0).<\/p>\n

      We will have to ignore these two variables in the analysis as PCA is for numeric data and cannot deal with categorical variables.<\/p>\n

      We will compute the principal components using the prcomp()<\/code><\/strong> function to achieve this.<\/p>\n

      Code:<\/strong><\/p>\n

      mtcars.pca<-prcomp(mtcars[,c(1:7,10,11)],center=TRUE, scale.=TRUE)\nsummary(mtcars.pca)<\/pre>\n
      Output:<\/strong><\/p>\n
      \"<\/a><\/strong><\/p>\n
      Viewing the results of PCA<\/h4>\n
      The summary()<\/code><\/strong> function provides us with a summary of the object mtcars.pca<\/code><\/strong>, which stores the principal components extracted from the data. These components are arranged in decreasing order of importance. The proportion of variance shows us how much variation does a particular component covers.<\/p>\n
      As you can see, the first component PC1<\/code><\/strong> covers almost 63% of the variance. If we use the second one as well, PC1<\/code><\/strong> and PC2<\/code><\/strong> represent almost 86% of the variance and, thus are sufficient to represent the original variables. As a result, we have reduced the 11 original variables to just two!<\/p>\n
      Plotting the results of the PCA<\/h4>\n
      Now we will plot the PCA. We will be using a biplot, which shows the principal components as well as the old variables. To do that, we will be using the ggbiplot<\/code><\/strong> library. The ggbiplot<\/code><\/strong><\/a> package requires the devtools<\/code><\/strong> package<\/strong><\/a>, so make sure to install, and include that first.<\/p>\n
      Code:<\/strong><\/p>\n
      install.packages(\u201cdevtools\u201d)\nlibrary(devtools)\ninstall.packages(\u201cvqv\/ggbiplot\u201d)<\/pre>\n
      Note<\/strong>: For R versions 3.4.4 and above, you can use the following command to install the ggbiplot<\/code><\/strong> package.<\/p>\n
      Code:<\/strong><\/p>\n
      install_github(\u201cvqv\/ggbiplot\u201d)<\/pre>\n
      Output:<\/strong><\/p>\n
      \"install<\/a><\/strong><\/p>\n
      Once installed, include the package<\/strong><\/a> in the session by using the following command:<\/p>\n
      Code:<\/strong><\/p>\n
      library(ggbiplot)<\/pre>\n
      Output:<\/strong><\/p>\n
      \"library(ggbiplot)<\/a><\/strong><\/p>\n
      Now that the ggbiplot<\/code><\/strong> package is ready, we can use the ggbiplot()<\/code><\/strong> function to plot the principal components.<\/p>\n
      Code:<\/strong><\/p>\n
      ggbiplot(mtcars.pca)<\/pre>\n
      Output:<\/strong><\/p>\n
      \"ggbiplot()<\/a><\/strong><\/p>\n
      As you can see, we have successfully performed PCA on our dataset and plotted the results.<\/p>\n
      Factor analysis in R<\/h3>\n
      Factor analysis (FA) or exploratory factor analysis<\/strong> is another technique to reduce the number of variables to a smaller set of factors. FA identifies the relationships among a set of variables and narrows it down to a smaller set.<\/p>\n
      We will be using the bfi<\/strong> dataset, which is a built-in dataset provided in R. It comprises 25 different personality factors. We will require the psych<\/code><\/strong> and the GPArotation<\/code><\/strong> packages. So, install and load them into the library. Then we will use the following command to find how many factors we need.<\/p>\n
      Code:<\/strong><\/p>\n
      parallel <- fa.parallel(bfi,fm=\"minres\",fa='fa')<\/pre>\n
      Output:<\/strong><\/p>\n
      Parallel analysis suggests that the number of factors = 7 and the number of components = NA<\/p>\n
      \"fa.parallel()<\/a><\/strong><\/p>\n
      Now that we know how many factors we need, we can perform the factor analysis using the fa()<\/code><\/strong> function<\/strong>.<\/p>\n
      Code:<\/strong><\/p>\n
      factors <- fa(bfi,nfactors=7,rotate='oblimin',fm='minres')\nprint(factors)<\/pre>\n
      Output:<\/strong><\/p>\n
      \"fa()<\/a><\/strong><\/p>\n
      Summary<\/h2>\n
      PCA and factor analysis in R are both multivariate analysis techniques. They both work by reducing the number of variables while maximizing the proportion of variance covered. The prime difference between the two methods is the new variables derived.<\/p>\n
      The principal components are normalized linear combinations of the original variables. The factors are measurement models of latent variables. While both techniques have the same purpose, they have different approaches and results.<\/p>\n
      In this tutorial, we learned what PCA and factor analysis are and also implemented the two techniques in R. We hope that this was an informative read to you.<\/p>\n
      Any difficulty while practicing PCA and factor analysis in R?<\/strong><\/p>\n
      Ask our TechVidvan<\/strong> experts.<\/p>\n
      Keep Executing buddy!!<\/p>\n","protected":false},"excerpt":{"rendered":"
      In this TechVidvan tutorial, we will learn about principal component analysis (PCA) and factor analysis in R. We will learn what these techniques are and where they are used. Finally, we will implement them...<\/p>\n","protected":false},"author":1,"featured_media":75901,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1020],"tags":[1494,1495,1496,1497,1498,1499,1500,1501,1502],"class_list":["post-75852","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-r","tag-factor-analysis-using-r","tag-pca-and-factor-analysis-in-r","tag-pca-in-r","tag-pca-in-r-example","tag-pca-in-r-step-by-step","tag-pca-plot-r","tag-principal-component-analysis-in-r","tag-principal-component-analysis-r-example","tag-principal-component-and-factor-analysis-in-r"],"yoast_head":"\nPCA and Factor Analysis in R - Methods, Functions, Datasets - TechVidvan<\/title>\n<meta name=\"description\" content=\"PCA and Factor Analysis in R help in reducing the number of variables. In this article, learn to implement these two techniques in R with their examples.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"PCA and Factor Analysis in R - Methods, Functions, Datasets - TechVidvan\" \/>\n<meta property=\"og:description\" content=\"PCA and Factor Analysis in R help in reducing the number of variables. In this article, learn to implement these two techniques in R with their examples.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/\" \/>\n<meta property=\"og:site_name\" content=\"TechVidvan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/TechVidvan\/\" \/>\n<meta property=\"article:published_time\" content=\"2020-01-27T09:29:41+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/01\/principal-component-and-factor-analysis-in-R.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"802\" \/>\n\t<meta property=\"og:image:height\" content=\"420\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"TechVidvan Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:site\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TechVidvan Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"PCA and Factor Analysis in R - Methods, Functions, Datasets - TechVidvan","description":"PCA and Factor Analysis in R help in reducing the number of variables. In this article, learn to implement these two techniques in R with their examples.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/","og_locale":"en_US","og_type":"article","og_title":"PCA and Factor Analysis in R - Methods, Functions, Datasets - TechVidvan","og_description":"PCA and Factor Analysis in R help in reducing the number of variables. In this article, learn to implement these two techniques in R with their examples.","og_url":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/","og_site_name":"TechVidvan","article_publisher":"https:\/\/www.facebook.com\/TechVidvan\/","article_published_time":"2020-01-27T09:29:41+00:00","og_image":[{"width":802,"height":420,"url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/01\/principal-component-and-factor-analysis-in-R.jpg","type":"image\/jpeg"}],"author":"TechVidvan Team","twitter_card":"summary_large_image","twitter_creator":"@vidvantech","twitter_site":"@vidvantech","twitter_misc":{"Written by":"TechVidvan Team","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/#article","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/"},"author":{"name":"TechVidvan Team","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22"},"headline":"PCA and Factor Analysis in R – Methods, Functions, Datasets","datePublished":"2020-01-27T09:29:41+00:00","mainEntityOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/"},"wordCount":1350,"commentCount":0,"publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/01\/principal-component-and-factor-analysis-in-R.jpg","keywords":["factor analysis using R","pca and factor analysis in R","pca in R","pca in R example","pca in R step by step","pca plot R","principal component analysis in R","principal component analysis R example","principal component and factor analysis in R"],"articleSection":["R Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/","url":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/","name":"PCA and Factor Analysis in R - Methods, Functions, Datasets - TechVidvan","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/#website"},"primaryImageOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/#primaryimage"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/01\/principal-component-and-factor-analysis-in-R.jpg","datePublished":"2020-01-27T09:29:41+00:00","description":"PCA and Factor Analysis in R help in reducing the number of variables. In this article, learn to implement these two techniques in R with their examples.","breadcrumb":{"@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/#primaryimage","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/01\/principal-component-and-factor-analysis-in-R.jpg","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/01\/principal-component-and-factor-analysis-in-R.jpg","width":802,"height":420,"caption":"PCA and factor analysis in R"},{"@type":"BreadcrumbList","@id":"https:\/\/techvidvan.com\/tutorials\/pca-and-factor-analysis-in-r\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/techvidvan.com\/tutorials\/"},{"@type":"ListItem","position":2,"name":"PCA and Factor Analysis in R – Methods, Functions, Datasets"}]},{"@type":"WebSite","@id":"https:\/\/techvidvan.com\/tutorials\/#website","url":"https:\/\/techvidvan.com\/tutorials\/","name":"TechVidvan Blogs","description":"","publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/techvidvan.com\/tutorials\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/techvidvan.com\/tutorials\/#organization","name":"TechVidvan","url":"https:\/\/techvidvan.com\/tutorials\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","width":200,"height":50,"caption":"TechVidvan"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/TechVidvan\/","https:\/\/x.com\/vidvantech"]},{"@type":"Person","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22","name":"TechVidvan Team","description":"The TechVidvan Team delivers practical, beginner-friendly tutorials on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our experts are here to help you upskill and excel in today\u2019s tech industry."}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/75852","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/comments?post=75852"}],"version-history":[{"count":0,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/75852\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media\/75901"}],"wp:attachment":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media?parent=75852"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/categories?post=75852"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/tags?post=75852"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}