{"id":1998,"date":"2017-10-05T05:06:42","date_gmt":"2017-10-05T05:06:42","guid":{"rendered":"http:\/\/techvidvan.com\/tutorials\/?p=358"},"modified":"2017-10-05T05:06:42","modified_gmt":"2017-10-05T05:06:42","slug":"hadoop-2-x-vs-hadoop-3-x","status":"publish","type":"post","link":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/","title":{"rendered":"20 Notable Difference Between Hadoop 2.x vs Hadoop 3.x"},"content":{"rendered":"

The objective of this Hadoop tutorial<\/strong> is to provide you a clearer understanding between different Hadoop version. In this blog we have covered top, 20 Difference between Hadoop 2.x vs\u00a0Hadoop 3.x.<\/p>\n

This blog covers the difference between Hadoop 2 and Hadoop 3 on the basis of different features.<\/p>\n

Difference Between Hadoop 2.x vs Hadoop 3.x<\/h3>\n

Apache Hadoop<\/strong> is an open source software framework for distributed storage & processing of huge amount of data sets.<\/p>\n

Hadoop 3.x was introduced to overcome the limitation of Hadoop<\/strong> 2.x. Hadoop 3.x has added some new features, although the old features are still used.<\/p>\n

Detailed feature wise comparison between Hadoop 2.x vs Hadoop 3.x are given below:<\/p>\n

a. License<\/h4>\n
    \n
  • Hadoop 2<\/strong>.x-<\/strong> Apache 2.0, open source<\/li>\n
  • Hadoop 3<\/strong>.x-<\/strong> Apache 2.0, open source<\/li>\n<\/ul>\n

    b. Minimum supported version of Java<\/h4>\n
      \n
    • Hadoop 2<\/strong>.x-<\/strong>\u00a0 Java 7.<\/li>\n
    • Hadoop 3<\/strong>.x-<\/strong>\u00a0 Java 8.<\/li>\n<\/ul>\n

      c. Fault Tolerance<\/h4>\n
        \n
      • Hadoop 2.x-<\/strong> In this version, replication handles fault tolerance.<\/li>\n
      • Hadoop 3.x-<\/strong> In this version, erasure coding handle fault tolerance.<\/li>\n<\/ul>\n

        d. Data Balancing<\/h4>\n
          \n
        • Hadoop 2.x-<\/strong> Uses HDFS<\/strong> Balancer for data balancing<\/li>\n
        • Hadoop 3.x-<\/strong> Uses Intra-data node balancer, which is invoked via the HDFS disk balancer CLI.<\/li>\n<\/ul>\n

          e. Storage Scheme<\/h4>\n
            \n
          • Hadoop 2.x-<\/strong> Uses 3X replication scheme.<\/li>\n
          • Hadoop 3.x-<\/strong> Uses Erasure coding.<\/li>\n<\/ul>\n

            f. Storage Overhead<\/h4>\n
              \n
            • Hadoop 2.x-<\/strong> In this version HDFS has 200% overhead in storage space.<\/li>\n
            • Hadoop 3.x-<\/strong> In this version HDFS has 50% overhead in storage space.<\/li>\n<\/ul>\n

              g. Storage Overhead Example<\/h4>\n
                \n
              • Hadoop 2.x-<\/strong> If there are 6 blocks, and 3x replication of each block, so it results in 18 blocks. It will occupy 18 blocks space.<\/li>\n
              • Hadoop 3.x-<\/strong> If there are 6 blocks, so it will occupy 9 block space i.e. 6 blocks and 3 for parity.<\/li>\n<\/ul>\n

                h. YARN Timeline Service<\/h4>\n
                  \n
                • Hadoop 2.x-<\/strong> Uses old timeline service which has scalability issues.<\/li>\n
                • Hadoop 3.x-<\/strong> This version improves the timeline service v2. It also improves the scalability and reliability of timeline service.<\/li>\n<\/ul>\n

                  j. Default Ports Range<\/h4>\n
                    \n
                  • Hadoop 2.x-<\/strong> In this version, default ports are Linux ephemeral port range. Hence at the time of startup, they will fail to bind.<\/li>\n
                  • Hadoop 3.x-<\/strong> While this version is moved out of ephemeral range.<\/li>\n<\/ul>\n

                    k. Tools<\/strong><\/h4>\n
                      \n
                    • Hadoop 2.x-<\/strong> Hive, pig, Tez, Hama, and other Hadoop tools are also available.<\/li>\n
                    • Hadoop 3.x-<\/strong> In this version also Hive, pig, Tez, Hama, and other Hadoop tools are available.<\/li>\n<\/ul>\n

                      l. Compatible File System<\/h4>\n
                        \n
                      • Hadoop 2.x-<\/strong> It supports HDFS\u00a0(Default FS), FTP File system: This also stores all its data on remotely accessible FTP servers. It also supports Amazon S3 (Simple Storage Service) file system Windows Azure Storage Blobs (WASB) file system.<\/li>\n
                      • Hadoop 3.x-<\/strong> It supports all the previous one as well as Microsoft Azure Data Lake filesystem.<\/li>\n<\/ul>\n

                        m. Datanode Resources<\/h4>\n
                          \n
                        • Hadoop 2.x-<\/strong> For the MapReduce Datanode resource is not dedicated. We can also use it for other application.<\/li>\n
                        • Hadoop 3.x-<\/strong> In this version also data node resource can be used for other Applications too.<\/li>\n<\/ul>\n

                          n. MR API Compatibility<\/h4>\n
                            \n
                          • Hadoop 2.x-<\/strong> MR API compatible with Hadoop 1.x program to execute on Hadoop 2.X<\/li>\n
                          • Hadoop 3.x-<\/strong> MR API is also compatible with running\u00a0Hadoop\u00a01.x programs to execute on Hadoop 3.X<\/li>\n<\/ul>\n

                            o. Support for Microsoft<\/h4>\n
                              \n
                            • Hadoop 2.x-<\/strong> It can be deployed on Windows.<\/li>\n
                            • Hadoop 3.x-<\/strong> It also supports for Microsoft windows.<\/li>\n<\/ul>\n

                              p. Slots\/container<\/h4>\n
                                \n
                              • Hadoop 2.x-<\/strong> Hadoop\u00a01.x works on the concept of slots while Hadoop 2.X works on the concept of the container.<\/li>\n
                              • Hadoop 3.x-<\/strong> Hadoop 3.x also works on the concept of a container.<\/li>\n<\/ul>\n

                                q. Single point of failure<\/h4>\n
                                  \n
                                • Hadoop 2.x-<\/strong> It has the features to overcome SPOF. So, whenever NameNode fails it recovers automatically.<\/li>\n
                                • Hadoop 3.x-<\/strong> It also has the features to overcome SPOF. So, whenever NameNode fails it recovers automatically no need of manual intervention.<\/li>\n<\/ul>\n

                                  r. HDFS Federation<\/h4>\n
                                    \n
                                  • Hadoop 2.x-<\/strong> In\u00a0Hadoop\u00a01.x only single NameNode to manage all Namespace. But Hadoop 2.x has multiple NameNode for multiple Namespace.<\/li>\n
                                  • Hadoop 3.x-\u00a0<\/strong>It also has multiple Namenode for multiple namespaces.<\/li>\n<\/ul>\n

                                    s. Scalability<\/h4>\n
                                      \n
                                    • Hadoop 2.x-<\/strong> We can scale up to 10000 Nodes per cluster.<\/li>\n
                                    • Hadoop 3.x-<\/strong> We can scale more than 10000 Nodes per cluster.<\/li>\n<\/ul>\n

                                      t. HDFS Snapshot<\/h4>\n
                                        \n
                                      • Hadoop 2.x-<\/strong> It adds the support for a snapshot. It also provides disaster recovery and protection for user error.<\/li>\n
                                      • Hadoop 3.x-<\/strong> It also support for the snapshot feature.<\/li>\n<\/ul>\n

                                        u. Platform<\/h4>\n
                                          \n
                                        • Hadoop 2.x-<\/strong> It serves as a platform for a wide variety of\u00a0data analytics. It is also possible to run event processing, streaming, and real-time operations.<\/li>\n
                                        • Hadoop 3.x-<\/strong> It is also possible to run event processing, streaming and real-time operation on the top of\u00a0YARN.<\/li>\n<\/ul>\n

                                          Conclusion<\/h3>\n

                                          In conclusion, Hadoop 3.0 has added new features like erasure coding to handle fault tolerance. Hadoop 3.x also reduces the storage overhead by 200% to 50%.<\/p>\n

                                          It also introduced a new command line tool called Disk balancer. Hence, Hadoop 3.x has improved overall performance.<\/p>\n

                                          If you find any other difference between Hadoop 2.x vs Hadoop 3.x, so do let us know in the comment section.<\/p>\n","protected":false},"excerpt":{"rendered":"

                                          The objective of this Hadoop tutorial is to provide you a clearer understanding between different Hadoop version. In this blog we have covered top, 20 Difference between Hadoop 2.x vs\u00a0Hadoop 3.x. This blog covers...<\/p>\n","protected":false},"author":1,"featured_media":73138,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[544],"tags":[538,457,539,541,619,620,621],"class_list":["post-1998","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hadoop","tag-apache-hadoop","tag-big-data","tag-big-data-hadoop","tag-hadoop","tag-hadoop-2-x","tag-hadoop-2-x-vs-hadoop-3-x","tag-hadoop-3-x"],"yoast_head":"\n20 Notable Difference Between Hadoop 2.x vs Hadoop 3.x - TechVidvan<\/title>\n<meta name=\"description\" content=\"Feature wise comparison between Hadoop 2 vs Hadoop 3 cover top 20 difference between Hadoop 2.x vs hadoop 3.x,Hadoop 2 Features,Hadoop 3 Features\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"20 Notable Difference Between Hadoop 2.x vs Hadoop 3.x - TechVidvan\" \/>\n<meta property=\"og:description\" content=\"Feature wise comparison between Hadoop 2 vs Hadoop 3 cover top 20 difference between Hadoop 2.x vs hadoop 3.x,Hadoop 2 Features,Hadoop 3 Features\" \/>\n<meta property=\"og:url\" content=\"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/\" \/>\n<meta property=\"og:site_name\" content=\"TechVidvan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/TechVidvan\/\" \/>\n<meta property=\"article:published_time\" content=\"2017-10-05T05:06:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/hadoop-2-x-vs-hadoop-3.x.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"802\" \/>\n\t<meta property=\"og:image:height\" content=\"420\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"TechVidvan Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:site\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TechVidvan Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"20 Notable Difference Between Hadoop 2.x vs Hadoop 3.x - TechVidvan","description":"Feature wise comparison between Hadoop 2 vs Hadoop 3 cover top 20 difference between Hadoop 2.x vs hadoop 3.x,Hadoop 2 Features,Hadoop 3 Features","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/","og_locale":"en_US","og_type":"article","og_title":"20 Notable Difference Between Hadoop 2.x vs Hadoop 3.x - TechVidvan","og_description":"Feature wise comparison between Hadoop 2 vs Hadoop 3 cover top 20 difference between Hadoop 2.x vs hadoop 3.x,Hadoop 2 Features,Hadoop 3 Features","og_url":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/","og_site_name":"TechVidvan","article_publisher":"https:\/\/www.facebook.com\/TechVidvan\/","article_published_time":"2017-10-05T05:06:42+00:00","og_image":[{"width":802,"height":420,"url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/hadoop-2-x-vs-hadoop-3.x.jpg","type":"image\/jpeg"}],"author":"TechVidvan Team","twitter_card":"summary_large_image","twitter_creator":"@vidvantech","twitter_site":"@vidvantech","twitter_misc":{"Written by":"TechVidvan Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/#article","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/"},"author":{"name":"TechVidvan Team","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22"},"headline":"20 Notable Difference Between Hadoop 2.x vs Hadoop 3.x","datePublished":"2017-10-05T05:06:42+00:00","mainEntityOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/"},"wordCount":800,"commentCount":0,"publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/hadoop-2-x-vs-hadoop-3.x.jpg","keywords":["apache hadoop","big data","big data hadoop","hadoop","Hadoop 2.x","Hadoop 2.x vs hadoop 3.x","hadoop 3.x"],"articleSection":["Hadoop Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/","url":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/","name":"20 Notable Difference Between Hadoop 2.x vs Hadoop 3.x - TechVidvan","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/#website"},"primaryImageOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/#primaryimage"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/hadoop-2-x-vs-hadoop-3.x.jpg","datePublished":"2017-10-05T05:06:42+00:00","description":"Feature wise comparison between Hadoop 2 vs Hadoop 3 cover top 20 difference between Hadoop 2.x vs hadoop 3.x,Hadoop 2 Features,Hadoop 3 Features","breadcrumb":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/#primaryimage","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/hadoop-2-x-vs-hadoop-3.x.jpg","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/hadoop-2-x-vs-hadoop-3.x.jpg","width":802,"height":420,"caption":"hadoop 2.x vs hadoop 3.x - Feature wise difference"},{"@type":"BreadcrumbList","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-2-x-vs-hadoop-3-x\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/techvidvan.com\/tutorials\/"},{"@type":"ListItem","position":2,"name":"20 Notable Difference Between Hadoop 2.x vs Hadoop 3.x"}]},{"@type":"WebSite","@id":"https:\/\/techvidvan.com\/tutorials\/#website","url":"https:\/\/techvidvan.com\/tutorials\/","name":"TechVidvan Blogs","description":"","publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/techvidvan.com\/tutorials\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/techvidvan.com\/tutorials\/#organization","name":"TechVidvan","url":"https:\/\/techvidvan.com\/tutorials\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","width":200,"height":50,"caption":"TechVidvan"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/TechVidvan\/","https:\/\/x.com\/vidvantech"]},{"@type":"Person","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22","name":"TechVidvan Team","description":"The TechVidvan Team delivers practical, beginner-friendly tutorials on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our experts are here to help you upskill and excel in today\u2019s tech industry."}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/1998","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/comments?post=1998"}],"version-history":[{"count":0,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/1998\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media\/73138"}],"wp:attachment":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media?parent=1998"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/categories?post=1998"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/tags?post=1998"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}