{"id":456,"date":"2017-10-07T05:50:35","date_gmt":"2017-10-07T05:50:35","guid":{"rendered":"http:\/\/techvidvan.com\/tutorials\/?p=456"},"modified":"2017-10-07T05:50:35","modified_gmt":"2017-10-07T05:50:35","slug":"automatic-failover-in-hadoop","status":"publish","type":"post","link":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/","title":{"rendered":"What is NameNode Automatic Failover in Hadoop HDFS?"},"content":{"rendered":"

In this Hadoop tutorial<\/strong>, we are going to discuss the concept of NameNode Automatic Failover in Hadoop\u00a0 First of all, we will see what is failover and types of failover. Then we will cover HDFS<\/strong> automatic failover in Hadoop.<\/p>\n

Components of Hadoop Automatic Failover in HDFS such as\u00a0ZooKeeper quorum, ZKFailoverController Process (ZKFC). At last, we will also discuss the roles of these two components in Hadoop<\/strong>.<\/p>\n

What is Failover?<\/h3>\n

The process in which system transfers its control to the secondary system when it detects a failure is known as failover<\/strong>.<\/p>\n

Failover is of two types:<\/p>\n

    \n
  • Graceful Failover –<\/strong> Administrator initiates Graceful Failover manually, for example, in the case of routine maintenance. The system will not activate a failover automatically from the active to standby namenode, even if the active node has failed in Graceful Failover.<\/li>\n<\/ul>\n
      \n
    • Automatic Failover –\u00a0<\/strong>Automatic Failover is the process in which system automatically transfers its control to the standby NameNode when the NameNode fails. In Hadoop Automatic failover occurs in case of NameNode failures. But in the case of NameNode failure, Failover will start automatically. This failover is Automatic failover.<\/li>\n<\/ul>\n

      NameNode Automatic Failover in Hadoop<\/h3>\n

      Automatic failover in Hadoop adds up below components to a Hadoop HDFS deployment:<\/p>\n

        \n
      • ZooKeeper quorum.<\/li>\n
      • ZKFailoverController Process (ZKFC).<\/li>\n<\/ul>\n

        1. ZooKeeper Quorum<\/h4>\n

        Zookeeper in Hadoop is a centralized service. It maintains configuration information, naming, and provide distributed synchronization. It also provides group services. Zookeeper also manages and coordinates a large cluster of machines.<\/p>\n

        For the below reasons, implementation of automatic HDFS failover relies on Zookeeper:<\/p>\n

          \n
        • Failure Detection-<\/strong> Zookeeper in Hadoop maintains a session with the namenodes. During a failure, the session will expire. So, the Zookeeper will inform other namenodes to start the failover process.<\/li>\n
        • Active NameNode Election-<\/strong> A simple mechanism provided by Zookeeper to only elect a node as active. At the time of active namenode failure, other namenode can take an exclusive lock in Zookeeper stating that it wants to become the next active namenode.<\/li>\n<\/ul>\n

          2. ZKFailoverController (ZKFC)<\/h4>\n

          ZKFC is a client of Zookeeper in Hadoop that monitors and manages the namenode status. Each of the machines which run namenode service also runs a ZKFS. It can handle:<\/p>\n

            \n
          • Health monitoring –<\/strong> With a health-check command ZKFS pings its local namenode. The ZKFS considers the namenode healthy, as long as the namenode responds at a time. The health monitor will mark a node as unhealthy, if the node has crashed, or entered an unhealthy state.<\/li>\n
          • Zookeeper session management –\u00a0<\/strong>ZKFC holds a session open in Zookeeper when the local namenode is healthy. It also holds a special \u201clock\u201d znode, if the local namenode is active. If the session expires, the lock will be automatically deleted.<\/li>\n
          • Zookeeper-based election –\u00a0<\/strong>In HDFS, if the local namenode is healthy, and the ZKFC sees that no other node currently holds the lock znode, it will itself try to get the lock. And, if it succeeds, then it has won the election\u201d, and it handles running a failover to make its local namenode active.<\/li>\n<\/ul>\n

            Conclusion<\/h3>\n

            Hence, Automatic Failover in Hadoop starts automatically in case of NameNode failure. Automatic failover adds ZooKeeper quorum and ZKFailoverController Process (ZKFC) components to an HDFS deployment.<\/p>\n

            Zookeeper in Hadoop is a service that provides the automatic failover capability in HDFS<\/strong>.<\/p>\n","protected":false},"excerpt":{"rendered":"

            In this Hadoop tutorial, we are going to discuss the concept of NameNode Automatic Failover in Hadoop\u00a0 First of all, we will see what is failover and types of failover. Then we will cover...<\/p>\n","protected":false},"author":1,"featured_media":73211,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[564],"tags":[538,457,539,541,607,543,608],"class_list":["post-456","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hdfs","tag-apache-hadoop","tag-big-data","tag-big-data-hadoop","tag-hadoop","tag-hadoop-automatic-failover","tag-hadoop-tutorial","tag-hdfs-automatic-failover"],"yoast_head":"\nWhat is NameNode Automatic Failover in Hadoop HDFS? - TechVidvan<\/title>\n<meta name=\"description\" content=\"NameNode Automatic Failover in Hadoop covers what is failover, HDFS Failover Types-graceful,Automatic. ZooKeeper quorum, ZKFailoverController Process (ZKFC)\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What is NameNode Automatic Failover in Hadoop HDFS? - TechVidvan\" \/>\n<meta property=\"og:description\" content=\"NameNode Automatic Failover in Hadoop covers what is failover, HDFS Failover Types-graceful,Automatic. ZooKeeper quorum, ZKFailoverController Process (ZKFC)\" \/>\n<meta property=\"og:url\" content=\"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/\" \/>\n<meta property=\"og:site_name\" content=\"TechVidvan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/TechVidvan\/\" \/>\n<meta property=\"article:published_time\" content=\"2017-10-07T05:50:35+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/NameNode-Automatic-Failover-in-Hadoop-01.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"TechVidvan Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:site\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TechVidvan Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"What is NameNode Automatic Failover in Hadoop HDFS? - TechVidvan","description":"NameNode Automatic Failover in Hadoop covers what is failover, HDFS Failover Types-graceful,Automatic. ZooKeeper quorum, ZKFailoverController Process (ZKFC)","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/","og_locale":"en_US","og_type":"article","og_title":"What is NameNode Automatic Failover in Hadoop HDFS? - TechVidvan","og_description":"NameNode Automatic Failover in Hadoop covers what is failover, HDFS Failover Types-graceful,Automatic. ZooKeeper quorum, ZKFailoverController Process (ZKFC)","og_url":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/","og_site_name":"TechVidvan","article_publisher":"https:\/\/www.facebook.com\/TechVidvan\/","article_published_time":"2017-10-07T05:50:35+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/NameNode-Automatic-Failover-in-Hadoop-01.jpg","type":"image\/jpeg"}],"author":"TechVidvan Team","twitter_card":"summary_large_image","twitter_creator":"@vidvantech","twitter_site":"@vidvantech","twitter_misc":{"Written by":"TechVidvan Team","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/#article","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/"},"author":{"name":"TechVidvan Team","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22"},"headline":"What is NameNode Automatic Failover in Hadoop HDFS?","datePublished":"2017-10-07T05:50:35+00:00","mainEntityOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/"},"wordCount":546,"commentCount":0,"publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/NameNode-Automatic-Failover-in-Hadoop-01.jpg","keywords":["apache hadoop","big data","big data hadoop","hadoop","Hadoop Automatic failover","hadoop tutorial","HDFS Automatic failover"],"articleSection":["HDFS Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/","url":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/","name":"What is NameNode Automatic Failover in Hadoop HDFS? - TechVidvan","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/#website"},"primaryImageOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/#primaryimage"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/NameNode-Automatic-Failover-in-Hadoop-01.jpg","datePublished":"2017-10-07T05:50:35+00:00","description":"NameNode Automatic Failover in Hadoop covers what is failover, HDFS Failover Types-graceful,Automatic. ZooKeeper quorum, ZKFailoverController Process (ZKFC)","breadcrumb":{"@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/#primaryimage","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/NameNode-Automatic-Failover-in-Hadoop-01.jpg","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/NameNode-Automatic-Failover-in-Hadoop-01.jpg","width":1200,"height":628,"caption":"Automatic Failover in Hadoop"},{"@type":"BreadcrumbList","@id":"https:\/\/techvidvan.com\/tutorials\/automatic-failover-in-hadoop\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/techvidvan.com\/tutorials\/"},{"@type":"ListItem","position":2,"name":"What is NameNode Automatic Failover in Hadoop HDFS?"}]},{"@type":"WebSite","@id":"https:\/\/techvidvan.com\/tutorials\/#website","url":"https:\/\/techvidvan.com\/tutorials\/","name":"TechVidvan Blogs","description":"","publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/techvidvan.com\/tutorials\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/techvidvan.com\/tutorials\/#organization","name":"TechVidvan","url":"https:\/\/techvidvan.com\/tutorials\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","width":200,"height":50,"caption":"TechVidvan"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/TechVidvan\/","https:\/\/x.com\/vidvantech"]},{"@type":"Person","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22","name":"TechVidvan Team","description":"The TechVidvan Team delivers practical, beginner-friendly tutorials on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our experts are here to help you upskill and excel in today\u2019s tech industry."}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/456","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/comments?post=456"}],"version-history":[{"count":0,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/456\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media\/73211"}],"wp:attachment":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media?parent=456"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/categories?post=456"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/tags?post=456"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}