{"id":1991,"date":"2017-09-28T11:51:59","date_gmt":"2017-09-28T11:51:59","guid":{"rendered":"https:\/\/techvidvan.com\/tutorials\/?p=191"},"modified":"2017-09-28T11:51:59","modified_gmt":"2017-09-28T11:51:59","slug":"hadoop-high-availability","status":"publish","type":"post","link":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/","title":{"rendered":"Understanding Hadoop High Availability Feature"},"content":{"rendered":"

This blog provides you a description of the Hadoop<\/strong> HDFS High Availability feature. In this blog firstly we will discuss what exactly high availability is, How Hadoop achieve high availability, what is the need of HDFS high availability feature.<\/p>\n

We will also cover the example of Hadoop high availability feature in this Big data Hadoop tutorial.<\/p>\n

What is Hadoop HDFS High Availability?<\/h3>\n

 <\/p>\n

Hadoop HDFS<\/strong>\u00a0is a distributed file system. HDFS distributes data among the nodes in the Hadoop cluster by creating a replica of the file. Hadoop framework store these replicas of files on the other machines present in the cluster.<\/p>\n

So, when an HDFS client wants to access his data, he can easily access that data from a number of machines present in the cluster. Data is easily available in the closest node in the cluster.<\/p>\n

At some unfavorable conditions like a failure of a node, the client can easily access their data from the other nodes. This feature of Hadoop is called High Availability<\/strong>.<\/p>\n

How is High Availability achieved in Hadoop?<\/h3>\n

In the HDFS cluster, there is a number of DataNodes<\/strong>. After the definite interval of time, all these DataNodes sends heartbeat messages to the NameNode<\/strong>. If the NameNode stops receiving heartbeat messages from any of these DataNodes, then it assumes it to be dead.<\/p>\n

After that, it checks for the data present in those nodes and then gives commands to the other datanode to create a replica of that data to other datanodes. Therefore data is always available.<\/p>\n

When a client asks for a data access in HDFS, first of all, NameNode searches for the data in that datanodes, in which data is quickly available. And then provides access to that data to the client.<\/p>\n

Clients don\u2019t have to search for the data in all the datanodes. HDFS Namenode itself makes data availability easy to the clients by providing the address of the datanode from where a user can directly read.<\/p>\n

Example of Hadoop High Availability<\/h3>\n

Hadoop HDFS provides High availability of data. When the client requests NameNode for data access, then the NameNode searches for all the nodes in which that data is available.<\/p>\n

After that, it provides access to that data to the user from the node in which data was quickly available.While searching for data on all the nodes in the cluster, if NameNode finds some node to be dead, then without user knowledge NameNode redirects the user to the other node in which the same data is available.<\/p>\n

Without any interruption, data is made available to the user. So in conditions of node failure also data is highly available to the users.<\/p>\n

What were the issues in legacy systems?<\/h3>\n
    \n
  • Data was unavailable due to machine crashing.<\/li>\n
  • HDFS client has to wait for a long period of time to access their data. Most of the time users have to wait for a particular period of time till the website becomes up.<\/li>\n
  • Limited functionalities and features.<\/li>\n
  • Due to unavailability of data, completion of many major projects at organizations gets extended for a long period of time and hence companies have to go through critical situations.<\/li>\n<\/ul>\n

    Conclusion<\/h3>\n

    Hence, In Hadoop data is highly available and accessible despite hardware failure due to multiple copies of data. So, if any node or machines crash or go down, the data we can access data from another path. Learn more HDFS Features.<\/p>\n

    If you find this blog helpful on Hadoop High Availability, so please share your thoughts in the comment section.<\/p>\n","protected":false},"excerpt":{"rendered":"

    This blog provides you a description of the Hadoop HDFS High Availability feature. In this blog firstly we will discuss what exactly high availability is, How Hadoop achieve high availability, what is the need...<\/p>\n","protected":false},"author":1,"featured_media":73145,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[544],"tags":[538,457,539,541,593,543,594],"class_list":["post-1991","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hadoop","tag-apache-hadoop","tag-big-data","tag-big-data-hadoop","tag-hadoop","tag-hadoop-high-availability","tag-hadoop-tutorial","tag-hdfs-high-availability"],"yoast_head":"\nUnderstanding Hadoop High Availability Feature - TechVidvan<\/title>\n<meta name=\"description\" content=\"Understand What is Hadoop High Availability feature,How to achieve High Availability in Hadoop HDFS,Need of HDFS High Availability,issues in legacy systems.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Understanding Hadoop High Availability Feature - TechVidvan\" \/>\n<meta property=\"og:description\" content=\"Understand What is Hadoop High Availability feature,How to achieve High Availability in Hadoop HDFS,Need of HDFS High Availability,issues in legacy systems.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/\" \/>\n<meta property=\"og:site_name\" content=\"TechVidvan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/TechVidvan\/\" \/>\n<meta property=\"article:published_time\" content=\"2017-09-28T11:51:59+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Hadoop-High-Availability-01.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"TechVidvan Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:site\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TechVidvan Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Understanding Hadoop High Availability Feature - TechVidvan","description":"Understand What is Hadoop High Availability feature,How to achieve High Availability in Hadoop HDFS,Need of HDFS High Availability,issues in legacy systems.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/","og_locale":"en_US","og_type":"article","og_title":"Understanding Hadoop High Availability Feature - TechVidvan","og_description":"Understand What is Hadoop High Availability feature,How to achieve High Availability in Hadoop HDFS,Need of HDFS High Availability,issues in legacy systems.","og_url":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/","og_site_name":"TechVidvan","article_publisher":"https:\/\/www.facebook.com\/TechVidvan\/","article_published_time":"2017-09-28T11:51:59+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Hadoop-High-Availability-01.jpg","type":"image\/jpeg"}],"author":"TechVidvan Team","twitter_card":"summary_large_image","twitter_creator":"@vidvantech","twitter_site":"@vidvantech","twitter_misc":{"Written by":"TechVidvan Team","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/#article","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/"},"author":{"name":"TechVidvan Team","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22"},"headline":"Understanding Hadoop High Availability Feature","datePublished":"2017-09-28T11:51:59+00:00","mainEntityOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/"},"wordCount":589,"commentCount":0,"publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Hadoop-High-Availability-01.jpg","keywords":["apache hadoop","big data","big data hadoop","hadoop","Hadoop High Availability","hadoop tutorial","HDFS High Availability"],"articleSection":["Hadoop Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/","url":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/","name":"Understanding Hadoop High Availability Feature - TechVidvan","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/#website"},"primaryImageOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/#primaryimage"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Hadoop-High-Availability-01.jpg","datePublished":"2017-09-28T11:51:59+00:00","description":"Understand What is Hadoop High Availability feature,How to achieve High Availability in Hadoop HDFS,Need of HDFS High Availability,issues in legacy systems.","breadcrumb":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/#primaryimage","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Hadoop-High-Availability-01.jpg","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Hadoop-High-Availability-01.jpg","width":1200,"height":628,"caption":"High Availability Feature in Hadoop"},{"@type":"BreadcrumbList","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-high-availability\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/techvidvan.com\/tutorials\/"},{"@type":"ListItem","position":2,"name":"Understanding Hadoop High Availability Feature"}]},{"@type":"WebSite","@id":"https:\/\/techvidvan.com\/tutorials\/#website","url":"https:\/\/techvidvan.com\/tutorials\/","name":"TechVidvan Blogs","description":"","publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/techvidvan.com\/tutorials\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/techvidvan.com\/tutorials\/#organization","name":"TechVidvan","url":"https:\/\/techvidvan.com\/tutorials\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","width":200,"height":50,"caption":"TechVidvan"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/TechVidvan\/","https:\/\/x.com\/vidvantech"]},{"@type":"Person","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22","name":"TechVidvan Team","description":"The TechVidvan Team delivers practical, beginner-friendly tutorials on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our experts are here to help you upskill and excel in today\u2019s tech industry."}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/1991","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/comments?post=1991"}],"version-history":[{"count":0,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/1991\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media\/73145"}],"wp:attachment":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media?parent=1991"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/categories?post=1991"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/tags?post=1991"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}