{"id":260,"date":"2017-10-03T06:59:40","date_gmt":"2017-10-03T06:59:40","guid":{"rendered":"http:\/\/techvidvan.com\/tutorials\/?p=260"},"modified":"2017-10-03T06:59:40","modified_gmt":"2017-10-03T06:59:40","slug":"hadoop-mapreduce-key-value-pair","status":"publish","type":"post","link":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/","title":{"rendered":"What is MapReduce Key Value Pair in Hadoop?"},"content":{"rendered":"

In this Hadoop\u00a0tutorial<\/strong>, we are going to provide you a complete introduction to MapReduce Key Value Pair.<\/p>\n

First of all we will discuss what is a key value pair in Hadoop, How key value pair is generated in MapReduce. At last we will explain MapReduce key value pair generation with examples.<\/p>\n

\"Key<\/a><\/p>\n

What is Key Value Pair in Hadoop?<\/h3>\n

Key-value pair in MapReduce is the record entity that Hadoop MapReduce accepts for execution.<\/p>\n

We use Hadoop mainly for data Analysis. It deals with structured, unstructured and semi-structured data. With Hadoop, if the schema is static we can directly work on the column instead of key value. But, if the schema is not static we will work on a key value.<\/p>\n

Keys value is not the intrinsic properties of the data.\u00a0 But they are chosen by user analyzing the data.<\/p>\n

MapReduce is the core component of Hadoop, which provides data processing. It performs processing by breaking the job by into two phases: Map phase<\/strong> and Reduce phase<\/strong>. Each phase has key-value as input and output.<\/p>\n

MapReduce Key value pair generation in Hadoop<\/h3>\n

In MapReduce job execution, before sending data to the mapper<\/strong>, first convert it into key-value pairs. Because mapper only key-value pairs of data.<\/p>\n

Key-value pair in MapReduce is generated as follows:<\/p>\n

InputSplit –\u00a0<\/strong>It is the logical representation of data which InputFormat<\/strong> generates. In MapReduce program it describes a unit of work that contains a single map task.<\/p>\n

RecordReader –\u00a0<\/strong>It communicates with the InputSplit. After that it converts the data into key value pairs suitable for reading by the Mapper. RecordReader by default uses TextInputFormat\u00a0 to convert data into key value pairs.<\/p>\n

In MapReduce job execution, the map function processes a certain key-value pair. Then emits a certain number of key-value pairs. The Reduce function processes the values grouped by the same key.<\/p>\n

Then emits another set of key-value pairs as the output.\u00a0 The Map output types should match the input types of the Reduce as shown below:<\/p>\n

    \n
  • Map:<\/strong>\u00a0(K1, V1) -> list (K2, V2)<\/li>\n
  • Reduce:<\/strong>\u00a0{(K2, list (V2}) -> list (K3, V3)<\/li>\n<\/ul>\n

    On what basis is a key-value pair generated in Hadoop?<\/h3>\n

    MapReduce Key-value pair generation totally depends on the data set. Also depends on the required output. Framework specifies key-value pair in 4 places: Map input\/output, Reduce input\/output.<\/p>\n

    1. Map Input<\/h4>\n

    Map Input by default takes the line offset as the key. The content of the line is value as Text. We can modify them; by using the custom input format.<\/p>\n

    2. Map Output<\/h4>\n

    The Map is responsible to filter the data. It also provides the environment to group the data on the basis of key.<\/p>\n

      \n
    • Key\u2013<\/strong> It is field\/ text\/ object on which the data groups and aggregates on the\u00a0reducer<\/strong>.<\/li>\n
    • Value\u2013<\/strong> It is the field\/ text\/ object which each individual reduces method handles.<\/li>\n<\/ul>\n

      3. Reduce Input<\/h4>\n

      Map output is input to reduce. So it\u2019s same as Map-Output.<\/p>\n

      4. Reduce Output<\/h4>\n

      It totally depends on the required output.<\/p>\n

      MapReduce Key-value Pair Example<\/h3>\n

      For example, the content of the file which\u00a0HDFS<\/strong>\u00a0stores are\u00a0Chandler is Joey Mark is John<\/strong>. So, now by using InputFormat, we will define how this file will split and read. By default, RecordReader uses TextInputFormat to convert this file into a key-value pair.<\/p>\n

        \n
      • Key \u2013\u00a0<\/strong>It is offset of the beginning of the line within the file.<\/li>\n
      • Value \u2013 \u00a0<\/strong>It is the content of the line, excluding line terminators.<\/li>\n<\/ul>\n

        Here, Key<\/strong>\u00a0is 0 and\u00a0Value<\/strong>\u00a0is Chandler is Joey Mark is John.<\/p>\n

        Conclusion<\/h3>\n

        In conclusion, we can say that, key-value is just a record entity that MapReduce accepts for execution. InputSplit and RecordReader generate Key-value pair. Hence, the key is byte offset and value is the content of the line.<\/p>\n

        Hope you liked this blog. If you have any suggestion or query related to MapReduce key value pair so please leave a comment in a section given below.<\/p>\n","protected":false},"excerpt":{"rendered":"

        In this Hadoop\u00a0tutorial, we are going to provide you a complete introduction to MapReduce Key Value Pair. First of all we will discuss what is a key value pair in Hadoop, How key value...<\/p>\n","protected":false},"author":1,"featured_media":73189,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[570],"tags":[538,457,539,541,576,577],"class_list":["post-260","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-mapreduce","tag-apache-hadoop","tag-big-data","tag-big-data-hadoop","tag-hadoop","tag-key-value-pair","tag-mapreduce-key-value-pair"],"yoast_head":"\nWhat is MapReduce Key Value Pair in Hadoop? - TechVidvan<\/title>\n<meta name=\"description\" content=\"Introduction to MapReduce Key value pair cover what is Hadoop Key Value Pair, how Hadoop generate Key value pair, Hadoop key value pair example in MapReduce\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What is MapReduce Key Value Pair in Hadoop? - TechVidvan\" \/>\n<meta property=\"og:description\" content=\"Introduction to MapReduce Key value pair cover what is Hadoop Key Value Pair, how Hadoop generate Key value pair, Hadoop key value pair example in MapReduce\" \/>\n<meta property=\"og:url\" content=\"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/\" \/>\n<meta property=\"og:site_name\" content=\"TechVidvan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/TechVidvan\/\" \/>\n<meta property=\"article:published_time\" content=\"2017-10-03T06:59:40+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Key-Value-Pairing-in-Hadoop-MapReduce-01.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"TechVidvan Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:site\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TechVidvan Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"What is MapReduce Key Value Pair in Hadoop? - TechVidvan","description":"Introduction to MapReduce Key value pair cover what is Hadoop Key Value Pair, how Hadoop generate Key value pair, Hadoop key value pair example in MapReduce","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/","og_locale":"en_US","og_type":"article","og_title":"What is MapReduce Key Value Pair in Hadoop? - TechVidvan","og_description":"Introduction to MapReduce Key value pair cover what is Hadoop Key Value Pair, how Hadoop generate Key value pair, Hadoop key value pair example in MapReduce","og_url":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/","og_site_name":"TechVidvan","article_publisher":"https:\/\/www.facebook.com\/TechVidvan\/","article_published_time":"2017-10-03T06:59:40+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Key-Value-Pairing-in-Hadoop-MapReduce-01.jpg","type":"image\/jpeg"}],"author":"TechVidvan Team","twitter_card":"summary_large_image","twitter_creator":"@vidvantech","twitter_site":"@vidvantech","twitter_misc":{"Written by":"TechVidvan Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/#article","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/"},"author":{"name":"TechVidvan Team","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22"},"headline":"What is MapReduce Key Value Pair in Hadoop?","datePublished":"2017-10-03T06:59:40+00:00","mainEntityOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/"},"wordCount":652,"commentCount":0,"publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Key-Value-Pairing-in-Hadoop-MapReduce-01.jpg","keywords":["apache hadoop","big data","big data hadoop","hadoop","Key Value Pair","MapReduce key-value pair"],"articleSection":["MapReduce Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/","url":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/","name":"What is MapReduce Key Value Pair in Hadoop? - TechVidvan","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/#website"},"primaryImageOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/#primaryimage"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Key-Value-Pairing-in-Hadoop-MapReduce-01.jpg","datePublished":"2017-10-03T06:59:40+00:00","description":"Introduction to MapReduce Key value pair cover what is Hadoop Key Value Pair, how Hadoop generate Key value pair, Hadoop key value pair example in MapReduce","breadcrumb":{"@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/#primaryimage","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Key-Value-Pairing-in-Hadoop-MapReduce-01.jpg","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2019\/11\/Key-Value-Pairing-in-Hadoop-MapReduce-01.jpg","width":1200,"height":628,"caption":"Key Value Pairing in Hadoop MapReduce"},{"@type":"BreadcrumbList","@id":"https:\/\/techvidvan.com\/tutorials\/hadoop-mapreduce-key-value-pair\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/techvidvan.com\/tutorials\/"},{"@type":"ListItem","position":2,"name":"What is MapReduce Key Value Pair in Hadoop?"}]},{"@type":"WebSite","@id":"https:\/\/techvidvan.com\/tutorials\/#website","url":"https:\/\/techvidvan.com\/tutorials\/","name":"TechVidvan Blogs","description":"","publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/techvidvan.com\/tutorials\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/techvidvan.com\/tutorials\/#organization","name":"TechVidvan","url":"https:\/\/techvidvan.com\/tutorials\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","width":200,"height":50,"caption":"TechVidvan"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/TechVidvan\/","https:\/\/x.com\/vidvantech"]},{"@type":"Person","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22","name":"TechVidvan Team","description":"The TechVidvan Team delivers practical, beginner-friendly tutorials on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our experts are here to help you upskill and excel in today\u2019s tech industry."}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/260","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/comments?post=260"}],"version-history":[{"count":0,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/260\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media\/73189"}],"wp:attachment":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media?parent=260"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/categories?post=260"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/tags?post=260"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}