{"id":79627,"date":"2020-08-25T09:00:07","date_gmt":"2020-08-25T03:30:07","guid":{"rendered":"https:\/\/techvidvan.com\/tutorials\/?p=79627"},"modified":"2020-08-25T09:00:07","modified_gmt":"2020-08-25T03:30:07","slug":"apache-sqoop-metastore","status":"publish","type":"post","link":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/","title":{"rendered":"Sqoop Metastore Tutorial – How to setup Sqoop Metastore"},"content":{"rendered":"

Sqoop Metastore is a tool that is used in configuring Sqoop to host the shared metadata repositories. In this Sqoop Metastore article, you will explore what it is, its syntax, arguments, and much more. So let’s start!!!<\/p>\n

 <\/p>\n

What is Sqoop Metastore?<\/h3>\n

Sqoop metastore is a tool for storing Sqoop job information in the central place. It helps collaboration between the Sqoop users and developers.<\/p>\n

For example<\/strong>, a user named user1 creates a job for loading some specific data, then any other user can easily access it from any other node in the cluster and just run it again.<\/p>\n

Purpose of Sqoop Metastore<\/h3>\n

The primary purpose of this tool is to configure Apache Sqoop to host the shared metadata repository. Multiple users or remote users can define and execute the saved jobs defined in this metastore.<\/p>\n

We have to configure the clients in the sqoop-site.xml<\/strong> or with the –meta-connect<\/strong> argument to connect to the metastore.<\/p>\n

How to setup Sqoop Metastore<\/h3>\n

Let us now see the procedure for setting up Sqoop metastore.<\/p>\n

1. <\/strong>First, we have to choose a server for hosting Sqoop metastore. In best practices, it is better to choose administrative or master server.<\/p>\n

2. <\/strong>In second step, we have to decide which user will execute the metastore. After deciding the user, we will create the user and the home directory if required. Then we have to configure the metastore detail in sqoop-xite.xml.<\/p>\n

3. <\/strong>Now we have to update the service configuration for accessing the metastore automatically.<\/p>\n

4. <\/strong>At last, we have to start the metastore.<\/p>\n

The syntax<\/h3>\n
$ sqoop metastore (generic-args) (metastore-args)\n$ sqoop-metastore (generic-args) (metastore-args)\n<\/pre>\n
We have to pass the Hadoop generic arguments before any metastore arguments. We can enter the metastore arguments in any order with respect to each other.<\/p>\n
The Metastore management options are:<\/strong><\/p>\n\n\n\n\n
Argument<\/b><\/td>\nDescription<\/b><\/td>\n<\/tr>\n
–shutdown<\/td>\nIt will shut down the running metastore instance on the same machine.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Running the sqoop-metastore launches the shared HSQLDB database instance on the current machine. The clients can connect to this metastore and create Sqoop jobs that can be shared between the users for execution.<\/p>\n
We can control the location of the metastore\u2019s files on the disk by configuring the sqoop.metastore.server.location<\/strong> property in the conf\/sqoop-site.xml <\/strong>configuration file. This should point to the directory on the local filesystem.<\/p>\n
Metastore is available over the TCP\/IP. We can control the port by the sqoop.metastore.server.port<\/strong> configuration parameter, and by default, it is set to 16000.<\/p>\n
We can connect to the metastore via sqoop.metastore.client.autoconnect.url<\/strong> or –meta-connect argument<\/strong> with the value<\/p>\n
jdbc:hsqldb:hsql:\/\/<server-name>:<port>\/sqoop.<\/pre>\n
For example<\/strong>,<\/p>\n
jdbc:hsqldb:hsql:\/\/metaserver.example.com:16000\/sqoop.\n<\/pre>\n
Sqoop hosts this metastore on the machine within the Hadoop cluster or somewhere else on the network.<\/p>\n
Summary<\/h3>\n
I hope after reading this article, you have clearly understood the Sqoop Metastore tool. If you have any queries related to this topic, then please do share it in the comment box.<\/p>\n","protected":false},"excerpt":{"rendered":"
Sqoop Metastore is a tool that is used in configuring Sqoop to host the shared metadata repositories. In this Sqoop Metastore article, you will explore what it is, its syntax, arguments, and much more....<\/p>\n","protected":false},"author":1,"featured_media":79673,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3163],"tags":[3185,3186,3187,3188],"class_list":["post-79627","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sqoop","tag-purpose-of-sqoop-metastore","tag-sqoop-metastore","tag-sqoop-metastore-arguments","tag-sqoop-metastore-examples"],"yoast_head":"\nSqoop Metastore Tutorial - How to setup Sqoop Metastore - TechVidvan<\/title>\n<meta name=\"description\" content=\"Learn about Apache Sqoop metastore - It is used to store Sqoop job information in a central place. Learn how to setup Apache Sqoop metastore and its syntax.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Sqoop Metastore Tutorial - How to setup Sqoop Metastore - TechVidvan\" \/>\n<meta property=\"og:description\" content=\"Learn about Apache Sqoop metastore - It is used to store Sqoop job information in a central place. Learn how to setup Apache Sqoop metastore and its syntax.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/\" \/>\n<meta property=\"og:site_name\" content=\"TechVidvan\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/TechVidvan\/\" \/>\n<meta property=\"article:published_time\" content=\"2020-08-25T03:30:07+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/08\/Sqoop-Metastore-TV.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"TechVidvan Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:site\" content=\"@vidvantech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TechVidvan Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Sqoop Metastore Tutorial - How to setup Sqoop Metastore - TechVidvan","description":"Learn about Apache Sqoop metastore - It is used to store Sqoop job information in a central place. Learn how to setup Apache Sqoop metastore and its syntax.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/","og_locale":"en_US","og_type":"article","og_title":"Sqoop Metastore Tutorial - How to setup Sqoop Metastore - TechVidvan","og_description":"Learn about Apache Sqoop metastore - It is used to store Sqoop job information in a central place. Learn how to setup Apache Sqoop metastore and its syntax.","og_url":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/","og_site_name":"TechVidvan","article_publisher":"https:\/\/www.facebook.com\/TechVidvan\/","article_published_time":"2020-08-25T03:30:07+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/08\/Sqoop-Metastore-TV.jpg","type":"image\/jpeg"}],"author":"TechVidvan Team","twitter_card":"summary_large_image","twitter_creator":"@vidvantech","twitter_site":"@vidvantech","twitter_misc":{"Written by":"TechVidvan Team","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/#article","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/"},"author":{"name":"TechVidvan Team","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22"},"headline":"Sqoop Metastore Tutorial – How to setup Sqoop Metastore","datePublished":"2020-08-25T03:30:07+00:00","mainEntityOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/"},"wordCount":481,"commentCount":0,"publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/08\/Sqoop-Metastore-TV.jpg","keywords":["Purpose of Sqoop Metastore","Sqoop Metastore","Sqoop Metastore arguments","Sqoop Metastore Examples"],"articleSection":["Sqoop Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/","url":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/","name":"Sqoop Metastore Tutorial - How to setup Sqoop Metastore - TechVidvan","isPartOf":{"@id":"https:\/\/techvidvan.com\/tutorials\/#website"},"primaryImageOfPage":{"@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/#primaryimage"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/#primaryimage"},"thumbnailUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/08\/Sqoop-Metastore-TV.jpg","datePublished":"2020-08-25T03:30:07+00:00","description":"Learn about Apache Sqoop metastore - It is used to store Sqoop job information in a central place. Learn how to setup Apache Sqoop metastore and its syntax.","breadcrumb":{"@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/#primaryimage","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/08\/Sqoop-Metastore-TV.jpg","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2020\/08\/Sqoop-Metastore-TV.jpg","width":1200,"height":628,"caption":"Sqoop Metastore"},{"@type":"BreadcrumbList","@id":"https:\/\/techvidvan.com\/tutorials\/apache-sqoop-metastore\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/techvidvan.com\/tutorials\/"},{"@type":"ListItem","position":2,"name":"Sqoop Metastore Tutorial – How to setup Sqoop Metastore"}]},{"@type":"WebSite","@id":"https:\/\/techvidvan.com\/tutorials\/#website","url":"https:\/\/techvidvan.com\/tutorials\/","name":"TechVidvan Blogs","description":"","publisher":{"@id":"https:\/\/techvidvan.com\/tutorials\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/techvidvan.com\/tutorials\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/techvidvan.com\/tutorials\/#organization","name":"TechVidvan","url":"https:\/\/techvidvan.com\/tutorials\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/","url":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","contentUrl":"https:\/\/techvidvan.com\/tutorials\/wp-content\/uploads\/2024\/03\/techvidvan-logo-200x50-1.webp","width":200,"height":50,"caption":"TechVidvan"},"image":{"@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/TechVidvan\/","https:\/\/x.com\/vidvantech"]},{"@type":"Person","@id":"https:\/\/techvidvan.com\/tutorials\/#\/schema\/person\/e9c26e74dd3d87421f7ada9433b8cd22","name":"TechVidvan Team","description":"The TechVidvan Team delivers practical, beginner-friendly tutorials on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Our experts are here to help you upskill and excel in today\u2019s tech industry."}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/79627","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/comments?post=79627"}],"version-history":[{"count":0,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/posts\/79627\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media\/79673"}],"wp:attachment":[{"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/media?parent=79627"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/categories?post=79627"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techvidvan.com\/tutorials\/wp-json\/wp\/v2\/tags?post=79627"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}