Sqoop Codegen is a tool that generates the Java classes that encapsulate and interpret the imported records. The article has explained the entire concept of Sqoop Codegen along with its syntax and arguments.<\/p>\n
Let us first start with an introduction to Sqoop Codegen.<\/p>\n
<\/p>\n
The primary purpose of the Codegen tool is to generate the Java classes that encapsulate and interpret the imported records. The Java definition of the record is instantiated as a part of the import process. This can also be performed separately.<\/p>\n
For example<\/strong>, if the Java source is lost, then it can be recreated. The new versions of the class can be created which use different delimiters between the fields, and so on.<\/p>\n The Syntax for Sqoop Codegen is:<\/p>\n The Hadoop generic arguments must precede the codegen arguments, and the codegen arguments can be passed in any order.<\/p>\n The Common arguments are:<\/strong><\/p>\n The Code generation arguments are:<\/b><\/p>\n The Output line formatting arguments are:<\/strong><\/p>\n The Input parsing arguments are:<\/strong><\/p>\n The Hive arguments are:<\/strong><\/p>\n If the Hive arguments were provided to the code generation tool, then the Sqoop generates the file containing HQL statements for creating a table and loading data.<\/p>\n This tool checks java code. If the java code is lost then it will be regenerated by using \u201ccodegen tool\u201d.<\/p>\n Example: In this example we are recreating the record interpretation code for the emp_info table of a demo_db database:<\/p>\n We can use the below command for verifying the generated java code<\/p>\n I hope after reading this article, you have understood the Sqoop Codegen. The article had explained the whole concept related to the Codegen tool. The article had enlisted the syntax, purpose, as well as the arguments for Sqoop Codegen.<\/p>\n If you have any doubts related to this tool, feel free to share your comment with us.<\/p>\n","protected":false},"excerpt":{"rendered":" Sqoop Codegen is a tool that generates the Java classes that encapsulate and interpret the imported records. The article has explained the entire concept of Sqoop Codegen along with its syntax and arguments. Let...<\/p>\n","protected":false},"author":1,"featured_media":79667,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3163],"tags":[3179,3180,3181],"class_list":["post-79621","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sqoop","tag-examples-of-sqoop-codegen","tag-sqoop-codegen-arguments","tag-sqoop-codegen"],"yoast_head":"\nSyntax for Sqoop Codegen<\/h3>\n
$ sqoop codegen (generic-args) (codegen-args)\n$ sqoop-codegen (generic-args) (codegen-args)\n<\/pre>\n
Sqoop Codegen Arguments<\/h3>\n
\n\n
\n Argument<\/b><\/td>\n Description<\/b><\/td>\n<\/tr>\n \n –connect <jdbc-uri><\/span><\/td>\n It specify the JDBC connect string<\/span><\/td>\n<\/tr>\n \n –connection-manager <class-name><\/span><\/td>\n It specify the connection manager class to be use<\/span><\/td>\n<\/tr>\n \n –driver <class-name><\/span><\/td>\n Manually specify JDBC driver class to use<\/span><\/td>\n<\/tr>\n \n –hadoop-mapred-home <dir><\/span><\/td>\n Override $HADOOP_MAPRED_HOME<\/span><\/td>\n<\/tr>\n \n –help<\/span><\/td>\n Print usage instructions<\/span><\/td>\n<\/tr>\n \n –password-file<\/span><\/td>\n It will set the path for a file containing authentication password<\/span><\/td>\n<\/tr>\n \n -P<\/span><\/td>\n Read the password from console<\/span><\/td>\n<\/tr>\n \n –password <password><\/span><\/td>\n Set authentication password<\/span><\/td>\n<\/tr>\n \n –username <username><\/span><\/td>\n Set authentication username<\/span><\/td>\n<\/tr>\n \n –verbose<\/span><\/td>\n It will print more information while working<\/span><\/td>\n<\/tr>\n \n –connection-param-file <filename><\/span><\/td>\n Optional properties file that provides connection parameters<\/span><\/td>\n<\/tr>\n \n –relaxed-isolation<\/span><\/td>\n It will set the connection transaction isolation to the read uncommitted for the mappers.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n \n\n
\n Argument<\/b><\/td>\n Description<\/b><\/td>\n<\/tr>\n \n –bindir <dir><\/span><\/td>\n Specify the Output directory for the compiled objects<\/span><\/td>\n<\/tr>\n \n –class-name <name><\/span><\/td>\n It will set the generated class name. This will override the –package-name. When it is combined with the –jar-file, then it will set the input class.<\/span><\/td>\n<\/tr>\n \n –jar-file <file><\/span><\/td>\n It will disable the code generation; use specified jar<\/span><\/td>\n<\/tr>\n \n –outdir <dir><\/span><\/td>\n Specify the Output directory for the generated code<\/span><\/td>\n<\/tr>\n \n –package-name <name><\/span><\/td>\n It will put the auto-generated classes in this package<\/span><\/td>\n<\/tr>\n \n –map-column-java <m><\/span><\/td>\n It is used for Overriding the default mapping from SQL type to the Java type for the configured columns.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n \n\n
\n Argument<\/b><\/td>\n Description<\/b><\/td>\n<\/tr>\n \n –enclosed-by <char><\/span><\/td>\n It will set the required field enclosing character<\/span><\/td>\n<\/tr>\n \n –escaped-by <char><\/span><\/td>\n It will set the escape character<\/span><\/td>\n<\/tr>\n \n –fields-terminated-by <char><\/span><\/td>\n It will set the field separator character<\/span><\/td>\n<\/tr>\n \n –lines-terminated-by <char><\/span><\/td>\n This argument will set end-of-line character<\/span><\/td>\n<\/tr>\n \n –mysql-delimiters<\/span><\/td>\n This argument specify to use MySQL\u2019s default delimiter set: fields: , lines: \\n escaped-by: \\ optionally-enclosed-by: ‘<\/span><\/td>\n<\/tr>\n \n –optionally-enclosed-by <char><\/span><\/td>\n It will set the field enclosing character<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n \n\n
\n Argument<\/b><\/td>\n Description<\/b><\/td>\n<\/tr>\n \n –input-enclosed-by <char><\/span><\/td>\n Sets the required field encloser<\/span><\/td>\n<\/tr>\n \n –input-escaped-by <char><\/span><\/td>\n Sets the input escape character<\/span><\/td>\n<\/tr>\n \n –input-fields-terminated-by <char><\/span><\/td>\n Sets the input field separator<\/span><\/td>\n<\/tr>\n \n –input-lines-terminated-by <char><\/span><\/td>\n Sets the input end-of-line character<\/span><\/td>\n<\/tr>\n \n –input-optionally-enclosed-by <char><\/span><\/td>\n Sets a field enclosing character<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n \n\n
\n Argument<\/b><\/td>\n Description<\/b><\/td>\n<\/tr>\n \n –hive-home <dir><\/span><\/td>\n Override $HIVE_HOME<\/span><\/td>\n<\/tr>\n \n –hive-import<\/span><\/td>\n Import tables into Hive. It uses the Hive\u2019s default delimiters if none are set.<\/span><\/td>\n<\/tr>\n \n –hive-overwrite<\/span><\/td>\n Overwrite the existing data in the Hive table.<\/span><\/td>\n<\/tr>\n \n –create-hive-table<\/span><\/td>\n If we set this option, then the Sqoop job will fail if a target hive table exits. This property, by default, is set to false.<\/span><\/td>\n<\/tr>\n \n –hive-table <table-name><\/span><\/td>\n It will set a table name to use while importing to Hive.<\/span><\/td>\n<\/tr>\n \n –hive-drop-import-delims<\/span><\/td>\n It will drop the <\/span>\\n<\/span><\/i>, <\/span>\\r<\/span><\/i>, and <\/span>\\01<\/span><\/i> from the string fields while importing to Hive.<\/span><\/td>\n<\/tr>\n \n –hive-delims-replacement<\/span><\/td>\n It will replace the <\/span>\\n<\/span><\/i>, <\/span>\\r<\/span><\/i>, and <\/span>\\01<\/span><\/i> from the string fields with the user defined string while importing to Hive.<\/span><\/td>\n<\/tr>\n \n –hive-partition-key<\/span><\/td>\n It will specify the name of the hive field to which the partition are sharded on.<\/span><\/td>\n<\/tr>\n \n –hive-partition-value <v><\/span><\/td>\n It will specify the String-value which serves as a partition key for this imported into the hive in this job.<\/span><\/td>\n<\/tr>\n \n –map-column-hive <map><\/span><\/td>\n Override the default mapping from the SQL type to the Hive type for the configured columns.<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n Advantage of Sqoop Codegen<\/h3>\n
Example Invocations<\/h3>\n
$ sqoop codegen --connect jdbc:mysql:\/\/localhost\/demo_db \\\n --table emp_info\n<\/pre>\n
$ hadoop fs -ls \/home\/demo_db\/emp_infojavacode\/result\n<\/pre>\n
Summary<\/h3>\n