Importing data from RDBMS to HDFS using SQOOP in talend

One Star

Importing data from RDBMS to HDFS using SQOOP in talend

Hi,
I'm using TOS for Bigdata. I wanted to fetch table from MySQL to HDFS using SQOOP component.
My job looks like this : tLibraryLoad >> tSqoopImport
When i run this job its giving the error "tool.ImportTool: Imported Failed: java.net.UnknownHostException: myHost".
Eventhough i changed myHost to ip, but still the same error.
If i fetch the data from Mysql to hdfs without using SQOOP in seperate job, working fine.
Here i have used : tMySqlInput >> tHDFSOutput
I don't know whats going wrong when i use tSqoopImport. I'm properly giving the NamenodeURI and JobTracker host.
So where i'm going wrong. In Hadoop lib i have added "mysql-connector-java-5.1.16-bin.jar" and "sqoop-1.4.3-cdh4.6.0.jar" jars as well.
I have attached the screen shot of the job.
Thanks,
ShreeCS
One Star

Re: Importing data from RDBMS to HDFS using SQOOP in talend

Hi,
Is there any updates on the above mentioned issue.
I have copied both mySQL and sqoop jar in the $HADOOP_HOME/lib directory. Still facing the same issue.
Error :
Starting job rdbms_sqoop at 12:42 09/04/2014.

14/04/09 12:42:25 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
connecting to socket on port 3513
connected
14/04/09 12:42:25 WARN sqoop.ConnFactory: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
14/04/09 12:42:25 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/04/09 12:42:25 INFO tool.CodeGenTool: Beginning code generation
14/04/09 12:42:26 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
14/04/09 12:42:26 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `orders` AS t LIMIT 1
14/04/09 12:42:26 INFO orm.CompilationManager: $HADOOP_MAPRED_HOME is not set
Note: \tmp\sqoop-Appladmin\compile\f5fc28709ceb16cf63d9020e0f0d11da\orders.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
14/04/09 12:42:28 INFO orm.CompilationManager: Writing jar file: \tmp\sqoop-Appladmin\compile\f5fc28709ceb16cf63d9020e0f0d11da\orders.jar
14/04/09 12:42:28 WARN manager.MySQLManager: It looks like you are importing from mysql.
14/04/09 12:42:28 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
14/04/09 12:42:28 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
14/04/09 12:42:28 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
14/04/09 12:42:28 INFO mapreduce.ImportJobBase: Beginning import of orders
14/04/09 12:42:28 WARN mapreduce.JobBase: SQOOP_HOME is unset. May not be able to find all job dependencies.
14/04/09 12:42:29 ERROR tool.ImportTool: Imported Failed: java.net.UnknownHostException: cldx-433-377
Exception in component tSqoopImport_1
java.lang.Exception: The Sqoop import job has failed
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.tSqoopImport_1Process(rdbms_sqoop.java:416)
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.tLibraryLoad_1Process(rdbms_sqoop.java:334)
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.runJobInTOS(rdbms_sqoop.java:636)
at newproject.rdbms_sqoop_0_1.rdbms_sqoop.main(rdbms_sqoop.java:502)
disconnected
Job rdbms_sqoop ended at 12:42 09/04/2014.

Thanks,
ShreeCS

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now