One Star

Extracting table from Mysql to hive using talend

Hi,
I want to extract a table from mysql to hive using talend (big data studio)....
Can anyone guide me how to do that???
16 REPLIES
Moderator

Re: Extracting table from Mysql to hive using talend

Hi,
You can use sqoop to import mysql table into hdfs firstly then using TalendHelpCenter:thiveload to write data into hive table.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Extracting table from Mysql to hive using talend

Thanks. Smiley Happy
Moderator

Re: Extracting table from Mysql to hive using talend

Hi seemi,
Is it working for you? Feel free to post your issue on forum.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Extracting table from Mysql to hive using talend

Hi.... I am able to extract tables to & from hdfs & mysql using sqoop, but the extraction is not in a table fromat from mysql to hive... I need help here...

& is there any other way to do this?? Coz... Talend is itself an ETL... (so I want to use its capablities to extract data using some mappings or anything else which is available .....
Moderator

Re: Extracting table from Mysql to hive using talend

Hi,
Hi.... I am able to extract tables to & from hdfs & mysql using sqoop, but the extraction is not in a table fromat from mysql to hive... I need help here...

What's the actual result? Could you please upload the screenshots with us?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Extracting table from Mysql to hive using talend

Hi,
Hi.... I am able to extract tables to & from hdfs & mysql using sqoop, but the extraction is not in a table fromat from mysql to hive... I need help here...

What's the actual result? Could you please upload the screenshots with us?
Best regards
Sabrina

Hi Sabrina,
Attathced is the screen shot...
I have loaded the data into mysql... Now I want to load the same mysql table into hive without using sqoop.... Plz help me out...
Regards,
Seemi.
Moderator

Re: Extracting table from Mysql to hive using talend

Hi,
It seems your screenshots are missing.
Pay more attention on limitation of Image Upload Section:20 images per post, each image must be less then 1024x768 pixels and 200 KB, png is preferred.
I have loaded the data into mysql... Now I want to load the same mysql table into hive without using sqoop.

Actually, you cannot load data from mysql to hive directly.
You could try tmysqlinput==>thdfsoutput then thiveload
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Extracting table from Mysql to hive using talend

Hi,
It seems your screenshots are missing.
Pay more attention on limitation of Image Upload Section:20 images per post, each image must be less then 1024x768 pixels and 200 KB, png is preferred.
I have loaded the data into mysql... Now I want to load the same mysql table into hive without using sqoop.

Actually, you cannot load data from mysql to hive directly.
You could try tmysqlinput==>thdfsoutput then thiveload
Best regards
Sabrina

Hi,
I mable to extract my table from mysql to hdfs... but it is creating an empty table in hive... plz guide...
One Star

Re: Extracting table from Mysql to hive using talend

Can anyone tell me what does the following error means???

Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
One Star

Re: Extracting table from Mysql to hive using talend

I have a table in hive.... now i want to create another table in hive using the contents of already existing table in hive.... I am trying to use insert query but not able to do it.....

my query is :
"INSERT into table
importedcustomer
(
SELECT *
FROM databasename.customer
)"


plz guide me....
Moderator

Re: Extracting table from Mysql to hive using talend

Hi,
Did you use TalendHelpCenter:tHiveRow to execute your query?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Extracting table from Mysql to hive using talend

Hi,
My issue is resolved now..... there were some permission errors......
Thnx a lot for your guidance..... Smiley Happy
One Star

Re: Extracting table from Mysql to hive using talend

Hi....
I am trying to join tables in mysql & create a new table with that..... but getting error in that..
attaching the image ....

Error is as follows :

Exception in component tELTMysqlOutput_1
java.sql.SQLException: Access denied for user ''@'localhost' (using password: NO)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1074)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4096)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4028)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:919)
at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:4612)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1296)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2412)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2445)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2230)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:813)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:399)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:334)
at java.sql.DriverManager.getConnection(DriverManager.java:579)
at java.sql.DriverManager.getConnection(DriverManager.java:221)
at sampleproject.tjoinsql_0_1.tjoinsql.tELTMysqlMap_1Process(tjoinsql.java:457)
at sampleproject.tjoinsql_0_1.tjoinsql.tMysqlConnection_1Process(tjoinsql.java:1399)
at sampleproject.tjoinsql_0_1.tjoinsql.runJobInTOS(tjoinsql.java:1617)
at sampleproject.tjoinsql_0_1.tjoinsql.main(tjoinsql.java:1482)
disconnected
Job tjoinsql ended at 14:59 26/05/2014.
Community Manager

Re: Extracting table from Mysql to hive using talend

Exception in component tELTMysqlOutput_1
java.sql.SQLException: Access denied for user ''@'localhost' (using password: NO)

The error indicates wrong username/password, check that you have configured right connection parameters on tELTMysqlMap.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Extracting table from Mysql to hive using talend

All the credentials are correct... but still it is showing the error..
Community Manager

Re: Extracting table from Mysql to hive using talend

Can you try to create a Mysql connection with the same credentials under Metadata of Repository perspective and check the connection? To me, it is really an authentication issue.

Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business