tSqoopImport for Teradata throwing an exception

One Star

tSqoopImport for Teradata throwing an exception

Hi,
I have configured terdata-sqoop-connector with sqoop to import the table from Teradata after following link <> and it is working fine (on command line).
And when I am trying the same thing with Talend to import the table from teradata using tSqoopImport, I am getting an Exception. The job designed for that is like below:
tLibrrayLoad1(sqoop-connector-teradata-1.2c5.jar) --onSubjobOK--> tLibrrayLoad2(tdgssconfig.jar) --onSubjobOK--> tLibrrayLoad3(terajdbc4.jar) --onSubjobOK--> tSqoopImport
Below is the Exception:
---------------------------
Starting job input_table_Copy1 at 18:25 03/02/2015.
connecting to socket on port 3965
connected
Warning: Parameter "NAMENODE_URI" is a new parameter of Implicit_Context_Context
Warning: Parameter "RESOURCE_MGR_NM" is a new parameter of Implicit_Context_Context
Warning: Parameter "RESOURCE_MGR_SCHD_ADDR" is a new parameter of Implicit_Context_Context
Warning: Parameter "USER_ID" is a new parameter of Implicit_Context_Context
Warning: Parameter "HDFS_TMP" is a new parameter of Implicit_Context_Context
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
: org.apache.sqoop.Sqoop - Running Sqoop version: 1.4.4-cdh5.0.0
: org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
: org.apache.sqoop.ConnFactory - Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
: org.apache.sqoop.manager.SqlManager - Using default fetchSize of 1000
: org.apache.sqoop.tool.CodeGenTool - Beginning code generation
: org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM test_table AS t WHERE 1=0
: org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM test_table AS t WHERE 1=0
: org.apache.sqoop.orm.CompilationManager - $HADOOP_MAPRED_HOME is not set
Note: /tmp/sqoop-hadoopuser/compile/3be2afc19ff0e382807e7ebe645aebf7/test_table.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
: org.apache.sqoop.orm.CompilationManager - Writing jar file: /tmp/sqoop-hadoopuser/compile/3be2afc19ff0e382807e7ebe645aebf7/test_table.jar
: org.apache.sqoop.mapreduce.ImportJobBase - Beginning import of test_table
: org.apache.hadoop.conf.Configuration.deprecation - mapred.jar is deprecated. Instead, use mapreduce.job.jar
: org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM test_table AS t WHERE 1=0
: org.apache.hadoop.conf.Configuration.deprecation - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
: org.apache.sqoop.mapreduce.JobBase - SQOOP_HOME is unset. May not be able to find all job dependencies.
: org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at ubuntuvm/10.100.100.1:8050
: org.apache.hadoop.mapreduce.JobSubmitter - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
: org.apache.sqoop.mapreduce.db.DBInputFormat - Using read commited transaction isolation
: org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat - BoundingValsQuery: SELECT MIN(INPUT_FIELD1), MAX(INPUT_FIELD1) FROM test_table
: org.apache.sqoop.mapreduce.db.TextSplitter - Generating splits for a textual index column.
: org.apache.sqoop.mapreduce.db.TextSplitter - If your database sorts in a case-insensitive order, this may result in a partial import or duplicate records.
: org.apache.sqoop.mapreduce.db.TextSplitter - You are strongly encouraged to choose an integral split column.
: org.apache.hadoop.mapreduce.JobSubmitter - number of splits:4
: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1422887942281_0012
: org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1422887942281_0012
: org.apache.hadoop.mapreduce.Job - The url to track the job:
: org.apache.hadoop.mapreduce.Job - Running job: job_1422887942281_0012
: org.apache.hadoop.mapreduce.Job - Job job_1422887942281_0012 running in uber mode : false
: org.apache.hadoop.mapreduce.Job -  map 0% reduce 0%
: org.apache.hadoop.mapreduce.Job - Task Id : attempt_1422887942281_0012_m_000002_0, Status : FAILED
Error: java.lang.NullPointerException
at com.teradata.tdgss.jtdgss.TdgssConfigApi.GetMechanisms(TdgssConfigApi.java:141)
at com.teradata.tdgss.jtdgss.TdgssManager.<init>(TdgssManager.java:43)
at com.teradata.tdgss.jtdgss.TdgssManager.<clinit>(TdgssManager.java:29)
at com.teradata.jdbc.jdbc.GenericTeraEncrypt.getTDgssVersion(GenericTeraEncrypt.java:640)
at com.teradata.jdbc.jdbc_4.parcel.ClientConfigParcel.toStream(ClientConfigParcel.java:54)
at com.teradata.jdbc.jdbc_4.io.TDPacket.addParcel(TDPacket.java:98)
at com.teradata.jdbc.jdbc.GenericLogonController.run(GenericLogonController.java:94)
at com.teradata.jdbc.jdbc_4.TDSession.<init>(TDSession.java:205)
at com.teradata.jdbc.jdk6.JDK6_SQL_Connection.<init>(JDK6_SQL_Connection.java:35)
at com.teradata.jdbc.jdk6.JDK6ConnectionFactory.constructSQLConnection(JDK6ConnectionFactory.java:25)
at com.teradata.jdbc.jdbc.ConnectionFactory.createConnection(ConnectionFactory.java:179)
at com.teradata.jdbc.jdbc.ConnectionFactory.createConnection(ConnectionFactory.java:169)
at com.teradata.jdbc.TeraDriver.doConnect(TeraDriver.java:232)
at com.teradata.jdbc.TeraDriver.connect(TeraDriver.java:158)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Error: java.lang.NullPointerException
at com.teradata.tdgss.jtdgss.TdgssConfigApi.GetMechanisms(TdgssConfigApi.java:141)
at com.teradata.tdgss.jtdgss.TdgssManager.<init>(TdgssManager.java:43)
at com.teradata.tdgss.jtdgss.TdgssManager.<clinit>(TdgssManager.java:29)
at com.teradata.jdbc.jdbc.GenericTeraEncrypt.getTDgssVersion(GenericTeraEncrypt.java:640)
at com.teradata.jdbc.jdbc_4.parcel.ClientConfigParcel.toStream(ClientConfigParcel.java:54)
at com.teradata.jdbc.jdbc_4.io.TDPacket.addParcel(TDPacket.java:98)
at com.teradata.jdbc.jdbc.GenericLogonController.run(GenericLogonController.java:94)
at com.teradata.jdbc.jdbc_4.TDSession.<init>(TDSession.java:205)
at com.teradata.jdbc.jdk6.JDK6_SQL_Connection.<init>(JDK6_SQL_Connection.java:35)
at com.teradata.jdbc.jdk6.JDK6ConnectionFactory.constructSQLConnection(JDK6ConnectionFactory.java:25)
at com.teradata.jdbc.jdbc.ConnectionFactory.createConnection(ConnectionFactory.java:179)
at com.teradata.jdbc.jdbc.ConnectionFactory.createConnection(ConnectionFactory.java:169)
: org.apache.hadoop.mapreduce.Job - Task Id : attempt_1422887942281_0012_m_000001_0, Status : FAILED
: org.apache.hadoop.mapreduce.Job - Task Id : attempt_1422887942281_0012_m_000000_0, Status : FAILED
at com.teradata.jdbc.TeraDriver.doConnect(TeraDriver.java:232)
at com.teradata.jdbc.TeraDriver.connect(TeraDriver.java:158)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Error: java.lang.NullPointerException
at com.teradata.tdgss.jtdgss.TdgssConfigApi.GetMechanisms(TdgssConfigApi.java:141)
at com.teradata.tdgss.jtdgss.TdgssManager.<init>(TdgssManager.java:43)
at com.teradata.tdgss.jtdgss.TdgssManager.<clinit>(TdgssManager.java:29)
at com.teradata.jdbc.jdbc.GenericTeraEncrypt.getTDgssVersion(GenericTeraEncrypt.java:640)
at com.teradata.jdbc.jdbc_4.parcel.ClientConfigParcel.toStream(ClientConfigParcel.java:54)
at com.teradata.jdbc.jdbc_4.io.TDPacket.addParcel(TDPacket.java:98)
at com.teradata.jdbc.jdbc.GenericLogonController.run(GenericLogonController.java:94)
at com.teradata.jdbc.jdbc_4.TDSession.<init>(TDSession.java:205)
at com.teradata.jdbc.jdk6.JDK6_SQL_Connection.<init>(JDK6_SQL_Connection.java:35)
at com.teradata.jdbc.jdk6.JDK6ConnectionFactory.constructSQLConnection(JDK6ConnectionFactory.java:25)
at com.teradata.jdbc.jdbc.ConnectionFactory.createConnection(ConnectionFactory.java:179)
at com.teradata.jdbc.jdbc.ConnectionFactory.createConnection(ConnectionFactory.java:169)
at com.teradata.jdbc.TeraDriver.doConnect(TeraDriver.java:232)
at com.teradata.jdbc.TeraDriver.connect(TeraDriver.java:158)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Error: java.lang.NullPointerException
at com.teradata.tdgss.jtdgss.TdgssConfigApi.GetMechanisms(TdgssConfigApi.java:141)
at com.teradata.tdgss.jtdgss.TdgssManager.<init>(TdgssManager.java:43)
at com.teradata.tdgss.jtdgss.TdgssManager.<clinit>(TdgssManager.java:29)
at com.teradata.jdbc.jdbc.GenericTeraEncrypt.getTDgssVersion(GenericTeraEncrypt.java:640)
at com.teradata.jdbc.jdbc_4.parcel.ClientConfigParcel.toStream(ClientConfigParcel.java:54)
at com.teradata.jdbc.jdbc_4.io.TDPacket.addParcel(TDPacket.java:98)
at com.teradata.jdbc.jdbc.GenericLogonController.run(GenericLogonController.java:94)
at com.teradata.jdbc.jdbc_4.TDSession.<init>(TDSession.java:205)
at com.teradata.jdbc.jdk6.JDK6_SQL_Connection.<init>(JDK6_SQL_Connection.java:35)
at com.teradata.jdbc.jdk6.JDK6ConnectionFactory.constructSQLConnection(JDK6ConnectionFactory.java:25)
at com.teradata.jdbc.jdbc.ConnectionFactory.createConnection(ConnectionFactory.java:179)
at com.teradata.jdbc.jdbc.ConnectionFactory.createConnection(ConnectionFactory.java:169)
: org.apache.hadoop.mapreduce.Job - Task Id : attempt_1422887942281_0012_m_000003_0, Status : FAILED
at com.teradata.jdbc.TeraDriver.doConnect(TeraDriver.java:232)
at com.teradata.jdbc.TeraDriver.connect(TeraDriver.java:158)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

One more question I have that, is there any way to configure the sqoop-teradata-connector in Talend or is there any straight forward approach to do it.
Please let me know the solution for this.

Thanks,
Bhavesh
Moderator

Re: tSqoopImport for Teradata throwing an exception

Hi bhavesh25shah,
On which official version did you get that ?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: tSqoopImport for Teradata throwing an exception

Thanks Sabrina for looking into this.
Do you mean version of Talend Studio?
I am using Talend Platform - Universal 5.5.1.r118616
While execution Talend using Sqoop version: 1.4.4-cdh5.0.0 (from log).
All installed component on my machine is 64-bit.

Please let me know if you want more details on this.
Thanks,
Bhavesh

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now