while loading data from hadoop to hive getting extra records

Highlighted
Eight Stars RAJ
Eight Stars

while loading data from hadoop to hive getting extra records

Hi Folks,

 

I am working in talend big data using one month license package. In this package there is no tHiveOutput component. so am using tHiveLoad component in my talend job for data loading from hadoop to hive.

I am getting data from Mssql then loaded into hadoop layer using text file format, then loaded data from hadoop to hive. This is talend job flow.

I have facing issue while loading data from hadoop to hive getting extra records. In tHDFSOutput component using field separator tilde(~) symbol and row separator \n. In tHiveLoad component  also using field separator tilde(~) symbol and row separator \n. I have tried lot methods unable to find the solution for this issue. 

Kindly share information as soon as earliest. 

 

for example :

Mssql table count : 1000

Hive table count : 1010

Thanks
RAJ
Eight Stars RAJ
Eight Stars

Re: while loading data from hadoop to hive getting extra records

Hi folks,
Kindly share your information for this issue.
Thanks
RAJ

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog