In Big data Job, look up Does it work?

Six Stars

In Big data Job, look up Does it work?

Hello everyone,

 

Look up in big data, will it works or not?

 

i developed the big data job which lookup from red shift. 

but its giving me duplicated in red shift. how to achieve ?

 

waiting for your reply..

Employee

Re: In Big data Job, look up Does it work?

Hi,

 

   Could you please refer the below link for your query?

 

https://help.talend.com/reader/j~SccKoZW8kEeYb0teesxw/f7739LCAGS8P4BiFbQCLzw

 

  Also could you please share the screenshot of your job flow and the tmap lookup in detail?

 

Warm Regards,
Nikhil Thampi

Please appreciate our members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)


Warm Regards,
Nikhil Thampi
Please appreciate our members by giving Kudos for spending their time for your query. If your query is answered, please mark the topic as resolved :-)
Highlighted
Thirteen Stars

Re: In Big data Job, look up Does it work?

HI Mahesh

 

batch job run in parallel and each worker make lookup independently
as a fast solution you could reduce the number of workers to 1

 

 

 

 

 

-----------
Six Stars

Re: In Big data Job, look up Does it work?

@vapukov could you explain me clearly how to do this

Six Stars

Re: In Big data Job, look up Does it work?

Big Data Spark Job:--

I am using tredshiftinput for lookup 

problem is i am unable to do lookup from red shift data 

its inserting the duplicates 

 

any solution how to do this 

 

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.