In Big data Job, look up Does it work?

Six Stars

In Big data Job, look up Does it work?

Hello everyone,

 

Look up in big data, will it works or not?

 

i developed the big data job which lookup from red shift. 

but its giving me duplicated in red shift. how to achieve ?

 

waiting for your reply..

Employee

Re: In Big data Job, look up Does it work?

Hi,

 

   Could you please refer the below link for your query?

 

https://help.talend.com/reader/j~SccKoZW8kEeYb0teesxw/f7739LCAGS8P4BiFbQCLzw

 

  Also could you please share the screenshot of your job flow and the tmap lookup in detail?

 

Warm Regards,
Nikhil Thampi

Please appreciate our members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

Highlighted
Forteen Stars

Re: In Big data Job, look up Does it work?

HI Mahesh

 

batch job run in parallel and each worker make lookup independently
as a fast solution you could reduce the number of workers to 1

 

 

 

 

 

-----------
Six Stars

Re: In Big data Job, look up Does it work?

@vapukov could you explain me clearly how to do this

Six Stars

Re: In Big data Job, look up Does it work?

Big Data Spark Job:--

I am using tredshiftinput for lookup 

problem is i am unable to do lookup from red shift data 

its inserting the duplicates 

 

any solution how to do this 

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now

How to Modernize Your Cloud Platform for Big Data Analytics With Talend and Micr...

Learn how<SPAN>to modernize your Cloud Platform for Big Data Analytics with Talend and Microsoft Azure</SPAN>

Read