Look up in big data, will it works or not?
i developed the big data job which lookup from red shift.
but its giving me duplicated in red shift. how to achieve ?
waiting for your reply..
Could you please refer the below link for your query?
Also could you please share the screenshot of your job flow and the tmap lookup in detail?
Please appreciate our members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)
batch job run in parallel and each worker make lookup independently
as a fast solution you could reduce the number of workers to 1
Big Data Spark Job:--
I am using tredshiftinput for lookup
problem is i am unable to do lookup from red shift data
its inserting the duplicates
any solution how to do this
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Pick up some tips and tricks with Context Variables
Take a look at this video about Talend Integration with Databricks
Learn how<SPAN>to modernize your Cloud Platform for Big Data Analytics with Talend and Microsoft Azure</SPAN>