Look up in big data, will it works or not?
i developed the big data job which lookup from red shift.
but its giving me duplicated in red shift. how to achieve ?
waiting for your reply..
Could you please refer the below link for your query?
Also could you please share the screenshot of your job flow and the tmap lookup in detail?
Please appreciate our members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)
batch job run in parallel and each worker make lookup independently
as a fast solution you could reduce the number of workers to 1
Big Data Spark Job:--
I am using tredshiftinput for lookup
problem is i am unable to do lookup from red shift data
its inserting the duplicates
any solution how to do this
Try Talend Cloud free for 30 days.
Introduction to Talend Open Studio for Data Integration.
Practical steps to developing your data integration strategy.
Create systems and workflow to manage clean data ingestion and data transformation.