how works parallelization

Seven Stars

how works parallelization

Hello, 

 

I wanted to check, if I want to develop a job which enriched with parallelization. Suppose, job contains, sort, tMap used for join 1,2 files/table, other component.  So I wanted to understand, how this will work?

what is best way to build the job, so performance will increase and in addition, output with and without parallel is same.

Forteen Stars

Re: how works parallelization

@mailforsaggy,to improve the performence,i will suggest you to store on disk option in tMap(Basic settings) and tSortRow.(Advanced Settings)

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Tags (1)
Seven Stars

Re: how works parallelization

@manodwhb, I can use store on disc option when data is huge or not fit in to the memory.

 

My question is different. My question is about implementation of parallelization. and How it work when we have component sort, tMap and etc in the job. what is run time behavior when job set to run in parallel consisting of these component.

Tags (1)
Forteen Stars

Re: how works parallelization

@mailforsaggy,did you checked below link?

https://help.talend.com/reader/TKUQ4WRBbYZRnl9OyAgr5w/cSnwqkJCdsct_heLy3lrAQ

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Tags (1)
Seven Stars

Re: how works parallelization

@manodwhd, I know how to enable the parallelization, but I wanted to understand how it behave at run time. likeif I join 3-4 files to tmap, then how it will join with main flow which is parallel and dedup component or with other components present in job.
Tags (1)

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog