Reading only xml file from un-archived folder through tfilelist

Four Stars

Reading only xml file from un-archived folder through tfilelist

Hi All ,
I am reading zip files from a folder unzipping that using tFileUnarchive. After Unzipping , i am reading only xml files using tfilelist after tFileUnarchive. I am iterating 2 files in parallel. But when i do so my whole process runs as many times as many are the files.
My workflow looks like this:
 tfilelist --> tfileunarchive --> tfilelist --> tfileinputxml --> tlogrow
i have 2 zip folders one contains 2 files (1 xml and 1jpeg) and other zip contains ( 1 xml ). I only want to process xml files after unzipping. When i iterate it with number of parallel execution as 2 it runs the workflow twice for 3 files.

Output that i am getting through this workflow is :
Process finished
273985234|ZO|1237351|JM|TestData|99
273985234|ZO|1237351|JM|TestData|99
273985234|ZO|1237351|JM|TestData|99
273985234|XO|1237161|SM|TestData|34
273985234|XO|1237161|SM|TestData|34
273985234|XO|1237161|SM|TestData|34

Expected Output:
273985234|ZO|1237351|JM|TestData|99
273985234|XO|1237161|SM|TestData|34

Please find attached screenshot of the workflow.

Thanks,
Saurabh.

Moderator

Re: Reading only xml file from un-archived folder through tfilelist

Hi,
A parallelization-enabled Iterate connection allows the component that receives threads from the connection to read those threads in parallel.
Have you set any file mask in tFileList_2?(*.xml)?


Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Reading only xml file from un-archived folder through tfilelist

Yes i have set the mask as "*.xml"
Moderator

Re: Reading only xml file from un-archived folder through tfilelist

Hi,

A parallelization-enabled Iterate connection allows the component that receives threads from the connection to read those threads in parallel.

Please disable option "Enable parallel exection" in iterate row to see if it works.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Reading only xml file from un-archived folder through tfilelist

No it still process the files twice.Only when i remove tfilelist_2 it process it once. But in that case it process even the image files that are their in unzipped folder and gives me error for those.
Four Stars

Re: Reading only xml file from un-archived folder through tfilelist

It only works when i introduce delete component , but even for that it works only with "Enable parallel exection" =1 
I am attaching the config details of tfilelist_2 and tfiledelete_1. Output that i am getting using the specified configurations is:

[font=Verdana, Helvetica, Arial, sans-serif]Starting job prcs_zip_chronological_order at 11:23 15/07/2015.[/font]


[font=Verdana, Helvetica, Arial, sans-serif][size=1][statistics] connecting to socket on port 3941[/size][/font]
[font=Verdana, Helvetica, Arial, sans-serif][size=1][statistics] connected[/size][/font]
[font=Verdana, Helvetica, Arial, sans-serif]Processing archive E:\Blogtestdata\abctest.zip, please wait...[/font]


[font=Verdana, Helvetica, Arial, sans-serif]Process finished[/font]
[font=Verdana, Helvetica, Arial, sans-serif]273985234|XO|1237161|SM|TestData|34[/font]
[font=Verdana, Helvetica, Arial, sans-serif]Processing archive E:\Blogtestdata\cdftest.zip, please wait...[/font]


[font=Verdana, Helvetica, Arial, sans-serif]Process finished[/font]
[font=Verdana, Helvetica, Arial, sans-serif]273985234|ZO|1237351|JM|TestData|99[/font]
[font=Verdana, Helvetica, Arial, sans-serif][size=1][statistics] disconnected[/size][/font]

[font=Verdana, Helvetica, Arial, sans-serif][size=1]Job prcs_zip_chronological_order ended at 11:23 15/07/2015. [exit code=0][/size][/font]
Thanks,
Saurabh.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now