cFile is picking up file even file is still being downloaded

Seven Stars

cFile is picking up file even file is still being downloaded

I have created 2 routes

 

Route 1 - download file from FTP using cFTP (cFTP-->cFile)

Route 2 - pick up file that was downloaded using cFTP & pass it on to DI job for further processing (cFile-->cTalendjob)

 

my issue is that for large file, cFile is even picking up the file that are still being downloaded from first route.

I have below setting on cFile since I want to be able to process same file again in case client post the same file again for any reason:

noop - true

"idempotentKey" : "${file:name}-${date:now:yyyyMMddHHmmssSSS}"

"delete" : "True"

 

 

Seven Stars

Re: cFile is picking up file even file is still being downloaded

Is it just me getting this issue or is it a bug?

 

I would really appreciate any response on this.

 

we are using enterprise version 6.4

Ten Stars

Re: cFile is picking up file even file is still being downloaded

Do you use OnComponentOk ? or OnSubjobOk? Could make a difference.
Suggest using OnSubjobOk and please post a screenshot of your jobFlow

Seven Stars

Re: cFile is picking up file even file is still being downloaded

these are ESB route components, it has only route option. not like DI that has all those options
Seven Stars

Re: cFile is picking up file even file is still being downloaded

these are ESB route components, it has only route option. not like DI that has all those options

i will try to post some scree shot tomorrow
Seven Stars

Re: cFile is picking up file even file is still being downloaded

here is some screenshot

 

route 1 - FTP pickup that pick up input file from FTP & copy that only shared location

route 2- pick up file from shared location & pass it on to DI job for further processing

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

An API-First Approach to Modernizing Applications

Learn how to use an API-First Approach to Modernize your Applications

Watch Now

Talend API Designer – Technical Overview

Take a look at this technical overview video of Talend API Designer

Watch Now

Getting Started with APIs

Find out how to get started with APIs

Read