Data validation - Empty file Handling

Seven Stars

Data validation - Empty file Handling

Hello,

 

I want to know how to implement some validation before processing a file.

 

In fact, i have a file (csv) have contain header and i want to check if the file is empty(excluding the header) then process to tmap else die.

 

Can you advice please?

 

Thank you.

 

Best regards,

asadasing

 

 

 

 

 

 


Accepted Solutions
Eight Stars

Re: Data validation - Empty file Handling

@asadasing i think you can use tFileRowCount component.

All Replies
Moderator

Re: Data validation - Empty file Handling

Hello,

In Talend, you can use tFileProperties to get the file properties, there is a field called size that populate the file size, if it is 0, means the file is empty. According to your description, a job design for example looks like:
tFileFlist--iterate--tFileProperties--tJavaRow--runIf_1-->normal processing flow
                                                                  --runIf_2--->another processing if the file is empty
on tJavaRow, check the file if it is empty and put the result to a global variable, for example:

if(input_row.size==0){ globalMap.put("isEmpty", true); }else{ globalMap.put("isEmpty", false); }

set the condition of runIf_1 as:
!(Boolean)globalMap.get("isEmpty")

set the condition of runIf_2 as:
(Boolean)globalMap.get("isEmpty")

Let us know if it is OK with you.

Best regards

Sabrina

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seven Stars

Re: Data validation - Empty file Handling

Hello,

Thank you for your response, in tfileproperties, it does the size calculation in bytes.
Hence, i cannot check if file = 0 ; I have tried to check the flow with an empty file and it returns 2 and now my empty contain header, i guess the value will change.

Can you advice please on how i can proceed?
Eight Stars

Re: Data validation - Empty file Handling

@asadasing i think you can use tFileRowCount component.

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch

Downloads and Trials

Test drive Talend's enterprise products.

Downloads