Record counting

Six Stars

Record counting

Hi!

 

I have a file of many lines, where the last column of the first row is equivalent to a number that is the total of the following lines, that is, if the number of that column is 1250, it means that there are 1250 lines after it.
How do you this in Talend?

 

Thanks! 


Accepted Solutions
Employee

Re: Record counting

Hi,

 

    We can do multiple ways in Talend.

 

    For example, you can send a copy of your data to taggregaterow with sum function to calculate the total rows. Or if the source is a DB, you can take the count from DB itself and store to a context variable. 

 

   Either way, once the total count is achieved, you can send them to a tfileoutputdelimited component with only header avlues and total count in tMap.

 

    Then you need to read the actual data and send them also to the same file but this time, you need to mark the append mode as selected (so that header data will be there).

 

    Since you have not shared the job flow screenshots or the sample data file, I will not be able to elaborate further. But these instructions should be sufficient for you to fix the problem. If you are stuck, please feel free to come back with above details and we are always here to help you :-)

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)


Warm Regards,
Nikhil Thampi
Please appreciate our members by giving Kudos for spending their time for your query. If your query is answered, please mark the topic as resolved :-)

All Replies
Employee

Re: Record counting

Hi,

 

    We can do multiple ways in Talend.

 

    For example, you can send a copy of your data to taggregaterow with sum function to calculate the total rows. Or if the source is a DB, you can take the count from DB itself and store to a context variable. 

 

   Either way, once the total count is achieved, you can send them to a tfileoutputdelimited component with only header avlues and total count in tMap.

 

    Then you need to read the actual data and send them also to the same file but this time, you need to mark the append mode as selected (so that header data will be there).

 

    Since you have not shared the job flow screenshots or the sample data file, I will not be able to elaborate further. But these instructions should be sufficient for you to fix the problem. If you are stuck, please feel free to come back with above details and we are always here to help you :-)

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)


Warm Regards,
Nikhil Thampi
Please appreciate our members by giving Kudos for spending their time for your query. If your query is answered, please mark the topic as resolved :-)

Cloud Free Trial

Try Talend Cloud free for 30 days.

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.