Record counting

Highlighted
Six Stars

Record counting

Hi!

 

I have a file of many lines, where the last column of the first row is equivalent to a number that is the total of the following lines, that is, if the number of that column is 1250, it means that there are 1250 lines after it.
How do you this in Talend?

 

Thanks! 


Accepted Solutions
Employee

Re: Record counting

Hi,

 

    We can do multiple ways in Talend.

 

    For example, you can send a copy of your data to taggregaterow with sum function to calculate the total rows. Or if the source is a DB, you can take the count from DB itself and store to a context variable. 

 

   Either way, once the total count is achieved, you can send them to a tfileoutputdelimited component with only header avlues and total count in tMap.

 

    Then you need to read the actual data and send them also to the same file but this time, you need to mark the append mode as selected (so that header data will be there).

 

    Since you have not shared the job flow screenshots or the sample data file, I will not be able to elaborate further. But these instructions should be sufficient for you to fix the problem. If you are stuck, please feel free to come back with above details and we are always here to help you :-)

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)


All Replies
Employee

Re: Record counting

Hi,

 

    We can do multiple ways in Talend.

 

    For example, you can send a copy of your data to taggregaterow with sum function to calculate the total rows. Or if the source is a DB, you can take the count from DB itself and store to a context variable. 

 

   Either way, once the total count is achieved, you can send them to a tfileoutputdelimited component with only header avlues and total count in tMap.

 

    Then you need to read the actual data and send them also to the same file but this time, you need to mark the append mode as selected (so that header data will be there).

 

    Since you have not shared the job flow screenshots or the sample data file, I will not be able to elaborate further. But these instructions should be sufficient for you to fix the problem. If you are stuck, please feel free to come back with above details and we are always here to help you :-)

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Downloads and Trials

Test drive Talend's enterprise products.

Downloads