How to get column name of csv file dyanamically

Highlighted
Four Stars

How to get column name of csv file dyanamically

Dear everybody,

 

I want to split an input csv file to multiple files by size and I have to get the header name which is defined on tFileInputDelimited component for creating partition files with header name and the flow is contained in the picture below:

 

Screenshot from 2019-09-10 10-56-28.png

I am using tJavaFlex to split files by size. Here is my code: (I have referenced: https://community.talend.com/t5/Design-and-Development/Split-source-based-on-byte-size/ta-p/35466)

 

// start part of your Java code
Integer iterator = 1;
Integer ByteCount = 0;
FileOutputStream fos = new
FileOutputStream(context.TgtFilePath+"TargetFile_"+iterator+".csv");


String tmpReadLine= row1._account_id_+","+ row1._user_id_+","+ row1._coupon_id_+","+ row1._redeemed_on_+","+ row1._status_+","+ row1._expiry_date_+","+ row1._added_on_+","+ row1._reason_+","+ row1._created_at_+","+ row1._updated_at_+"\n"; //Read input fields

byte[] contentInBytes = tmpReadLine.getBytes();//Convert them to Byte array

ByteCount=ByteCount+contentInBytes.length; // Summation of line bytes read

if ( ByteCount > context.fileSizeLimit * 1048576 ) {
    
    // Check if bytes read hasn't crossed the threshold
    ByteCount = contentInBytes.length;

    fos.flush();
    fos.close();
    iterator = iterator+ 1;

    // Threshold crossed write to new file
    
    fos = new FileOutputStream(context.TgtFilePath+"TargetFile_"+iterator+".csv");
    String headerFile = "account_id,user_id,coupon_id,redeemed_on,status,expiry_date,added_on,reason,created_at,updated_at\n";
	byte[] contentHeader = headerFile.getBytes();
	fos.write(contentHeader);

    fos.write(contentInBytes);
    
} else {
    fos.write(contentInBytes); // else write to same file
}

fos.close();

I want to get tmpReadLine dyanamically based on CSV file schema in my code. 

 

Who can help me for this case please!!!

Thank you guys!

 

Highlighted
Community Manager

Re: How to get column name of csv file dyanamically

I'm not entirely sure what you are trying to do. Are you trying to make this code....

String tmpReadLine= row1._account_id_+","+ row1._user_id_+","+ row1._coupon_id_+","+ row1._redeemed_on_+","+ row1._status_+","+ row1._expiry_date_+","+ row1._added_on_+","+ row1._reason_+","+ row1._created_at_+","+ row1._updated_at_+"\n"; //Read input fields

....more dynamic to take whatever the schema is that is passed into the component?

 

Highlighted
Four Stars

Re: How to get column name of csv file dyanamically

Yes, @rhall_2_0. I want to tmpReadLine get index of header name of CSV file. Example: CSV file has : A, B, C columns and tmpReadLine can get row1.A, row1.B, row1.C values dyanamically

Highlighted
Community Manager

Re: How to get column name of csv file dyanamically

You can get a String representation of the header and values using this code.....

System.out.println(row1.toString());

To get just the values in the order they are supplied you can use this....

System.out.println(row1.toLogString());

Obviously you would need to do some String manipulation, but it looks like your Java is good enough to figure that out :-)

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog