Talend Connect
Virtual Summit
JOIN US!
And visit the Customer
& Community Lounge.
May 27-28, wherever you are.

tFileOutputDelimited and that LAST row separator

Highlighted
Four Stars

tFileOutputDelimited and that LAST row separator

Hello:
I am taking a 1-column data flow and outputting it to a file using tFileOutputDelimited to create JSON files. I'm using ",\n" as the Row Separator, and it works great.
Except for the last row, which always has that annoying last "," on the end. I haven't found any way to remove it either, aside from editing the file output and removing it manually.
Anyone have any ideas, either using Talend components or maybe some java?
Thanks...
Highlighted
Community Manager

Re: tFileOutputDelimited and that LAST row separator

Hi
You could read each row from the file using tFileInputFullRow, and then remove the last common "," if it has, output the whole row to a final file.
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Highlighted
Four Stars

Re: tFileOutputDelimited and that LAST row separator

Great idea - except: how would I know which row is the last one? See, all of the rows end in ","
Unless maybe there's a way to read in the entire file and remove all the "\n" characters so that the entire thing can be addressed in a tMap component, where the last character can be snipped off?
Or am I misunderstanding totally (the more likely case :-)
Highlighted
One Star

Re: tFileOutputDelimited and that LAST row separator

Hi,
Maybe you could use the lastIndexOf methode of the String object and then make a substring?
Highlighted
Four Stars

Re: tFileOutputDelimited and that LAST row separator

assuming that I get the file into one long string I think that would work. But some of these files can get to be very large. There must be a better way ...
Highlighted
Community Manager

Re: tFileOutputDelimited and that LAST row separator

Hi
It always read the whole content of file, the efficiency could be slow if the file is very large, I don't know the details about how do you generate the file, maybe you should consider the solution during the file generating...
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Highlighted
One Star

Re: tFileOutputDelimited and that LAST row separator

Hi,
Have you considered using tFileOutputJSON? Here's a simple 2-component blog post on producing a JSON file from a single data flow.
http://bekwam.blogspot.com/2011/01/text-file-to-json-with-talend-open.html
Highlighted
One Star

Re: tFileOutputDelimited and that LAST row separator

Assuming your file is like:
blablabla,
blablabla,
.....,
blablablaba,
You could first use a tfilerowcount to get the number of rows and save it in a context variable like context.rowcount. Then you use a a tjava as an output of this file. You tjava is like this:

String result="";
int nb=0;
InputStream ips=new FileInputStream("C:\\YOURFILE.csv");
InputStreamReader ipsr=new InputStreamReader(ips);
BufferedReader br=new BufferedReader(ipsr);
String ligne;
while ((ligne=br.readLine())!=null){
nb++;
if(nb==context.rowcount)
result=ligne.remplace(',','');
result+=ligne+"\n";
}
br.close();
globalMap.put("mailBody", mailBody);
Then you put the result string in a context variable and you use it to put it in a file. (I didn't try it but I think it could work).

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog