Four Stars

Input Select Columns

Hello, I'm a bit new to using Talend, but I'm getting the hang of it a little.

 

I'm creating a job to automatically import files to my sql system, but I'm having some problems with the CSV files.

 

Each file has over 500 columns, but I only need roughly the first 50 columns. However, when I use the tFileDelimitedImport with a schema of the first 50, the file throws an error and spits out a table with the proper headings, but no data. 

I've had success with going into the file and deleting everything to the right of where I've measured the schema, which suggests to me I need to create an import schema for the rest of the file.

 

However, I don't want to measure out the next 400 column names, even if it's just copy-pasting them, so I'd like to know if there's a way to only import the first X number of columns, or to just import a selected number of columns.

 

Thank you for your help.

2 REPLIES
Seven Stars

Re: Input Select Columns

The most easy way is extract complete linestring and split it to array and back to linestring and write to newoutfile


Recode below .. flexible mapping...

// Header ... on first
String[] headerArr = inputrow.completeLineFromFile.split(";");
// Main ... rows
String[] rowArr = inputrow.completeLineFromFile.split(";");
String[] outputString = {49}
for (int i=0; i<50; i++){
outputString[i]= rowArr[i].trim();
}
rowOutToNewFile.arrayLineCol = Arrays.toString(outputString);

 

 

Thirteen Stars TRF
Thirteen Stars

Re: Input Select Columns

What if your schema matches the first 50 columns and you untick the option "Check each row structure against schema"?

TRF