Input Select Columns

Four Stars

Input Select Columns

Hello, I'm a bit new to using Talend, but I'm getting the hang of it a little.


I'm creating a job to automatically import files to my sql system, but I'm having some problems with the CSV files.


Each file has over 500 columns, but I only need roughly the first 50 columns. However, when I use the tFileDelimitedImport with a schema of the first 50, the file throws an error and spits out a table with the proper headings, but no data. 

I've had success with going into the file and deleting everything to the right of where I've measured the schema, which suggests to me I need to create an import schema for the rest of the file.


However, I don't want to measure out the next 400 column names, even if it's just copy-pasting them, so I'd like to know if there's a way to only import the first X number of columns, or to just import a selected number of columns.


Thank you for your help.

Ten Stars

Re: Input Select Columns

The most easy way is extract complete linestring and split it to array and back to linestring and write to newoutfile

Recode below .. flexible mapping...

// Header ... on first
String[] headerArr = inputrow.completeLineFromFile.split(";");
// Main ... rows
String[] rowArr = inputrow.completeLineFromFile.split(";");
String[] outputString = {49}
for (int i=0; i<50; i++){
outputString[i]= rowArr[i].trim();
rowOutToNewFile.arrayLineCol = Arrays.toString(outputString);



Fifteen Stars TRF
Fifteen Stars

Re: Input Select Columns

What if your schema matches the first 50 columns and you untick the option "Check each row structure against schema"?


Cloud Free Trial

Try Talend Cloud free for 30 days.


Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.