File processing using Dynamic Schema

Highlighted
Five Stars

File processing using Dynamic Schema

Currently using Talend Data Integration Version: 6.4.1 Build id: 20170623_1246 (aka Enterprise)

 

Problem:

We are receiving clinical text csv files with differing numbers of columns in the files.

There is a core set of columns we want to retrieve from all of the files.  There is also 

a set of columns that will be listed multiple times which we want to concatenate

together and continue to pass along in the flow.

 

So how to I retrieve the "dynamic schema" column information and the value of the column coming from a  component?

 

Thank you

Peter

Moderator

Re: File processing using Dynamic Schema

Hello,


@pdiazdeleon wrote:

There is a core set of columns we want to retrieve from all of the files.  There is also 

a set of columns that will be listed multiple times which we want to concatenate

together and continue to pass along in the flow.

 

 

 



With your job requirement, could you please elaborate your case with an example with input and expected output values?

Here is an article that introduces the dynamic schema feature:https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/2...

Hope it helps.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch

Downloads and Trials

Test drive Talend's enterprise products.

Downloads