File processing using Dynamic Schema

Highlighted
Five Stars

File processing using Dynamic Schema

Currently using Talend Data Integration Version: 6.4.1 Build id: 20170623_1246 (aka Enterprise)

 

Problem:

We are receiving clinical text csv files with differing numbers of columns in the files.

There is a core set of columns we want to retrieve from all of the files.  There is also 

a set of columns that will be listed multiple times which we want to concatenate

together and continue to pass along in the flow.

 

So how to I retrieve the "dynamic schema" column information and the value of the column coming from a  component?

 

Thank you

Peter

Moderator

Re: File processing using Dynamic Schema

Hello,


@pdiazdeleon wrote:

There is a core set of columns we want to retrieve from all of the files.  There is also 

a set of columns that will be listed multiple times which we want to concatenate

together and continue to pass along in the flow.

 

 

 



With your job requirement, could you please elaborate your case with an example with input and expected output values?

Here is an article that introduces the dynamic schema feature:https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/2...

Hope it helps.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download