Loading files with different schema

Six Stars

Loading files with different schema

Hi Team,

I have multiple file with different structure.

Is there a way you create one dynamic schema and load all files on the fly.

If not, can it be done using Spark? If yes, how do you do it?

Highlighted
Moderator

Re: Loading files with different schema

Hello,

With talend subscription solution, here is a dynamic schema feature which allows you to design schema with an unknown column structure (unknown name and number of columns).

Where are your input files from?

Here is a community article:https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/2...

Let us know if it is what you are looking for.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Six Stars

Re: Loading files with different schema

Thanks for the reply! Is it possible to load multiple files to different targets?
Example only shows loading one file at a time to a db, and not the use case i'm looking at.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog