JSON Dynamic Schema processing

Four Stars

JSON Dynamic Schema processing

HI All,

Do we have any capability to process JSON schema dynamically? The input JSON file will be evolved with new elements over a period of time and Talend job needs to accommodate the schema changes with least code changes. any hints are appreciated. file needs to moved from one S3 bucket to another S3 bucket. 

Community Manager

Re: JSON Dynamic Schema processing

Do you need some transformation? Why don't you use tS3Copy to copy the file from one bucket to another bucket?
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: JSON Dynamic Schema processing

Hi Shong,

We need basic transformation. but our main requirement is to dynamically extracts JSON key/value pair with minimum changes in the job. so for e.g as of today we have to extract 50 elements from JSON...later it will change to 70..100...150..and so on...so that will need change in the Job to map additional/new JSON elements. 

 

Community Manager

Re: JSON Dynamic Schema processing

It's impossible, because we have to define the mapping between columns and the query elements when using tFileInputJson to extract data from an input json file.

----------------------------------------------------------
Talend | Data Agility for Modern Business

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog