Hi, I am developing an ETL job whereby a variety of different JSON schemas are converted to RDBMS. Each key in the JSON is mapped 1 to 1 to a column in a table. Each JSON schema has about 5 columns that are the same across all schemas, the rest are different. Right now, I am manually creating a JSON file in metadata for each schema, and manually mapping it to the corresponding table. However, in a few months, the amount of different JSON schemas received will increase exponentially, so manually creating a metadata file and mapping for each schema becomes unfeasible. Is there a way I can build a job that generically takes any JSON schema I throw at it, and automatically maps the value to a column in a table (where the name of a column = the JSON key), maybe using tSetDynamicSchema, or write/extract dynamic fields? Thanks, Nilan edit: Just to add, each different type of schema goes into a different table. The data from the JSON defines what the target table is, which is currently achieved by setting the relevant key/value from the JSON as a global variable, then using that variable as the table name in tMySQL input. edit2: I have also seen the page: https://help.talend.com/pages/viewpage.action?pageId=5671283 which shows how to deal with changing data structures. However the dynamic data type isn't supported by tFileInputJSON, so there may be a workaround for this?
Hi All, Elaborating the issue.. Source System : CouchDB - JSON format.. Target System : SQL Server.. There is another application running on top of the ETL system, Which generates the required views over the couchDB, as well as, creates the table to the SQL server and then triggers the ETL job. Kindly suggest some solution. Regards, Mitul