I have a requirement to load multiple csv files to different tables in a database. Since the CSVs are dynamic the schema also should be dynamic. I have a Talend licensed version and using dynamic schema. My work flow is like:
The schema i defined is
id : String
The error I am getting at the time of postgres insert is
[ERROR]: local_project.s3_loader_0_1.S3_Loader - tPostgresqlOutput_1 - Batch entry 0 INSERT INTO "public"."S3-Loader-Test" ("id", ) VALUES (576,Test,41:02.4,TRUE,, ) was aborted. Call getNextException to see the cause.
caused by: ERROR: syntax error at or near ")"
It is not considering and splitting dynamic field "others" while inserting.
Also if I define schem with only one dynamic column then it is considering the whole CSV row as single column and record and trying to insert.
Could somebody correct me? or point me the right solution?
Solved! Go to Solution.
For the purpose of testing, are you able to get all your output files by using tfileoutputdelimited component?
The work flow looks like:tfilelist-->tfileinputdelimited-->tfileoutputdelimited(Append option checked).
Great it is fixed. Thanks for posting your solution here.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema