I have a requirement to load multiple csv files to different tables in a database. Since the CSVs are dynamic the schema also should be dynamic. I have a Talend licensed version and using dynamic schema. My work flow is like:
The schema i defined is
id : String
The error I am getting at the time of postgres insert is
[ERROR]: local_project.s3_loader_0_1.S3_Loader - tPostgresqlOutput_1 - Batch entry 0 INSERT INTO "public"."S3-Loader-Test" ("id", ) VALUES (576,Test,41:02.4,TRUE,, ) was aborted. Call getNextException to see the cause.
caused by: ERROR: syntax error at or near ")"
It is not considering and splitting dynamic field "others" while inserting.
Also if I define schem with only one dynamic column then it is considering the whole CSV row as single column and record and trying to insert.
Could somebody correct me? or point me the right solution?
Solved! Go to Solution.
For the purpose of testing, are you able to get all your output files by using tfileoutputdelimited component?
The work flow looks like:tfilelist-->tfileinputdelimited-->tfileoutputdelimited(Append option checked).
Great it is fixed. Thanks for posting your solution here.
Watch the recorded webinar!
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.
Test drive Talend's enterprise products.