I have a requirement to load multiple csv files to different tables in a database. Since the CSVs are dynamic the schema also should be dynamic. I have a Talend licensed version and using dynamic schema. My work flow is like:
The schema i defined is
id : String
The error I am getting at the time of postgres insert is
[ERROR]: local_project.s3_loader_0_1.S3_Loader - tPostgresqlOutput_1 - Batch entry 0 INSERT INTO "public"."S3-Loader-Test" ("id", ) VALUES (576,Test,41:02.4,TRUE,, ) was aborted. Call getNextException to see the cause.
caused by: ERROR: syntax error at or near ")"
It is not considering and splitting dynamic field "others" while inserting.
Also if I define schem with only one dynamic column then it is considering the whole CSV row as single column and record and trying to insert.
Could somebody correct me? or point me the right solution?
Solved! Go to Solution.
For the purpose of testing, are you able to get all your output files by using tfileoutputdelimited component?
The work flow looks like:tfilelist-->tfileinputdelimited-->tfileoutputdelimited(Append option checked).
Great it is fixed. Thanks for posting your solution here.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Pick up some tips and tricks with Context Variables
Learn how media organizations have achieved success with Data Integration
Accelerate your data lake projects with an agile approach