Error while loading data from csv to postgres using dynamic schema

Five Stars

Error while loading data from csv to postgres using dynamic schema

Hi,

    I have a requirement to load multiple csv files to different tables in a database. Since the CSVs are dynamic the schema also should be dynamic. I have a Talend licensed version and using dynamic schema. My work flow is like:

tFileList---->tFileInputFlowDelemited-->tMap-->tPostgresqlOutput

 

The schema i defined is 

id : String

others: dynamic

 

The error I am getting at the time of postgres insert is

[ERROR]: local_project.s3_loader_0_1.S3_Loader - tPostgresqlOutput_1 - Batch entry 0 INSERT INTO "public"."S3-Loader-Test" ("id", ) VALUES (576,Test,41:02.4,TRUE,, ) was aborted. Call getNextException to see the cause.
caused by: ERROR: syntax error at or near ")"

 

It is not considering and splitting dynamic field "others" while inserting.

Also if I define schem with only one dynamic column then it is considering the whole CSV row as single column and record and trying to insert. 

 

Could somebody correct me? or point me the right solution?


Accepted Solutions
Five Stars

Re: Error while loading data from csv to postgres using dynamic schema

Hi,

Thanks for the reply. I found the issue. The issue was related to the field separator. Once corrected it worked.


All Replies
Moderator

Re: Error while loading data from csv to postgres using dynamic schema

Hello,

For the purpose of testing, are you able to get all your output files by using tfileoutputdelimited component?

The work flow looks like:tfilelist-->tfileinputdelimited-->tfileoutputdelimited(Append option checked).

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error while loading data from csv to postgres using dynamic schema

Hi,

Thanks for the reply. I found the issue. The issue was related to the field separator. Once corrected it worked.

Moderator

Re: Error while loading data from csv to postgres using dynamic schema

Hello,

Great it is fixed. Thanks for posting your solution here.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.