I can't download and send a simple csv file from Dropbox to Postgresql (or any DB)

Five Stars

I can't download and send a simple csv file from Dropbox to Postgresql (or any DB)

I'm new to Talend and this problem seems to be extremely simple. I tried dozens of options and nothing seems to work. I would really appreciate help.

I have a file on Dropbox at the following URL:
https://www.dropbox.com/s/1lqyxvzv9p87u72/from_postgresql.csv?dl=1

 01 - Copy.pngDropbox file

When you change the last parameter to "dl=1" you can download the file directly without the interface (it's a good trick, I know).

I can dowload the file using the tRest component but instead I keep passing the body of the response on the "main" rows (row5, row7,row8 with 1 row transferred) instead of the actual rows from the csv file.

The RAW Response Body from the GET call looks like (tested on Postman):
ID,Name
1,Dropbox1
2,Dropbox2
3,Dropbox3

[..]
100,Dropbox100

The two subjobs:
02.pngTalend Job

The only way that I've found that works is presented on the second subjob (100 rows transfered) where I access the file saved on the first subjob by the tFileOutputRaw, but this doesn't make any sense to me. Do I need to really save the file and access on a separate subjob?

Can someone provide some guidance?

[Updates]

03.pngtRest config

 

 


Accepted Solutions
Eleven Stars

Re: I can't download and send a simple csv file from Dropbox to Postgresql (or any DB)

if you want you can use tCreateTemporaryFile to use a file automatically deleted after the task.

Francois Denis

Tag as "solved" for others! Kudos to thanks!


All Replies
Seven Stars

Re: I can't download and send a simple csv file from Dropbox to Postgresql (or any DB)

Can you post the Component configuration of the tRest_2 component.

 

Thanks

 

David

Regards

David

Don't forget to give Kudos when an answer is helpful or the solution.
Eleven Stars

Re: I can't download and send a simple csv file from Dropbox to Postgresql (or any DB)

the tRest did not guive you the content of your file you have to open it once it is donwloaded (onSubjodOk).

 

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Five Stars

Re: I can't download and send a simple csv file from Dropbox to Postgresql (or any DB)

Hey @david_beaty, Thanks for the help.
I updated the question with the tRest component configuration.

Five Stars

Re: I can't download and send a simple csv file from Dropbox to Postgresql (or any DB)

Thanks @fdenis
This is exactly what I'm doing on the second sub-job

It doesn't make sense to me to create a local file to temporarily hold the HTTP Body response, though.

I'm trying to find a way to directly send the body to the postgresql output component. However, if it's the way that works on Talend (two sub-jobs for a simple transfer), I think that my only option will be to change my mindset.

Eleven Stars

Re: I can't download and send a simple csv file from Dropbox to Postgresql (or any DB)

if you want you can use tCreateTemporaryFile to use a file automatically deleted after the task.

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Five Stars

Re: I can't download and send a simple csv file from Dropbox to Postgresql (or any DB)

Thanks @fdenis
Well, it's not the prettiest job that I've ever seen, but it works (deletes the temp file after the job execution).

 

tCreateTemporaryFile

https://help.talend.com/reader/WWQ40R_iTE5~~9VkUQrjgQ/LVyqOf5jpvzw3jzwz3wTJw

 

Use the global variables to access the temporary file and access the file on a sub-job and then send to PostgreSQL.

sdadasd01.png