I'm new to Talend and this problem seems to be extremely simple. I tried dozens of options and nothing seems to work. I would really appreciate help.
I have a file on Dropbox at the following URL:
When you change the last parameter to "dl=1" you can download the file directly without the interface (it's a good trick, I know).
I can dowload the file using the tRest component but instead I keep passing the body of the response on the "main" rows (row5, row7,row8 with 1 row transferred) instead of the actual rows from the csv file.
The RAW Response Body from the GET call looks like (tested on Postman):
The two subjobs:
The only way that I've found that works is presented on the second subjob (100 rows transfered) where I access the file saved on the first subjob by the tFileOutputRaw, but this doesn't make any sense to me. Do I need to really save the file and access on a separate subjob?
Can someone provide some guidance?
Solved! Go to Solution.
Can you post the Component configuration of the tRest_2 component.
the tRest did not guive you the content of your file you have to open it once it is donwloaded (onSubjodOk).
Tag as "solved" for others! Kudos to thanks!
Hey @david_beaty, Thanks for the help.
I updated the question with the tRest component configuration.
This is exactly what I'm doing on the second sub-job
It doesn't make sense to me to create a local file to temporarily hold the HTTP Body response, though.
I'm trying to find a way to directly send the body to the postgresql output component. However, if it's the way that works on Talend (two sub-jobs for a simple transfer), I think that my only option will be to change my mindset.
Well, it's not the prettiest job that I've ever seen, but it works (deletes the temp file after the job execution).
Use the global variables to access the temporary file and access the file on a sub-job and then send to PostgreSQL.
Join us at the Community Lounge.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Pick up some tips and tricks with Context Variables
Learn how media organizations have achieved success with Data Integration
Create systems and workflow to manage clean data ingestion and data transformation.