Hey guys, this is my first post and I am new to ETL and Talend. I've created a new job that makes a .txt file from Mixpanel's API, cleans it up and then inserts the data into our DataWarehouse. This job is working flawlessly, however, it is a manual process that I now need to automate. Each day I have to go and update the URI on my tHttpRequest Component with yesterday's date to then retrieve the latest full day of data. The URI looks like this: "https://data.mixpanel.com/api/2.0/export/?from_date=2017-08-21&to_date=2017-08-21" (and yes the dates are inclusive). Now, I've looked at Context Variables and other ways to attempt this but none of it is making sense to me. Is there a way that I can have the from_date and to_date update with the previous day every day automatically? If so, the steps to accomplish this would be incredibly helpful. I've attached a screenshot of the job if it helps any of you visual people out there. Thank you!
Solved! Go to Solution.
Thanks for your input, this put me on the path that led to a resolution. Here is what I did:
This helped me iterate through a date value in a tHttpRequest Component's URI perfectly. Thanks again!
Can you please share your configuration for thttprequest .
I am having trouble setting up the connection . This is what i used -
Method - POST
checked Need authentication and provided user - **************
getting error as - 411 Length Required
Have also tried Content-Length:0 in header but ran into error.
Can you please share your configuration for thttprequest.