XML API - Loop through list to update dynamic URL HTTP GET

[DavidETL]

Hi All,

So far have had fantastic support from this community and the product is exceeding my expectations every day. With some help in an earlier post https://community.talend.com/t5/Design-and-Development/XML-API-to-CSV-Content-is-not-allowed-in-prol...i was able to succesfully build my first ETL job.

Now i am working on a second part of the solution which requires

1. Firstly get a list of valid upcoming dates and race types (https://tatts.com/pagedata/racing/racing.xml)
   • @MeetingDate, @MeetingCode
2.  Then from the list above it passes to the next HTTP request to loop through each date and meeting code
   • https://tatts.com/pagedata/racing/2017/7/15/BRFields.xml

So the values thet need to get passed from the first XML list are 

• @MeetingDate: "2017/7/15" in the URL (this can change, e.g 2017/7/16, etc what ever is in the list)
• @MeetingCode: "BR" in the URL (this can change, either BR, NR,TR,DR etc etc
• Both meeting date and code will always be unique on a specific day. there will never been the same date and code together on the same date.

At present i have a simple job build which basically calls the current date and rotates

tloop:

tjava:

HTTPRequest

Thanks All, any ideas would be much appreciated

ill continue working through it if i find anything ill post back

regards

david

[shong · Accepted solution]

Hi David
You need to use a tFlowToiterate to iterate each pair of MeetingDate and MeetingCode extracted from the first XML list, eg:
tHttpRequest1
|
onsubjobok
|
tFileInputXML--main(row1)--tFlowToIterate--iterate--tJava--oncomponentok--tHttpRequest2--oncomponentok--other components

on tHttpRequest: call the first API (https://tatts.com/pagedata/racing/racing.xml) and store the response to a XML file.

tFileInputXML: read the response XML file, and extract two fields: MeetingDate and MeetingCode

on tJava: build the dynamic URL for used later.
String stringDate=(TalendDate.formatDate("yyyy/MM/dd", row1.MeetingDate)).replaceAll("/0","/");
context.URL="https://tatts.com/pagedata/racing/"+stringDate+"/"+row1.MeetingCode+"Fields.xml";

Regards
Shong

[iburtally]

Sorry if I missed it, but I did not understand your question. What is the issue you are facing?

[shong · Accepted solution]

Hi David
You need to use a tFlowToiterate to iterate each pair of MeetingDate and MeetingCode extracted from the first XML list, eg:
tHttpRequest1
|
onsubjobok
|
tFileInputXML--main(row1)--tFlowToIterate--iterate--tJava--oncomponentok--tHttpRequest2--oncomponentok--other components

on tHttpRequest: call the first API (https://tatts.com/pagedata/racing/racing.xml) and store the response to a XML file.

tFileInputXML: read the response XML file, and extract two fields: MeetingDate and MeetingCode

on tJava: build the dynamic URL for used later.
String stringDate=(TalendDate.formatDate("yyyy/MM/dd", row1.MeetingDate)).replaceAll("/0","/");
context.URL="https://tatts.com/pagedata/racing/"+stringDate+"/"+row1.MeetingCode+"Fields.xml";

Regards
Shong

[DavidETL]

Hi Shong

I am getting this error in the JAVA

Error in the component's properties:URL cannot be resolved or is not a field
URL cannot be resolved or is not a field

Code:

String stringDate=(TalendDate.formatDate("yyyy/MM/dd",row1.MeetingDate)).replaceAll("/0","/");
context.URL="https://tatts.com/pagedata/racing/"+stringDate+"/"+row1.MeetingCode+"Fields.xml";

any ideas?