I'm new with Talend and using Talend Open Studio 6.4 to build a job getting all the issues from JIRA API request to export into JSON file
I'm using tRestClient, GET method to call the API from JIRA, howevere, the results only return restricted by maxResults. So, if I have more than 200,000 records, and the maxResults is 1000, I can get only 1000 records from the API. The problem is I dont know how many records I have to set the maxResults, also I understand that the maxResults is not set to get all the records from the API
I did some investigation and came to know about Pagination. What I did is setting startAt, maxResults and total from API call as context variable, creating a new context variable is sequence to execute the API call in orders which will increase by 1 whenever the subjob is done, then update the context variable startAt equals to sequence*maxResults when the subjob is done and looping the subjob, with the condition in tLoop is the context variable startAt < context variable total, however, I was not successful.
Is there another way to accomplish this work. I attached the screen for what I've done here so you can see. It would be more beneficial if you can share the screenshot of the components used along with any java codes written for this purpose. Thanks in advance for your help.
There is a tJiraInput component which is available since version 6.2.1.
Introduction to Talend Open Studio for Data Integration.
Practical steps to developing your data integration strategy.
Create systems and workflow to manage clean data ingestion and data transformation.