Fetch data from Rally API by traversing through its child records

Highlighted
Seven Stars

Fetch data from Rally API by traversing through its child records

Hi All,

 

I am working on Rally API to fetch JSON response. Now, I have to fetch all the User Stories under my project.
As there is no direct URL to fetch all User Stories at a time,I have to traverse the User stories through all its children.

Now, I need help in looping the Use Stories by traversing through its children and again repeat the same for other children under the parent User story.

 

Sample source data:

{
"FormattedID": "US1",
"Children": 
{
"_rallyAPIMajor": "2", 
"_rallyAPIMinor": "0", 
"_ref": "https://rally1.rallydev.com/slm/webservice/v2.0/HierarchicalRequirement/111111/Children", 
"_type": "HierarchicalRequirement", 
"Count": 0
}
}

{
"FormattedID": "US2",
"Children": 
{
"_rallyAPIMajor": "2", 
"_rallyAPIMinor": "0", 
"_ref": "https://rally1.rallydev.com/slm/webservice/v2.0/HierarchicalRequirement/22222/Children", 
"_type": "HierarchicalRequirement", 
"Count": 2
}
}


I have count field to verify whether a UserStory(US) have children or not.We need not traverse to its child if count is 0. Now, I have count =2 for US2, so I have to fetch reference Object URL (https://rally1.rallydev.com/slm/webservice/v2.0/HierarchicalRequirement/22222/Children) and fetch corresponding 2 child Usertories info(US4, US5), etc...

 

Assume below is the flow:

Capture.PNG

 

Expected O/P:
To fetch all user user stories under F1 (US1 to US9), F2, etc by traversing to its child and again back to parent.

Any suggestions would be of great help. Please let me know if more clarification is needed.

Seven Stars

Re: Fetch data from Rally API by traversing through its child records

@rhall_2_0: Your inputs would be very helpful for me

Community Manager

Re: Fetch data from Rally API by traversing through its child records

This is a tricky problem because you essentially need to use recursion....which you cannot carry out with a Talend Job. I would actually need to be able to have a play with this before I could guarantee a solution would work, but I can tell you what I would try.

 

I've used this technique when I have no idea how to stop a tLoop until I get to the known end or some data I am looking for. I set the tLoop to be a while loop with a boolean test. At the end of each iteration, I use a tJava to set the boolean (stored in the globalMap) to a value to stop the tLoop or allow it to continue.

 

Given your scenario here, I would use that method to keep looping until the end while at the same time creating a list of URLs I am yet to call. I would probably use a bit of Java here and maybe even my own class to store the data in an ArrayList or HashMap. Each time a URL is called, you would be able to update the related object in your code with the fact you had called that URL and the addition of further URLs (of children) to call. 

 

I don't think this will be easy, but it won't be impossible. As I said, this is one that would challenge me and I can only give you a hint as to how I would approach it given I can't try it.

Seven Stars

Re: Fetch data from Rally API by traversing through its child records

@rhall_2_0  Thank you very much for the detailed response. But I have no/minimal Java knowledge, so helpful if your suggestions include more of talend components than Java code.

Community Manager

Re: Fetch data from Rally API by traversing through its child records

I am not aware of any Talend components included or in the Exchange that will do this with without some coding. That's not to say there are none. Unfortunately this is quite a tricky problem. Every time I have done something similar in the past I have had to use code. It is REALLY beneficial in all aspects of Talend Studio to know a bit of Java. It will open all sorts of doors to you.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now