[resolved] Calling paginated REST API

One Star

[resolved] Calling paginated REST API

Hi,
I have 2 questions - 
1) First can we directly use the REST response (XML/JSON data) and parse its contents?
2) I have to call a REST api which has paginated response. So would need to call till all the data is exhausted. How can we achieve the same.
-Manoj

Accepted Solutions
Sixteen Stars

Re: [resolved] Calling paginated REST API

1) Yes you can. This is very simple with Talend.....simpler with XML than JSON though.
2) Paginated responses can be a bit fiddly, but entirely possible. I have actually written a tutorial about how to get Spotify listening stats from Facebook, that has to deal with a REST service making use of paginated results. My tutorial can be found here. It is probably more complicated than your job will need to be, but should give you a good idea of how to deal with getting all of the REST response pages. You can download the job at the bottom of the tutorial and if you have Facebook with a Spotify account linked, can make use of it.
Hope it helps

All Replies
Sixteen Stars

Re: [resolved] Calling paginated REST API

1) Yes you can. This is very simple with Talend.....simpler with XML than JSON though.
2) Paginated responses can be a bit fiddly, but entirely possible. I have actually written a tutorial about how to get Spotify listening stats from Facebook, that has to deal with a REST service making use of paginated results. My tutorial can be found here. It is probably more complicated than your job will need to be, but should give you a good idea of how to deal with getting all of the REST response pages. You can download the job at the bottom of the tutorial and if you have Facebook with a Spotify account linked, can make use of it.
Hope it helps
One Star

Re: [resolved] Calling paginated REST API

Thanks a lot. This helped.
Five Stars

Re: [resolved] Calling paginated REST API

Page is not open.
Five Stars

Re: [resolved] Calling paginated REST API

Is the tutorial still online somewhere? 

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.