Spooling n-Records before Executing a Post Using tRest

Highlighted
Five Stars

Spooling n-Records before Executing a Post Using tRest

TLDR: I need to spool 100 records from a larger file and then execute a post against my API using tRest.

 

I have been trying to figure out how I would execute this scenario:

     Currently I am posting 1 record as a Post to my tRest component using an iterate connector, the service returns the response and I continue processing.

What I need to do is post as many as 100 records to the API in a chunked fashion from the source file (250K lines), I need to spool up to 100 records then post them. 

 

Current Flow

File Input (500K Lines) --> tMap -->Iterate --->tRest (post single) --> tHashOutPut (to Store response)

 

Future Flow

File Input (500K Lines) --> tMap -->Spool-100-Lines --->tRest (post array of 100) --> tHashOutPut (to Store response)

 

 

 

 

 

Carolus Holman
Tags (1)

What’s New for Talend Spring ’19

Join us live for a sneak peek!

Sign up now

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch

Downloads and Trials

Test drive Talend's enterprise products.

Downloads

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Download