tSQSInput only reads 3 messages when value is set to 10

Five Stars

tSQSInput only reads 3 messages when value is set to 10

Talend Big Data 6.4.1

 

I'm using the tSQSInput component to retrieve messages from an SQS queue.

Screen Shot 2019-01-24 at 10.31.24 AM.png

I iterate and perform actions on the data until the queue is empty. Although, with the "Max number of message to return per request" set to 10, I only ever see a maximum of 3 being processed (and in flight on AWS console). Is this expected behavior from this component?

 

Thanks for any assistance.

Highlighted
Forteen Stars

Re: tSQSInput only reads 3 messages when value is set to 10

Hi,

 

try to increase custom wait time

-----------
Highlighted
Five Stars

Re: tSQSInput only reads 3 messages when value is set to 10

@vapukov  I tried increasing custom wait time to the max of 20 and the component still only reads 3 messages at a time.

In the image below, parallel iteration is set to 10 but there are only ever 3 executions running.

Screen Shot 2019-01-25 at 9.46.32 AM.png

Screen Shot 2019-01-25 at 9.49.36 AM.png

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now