I have a requirement to upload rows from a CSV file with a delay in between each row to allow batch processing to take place. I am running one row at a time as the logic behind the application is complex and, to stop 'too many SQL statements' exceptions, a batch size of 1 is to be used.
My question is' How can I setup Talend to do this?
I have a tFileInputDelimited linked to a tSleep then linked to a tSalesforceOutput component. I have set the attribute "Rows to Commit" to be 1 and so I have 2 transactions in Salesforce; one for each record in my test data. I am trying use the tSleep to put a 20 second pause between each call to Salesforce. The result is that the 2 records are created in seperate transactions but updated with no delay. What other components/configuration do I need?
Any help would be appreciated.
I had the batch setting to 1 and so was working correctly - I just needed to insert a tJavaRow component and add the line Thread.sleep(integer) to pause between each update.
So now I have a tFileInputDelimited linked to tJavaRow then to tSalesforceOuput. It all works as expected now.
Introduction to Talend Open Studio for Data Integration.
Practical steps to developing your data integration strategy.
Create systems and workflow to manage clean data ingestion and data transformation.