tSalesforceBulkExec Exeeded number of records : 10002

One Star

tSalesforceBulkExec Exeeded number of records : 10002

Does anyone have a recipes for dealing with large data sets and tSalesforceBulkExec?
When I try importing large files at the moment it throws an exception complaining about the number of records in the file exceeding 10001.
cheers,
Bruce
Employee

Re: tSalesforceBulkExec Exeeded number of records : 10002

Hi Bruce,
Please check the advanced settings of tSalesforceBulkExec, notice the "Rows to commit" field, be sure that the number of your data sets(row number) is less than this value, also the next one "Bytes to commit" is another limited field, bulkExec only can run under both of them. (we recommend that you could try to find detail help from RG firstly).
regard,
nsun
One Star

Re: tSalesforceBulkExec Exeeded number of records : 10002

Do you have any experience using the Talend tSalesforceBulkExec component – I was using this recently and it happily creates a bulk API job and delivers batches of the source data correctly.  In fact the job runs perfectly as long as I don't try to write output or reject rows from the component.  I'm uploading about 400K records in a serial bulk api job.
If I add outputs from the tSalesforceBulkExec component, when results start to come back from the job – I always get the error below.  I’m using the latest Talend Open Studio  - 6.2.0.  When I google I see comments that this was a problem but it was fixed in version 4.x – but doesn’t look like it from my experience.
Or if there is some other way to use the BulkAPI for Salesforce in talend – could you share how you setup the jobs – maybe you used a different component in Talend?
Starting job BulkLoaderBB at 11:52 08/07/2016.
connecting to socket on port 3956
connected
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
Exception in component tSalesforceBulkExec_1
com.sforce.async.CSVReader$CSVParseException: Exceeded number of records : 10002. Number of records should be less than or equal to 10001
      at com.sforce.async.CSVReader.checkRecordExceptions(CSVReader.java:159)
      at com.sforce.async.CSVReader.nextRecord(CSVReader.java:189)
      at org.talend.components.salesforce.runtime.SalesforceBulkRuntime.getBaseFileRow(SalesforceBulkRuntime.java:315)
      at org.talend.components.salesforce.runtime.SalesforceBulkRuntime.getBatchLog(SalesforceBulkRuntime.java:344)
      at org.talend.components.salesforce.runtime.SalesforceBulkExecReader.advance(SalesforceBulkExecReader.java:93)
      at local_project.bulkloaderbb_0_1.BulkLoaderBB.tSalesforceBulkExec_1Process(BulkLoaderBB.java:409)
      at local_project.bulkloaderbb_0_1.BulkLoaderBB.runJobInTOS(BulkLoaderBB.java:669)
      at local_project.bulkloaderbb_0_1.BulkLoaderBB.main(BulkLoaderBB.java:526)
disconnected
 
 
Moderator

Re: tSalesforceBulkExec Exeeded number of records : 10002

Hi malcolm.smith,
We have replied to your another topic:https://www.talendforge.org/forum/viewtopic.php?id=51732. Could you please take a look at it?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.