I am trying to learn SalesForce Integration. Can anyone please tell me difference between:-
Solved! Go to Solution.
The Generic out put component will do inert/upsert/update/delete operation on specific SF table,even with Bulk one also similar but the Bulk one first it will cretae a Bulk file and then it will try to the operations based on the Bulk file.
But why would we want bulk file to be first created on server and then upload that bulk file content to SalesForce. Is there some specific advantage of creating this bulk file?
The main difference is that they do not use the Salesforce API - always consider the Bulk API as soon as you have a huge data volume or want to deal easilly with rejected records.
in the normal out it will process record by record,where from Bulk file it will do Bulk operation,Bulk one would be faster.
tSalesforceOutput doesn't strickly work row by row, it has a "Commit Level" which fixe the transaction size.
As soon as 1 row is rejected due to a internal Salesforce reason, the whole transaction is lost.
With Bulk API, if a record is rejected the others are nore affected.
based on the commit level it will insrt row by row. correct that where in batch ,if any bad records is there means entire batch willl reject.
Did you mean if 5 records have to be inserted using tSalesForceOutPutBulkExec and out of these 5 records 1 record get some error then 4 records will gets inserted and 1 record will be rejected.
However if 5 records have to be inserted using tSalesForceOutPut and out of these 5 records 1 record get then all of the 5 records will get rejected.
in Bulk all the records rejected if 1 record having error means,
in output,it will insert 4 records and 1 record will be rejected where in same scenario.
I just tried it what you said however it is behaving in opposite manner.
When I tried to add 5 records in Sales Force using tSalesForceOutPut and 1 record have some error then no record gets added infact we got run time error as mentioned below. Also job didn't even completed its full flow that is flow doesn't enter in to Reject or Main flow.
Exception in component tSalesforceOutput_1 (testSalesForceBulkJob)
java.io.IOException: Required fields are missing: [Name]
However when I tried to add 5 records in Sales Force using tSalesForceOutPutBulkExec and 1 record have some error then 4 records got added and 1 record not got added. More over there is no error when the job is run as is mentioned in tSalesForceOutPut scenario. In this scenario job completed its flow completely that is flow enter successfully in to Reject or Main flow.
Untick the option "Cease on Error" if you din't want the job to stop in case of error.
You may also refer to this post where I tried to explain how it works.
Thanks for the answer. Job is not now not breaking but no response from Sales Force is coming, i.e in main flow and reject flow it is showing 0 rows.
It seems that if we use tSalesForceOutPutBulkExec then if a record is rejected i.e. got failed while trying to add then others records are not effected. e.g. 1 out of 5 records are rejected then remaining 4 will get added in to Sales Force.
Where as if we use tSalesForceOutPut then if a record is rejected i.e. got failed while trying to add then others records will also be effected. e.g. 1 out of 5 records are rejected then remaining 4 will also not get added in to Sales Force and also there will also be no response from Sales Force regarding success and failure as mentioned in my first paragraph.
I think I resolved the problem by setting Commit Level To1. Now tSalesForceOutPut is able to add 4 success records in to Sales Force and also I got response for failure.
But I think it will be slow as Commit Level is set to 1. As transaction will be created for each record. Please confirm.
Here is my understanding as per your comment.
Please let me know if my understanding is correct.
So in case of bulk addition using tSalesForceOutPutBulkExec if there are 100 records and batch size of 50 and first batch got failed due to Sales Force Internal issue then on response I will receive 50 records for success flow and 50 records for reject flow.
It means if one batch fails then all the records of all the batches will have the same error message received from sales force.
you will get below kind of error.
Exception in component tSalesforceOutputBulkExec_2_tSalesforceBulkExec
java.io.IOException: [AsyncApiException exceptionCode='InvalidBatch'
exceptionMessage='Records not processed'
Caused by: [AsyncApiException exceptionCode='InvalidBatch'
exceptionMessage='Records not processed'
From the error message it seems that in such case when a batch will fail then the complete job will get fail. That is flow will not move forward from tSalesForOutPutBulkExec. Please confirm.
In this case, the job failed because the job failed on salesforce side.
The reason may be because you try to touch an unknown field or object.
So it is normal that the job failed.
If you have a valid batch with rejected records, the job will not fail and the job will continue.
Responding to your line
"If you have a valid batch with rejected records, the job will not fail and the job will continue."
This error will come when some of the records sent have some issue, e.g. required fields missing.
But in case some records of a batch gets failed due to Sales Force Internal Reason in that scenario not only that batch but all the batches of that job cycle will get failed. And in that scenario either the job will get failed or we will get error response for all records of all batches in "Reject" Flow.
Please confirm if I am wrong or right?
There appears to be an issue between the actual output rows from a tSalesforce component based on the commit level. See this bug: https://jira.talendforge.org/browse/TDI-40145 for details.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema