Difference between Sales Force OutPut Components

Seven Stars

Difference between Sales Force OutPut Components

Hi,

 

I am trying to learn SalesForce Integration. Can anyone please tell me difference between:-

 

  1. tSalesForceOutPut
  2. tSalesForceOutPutBulkExec

 

Regards

Tarunjit Singh


Accepted Solutions
Twelve Stars

Re: Difference between Sales Force OutPut Components

if the any one batch has issue then all the records wil be failed in BULK mode.

Manohar B

All Replies
Twelve Stars

Re: Difference between Sales Force OutPut Components

The Generic out put component will do inert/upsert/update/delete operation on specific SF table,even with Bulk one also similar but the Bulk one first it will cretae a Bulk file and then it will try to the operations based on the Bulk file.

 

https://help.talend.com/reader/iYcvdknuprDzYycT3WRU8w/DdNFQ29mYUxqVhLa8fOMQw

 

 

Manohar B
Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

 

But why would we want bulk file to be first created on server and then upload that bulk file content to SalesForce. Is there some specific advantage of creating this bulk file?

 

Regards

Tarunjit Singh

Forteen Stars TRF
Forteen Stars

Re: Difference between Sales Force OutPut Components

The main difference is that they do not use the Salesforce API - always consider the Bulk API as soon as you have a huge data volume or want to deal easilly with rejected records.

 


TRF
Twelve Stars

Re: Difference between Sales Force OutPut Components

in the normal out it will process record by record,where from Bulk file it will do Bulk operation,Bulk one would be faster.

Manohar B
Forteen Stars TRF
Forteen Stars

Re: Difference between Sales Force OutPut Components

tSalesforceOutput doesn't strickly work row by row, it has a "Commit Level" which fixe the transaction size.

As soon as 1 row is rejected due to a internal Salesforce reason, the whole transaction is lost.

With Bulk API, if a record is rejected the others are nore affected.


TRF
Twelve Stars

Re: Difference between Sales Force OutPut Components

based on the commit level it will insrt row by row. correct that where in batch ,if any bad records is there means entire batch willl reject.

Manohar B
Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

 

Did you mean if 5 records have to be inserted using tSalesForceOutPutBulkExec and out of these 5 records 1 record get some error then 4 records will gets inserted and 1 record will be rejected.

 

However if 5 records have to be inserted using tSalesForceOutPut and out of these 5 records 1 record get then all of the 5 records will get rejected.

 

Regards

Tarunjit Singh

Twelve Stars

Re: Difference between Sales Force OutPut Components

in Bulk all the records rejected if 1 record having error means,

in output,it will insert 4 records and 1 record will be rejected where in same scenario.

Manohar B
Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

 

I just tried it what you said however it is behaving in opposite manner.

 

When I tried to add 5 records in Sales Force using tSalesForceOutPut and 1 record have some error then no record gets added infact we got run time error as mentioned below. Also job didn't even completed its full flow that is flow doesn't enter in to Reject or Main flow.

Error Message

Exception in component tSalesforceOutput_1 (testSalesForceBulkJob)

java.io.IOException: Required fields are missing: [Name]

 

However when I tried to add 5 records in Sales Force using tSalesForceOutPutBulkExec and 1 record have some error then 4 records got added and 1 record not got added. More over there is no error when the job is run as is mentioned in tSalesForceOutPut  scenario. In this scenario job completed its flow completely that is flow enter successfully in to Reject or Main flow.

Forteen Stars TRF
Forteen Stars

Re: Difference between Sales Force OutPut Components

Untick the option "Cease on Error" if you din't want the job to stop in case of error.

You may also refer to this post where I tried to explain how it works.


TRF
Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

  

Thanks for the answer. Job is not now not breaking but no response from Sales Force is coming, i.e in main flow and reject flow it is showing 0 rows.

 

It seems that if we use tSalesForceOutPutBulkExec then if a record is rejected i.e. got failed while trying to add then others records are not effected. e.g. 1 out of 5 records are rejected then remaining 4 will get added in to Sales Force.

 

Where as if we use tSalesForceOutPut then if a record is rejected i.e. got failed while trying to add then others records will also be effected. e.g. 1 out of 5 records are rejected then remaining 4 will also not get added in to Sales Force and also there will also be no response from Sales Force regarding success and failure as mentioned in my first paragraph.

Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

 

I think I resolved the problem by setting Commit Level To1. Now tSalesForceOutPut is able to add 4 success records in to Sales Force and also I got response for failure.

 

But I think it will be slow as Commit Level is set to 1. As transaction will be created for each record. Please confirm. 

Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

 

Here is my understanding as per your comment.

 

  1. Suppose there are 50 records being added from tSalesForceOutPutBulkExec and few of the records got failed due to sales force internal error. In that scenario whole of the batch will get failed.
  2. Suppose there are 50 records being added from tSalesForceOutPut and few of the records got failed due to sales force internal error. In that scenario not whole of the batch will get failed only those of the records will get failed.

Please let me know if my understanding is correct.

 

 

Twelve Stars

Re: Difference between Sales Force OutPut Components

Yes correct..

Manohar B
Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

 

So in case of bulk addition using tSalesForceOutPutBulkExec if there are 100 records and batch size of 50 and first batch got failed due to Sales Force Internal issue then on response I will receive 50 records for success flow and 50 records for reject flow.

 

Please confirm.

Twelve Stars

Re: Difference between Sales Force OutPut Components

if the any one batch has issue then all the records wil be failed in BULK mode.

Manohar B
Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

 

It means if one batch fails then all the records of all the batches will have the same error message received from sales force.

Twelve Stars

Re: Difference between Sales Force OutPut Components

you will get below kind of error.

 

Exception in component tSalesforceOutputBulkExec_2_tSalesforceBulkExec
java.io.IOException: [AsyncApiException exceptionCode='InvalidBatch'
exceptionMessage='Records not processed'
]

at org.talend.components.salesforce.runtime.SalesforceBulkExecReader.start(SalesforceBulkExecReader.java:76)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tSalesforceInput_3Process(DataMig_OpportunityItem_Unify.java:21166)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tHashInput_10Process(DataMig_OpportunityItem_Unify.java:18922)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tHashInput_1Process(DataMig_OpportunityItem_Unify.java:14040)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tHashInput_18Process(DataMig_OpportunityItem_Unify.java:10458)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFileList_2Process(DataMig_OpportunityItem_Unify.java:4397)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tSalesforceConnection_3Process(DataMig_OpportunityItem_Unify.java:3942)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tSalesforceConnection_1Process(DataMig_OpportunityItem_Unify.java:3713)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tLoop_1Process(DataMig_OpportunityItem_Unify.java:3416)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_3Process(DataMig_OpportunityItem_Unify.java:2781)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFileList_1Process(DataMig_OpportunityItem_Unify.java:1923)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_15Process(DataMig_OpportunityItem_Unify.java:23526)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFTPGet_1Process(DataMig_OpportunityItem_Unify.java:23358)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFTPConnection_1Process(DataMig_OpportunityItem_Unify.java:23019)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_12Process(DataMig_OpportunityItem_Unify.java:22781)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_9Process(DataMig_OpportunityItem_Unify.java:22613)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFixedFlowInput_2Process(DataMig_OpportunityItem_Unify.java:24047)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_10Process(DataMig_OpportunityItem_Unify.java:24299)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tSetProxy_1Process(DataMig_OpportunityItem_Unify.java:37203)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.runJobInTOS(DataMig_OpportunityItem_Unify.java:37606)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.main(DataMig_OpportunityItem_Unify.java:37298)
Caused by: [AsyncApiException exceptionCode='InvalidBatch'
exceptionMessage='Records not processed'

Manohar B
Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

 

From the error message it seems that in such case when a batch will fail then the complete job will get fail. That is flow will not move forward from tSalesForOutPutBulkExec. Please confirm.

Twelve Stars

Re: Difference between Sales Force OutPut Components

Yes,it will not move forward.

Manohar B
Forteen Stars TRF
Forteen Stars

Re: Difference between Sales Force OutPut Components

In this case, the job failed because the job failed on salesforce side.

The reason may be because you try to touch an unknown field or object.

So it is normal that the job failed.

If you have a valid batch with rejected records, the job will not fail and the job will continue.


TRF
Seven Stars

Re: Difference between Sales Force OutPut Components

Hi,

 

Responding to your line

"If you have a valid batch with rejected records, the job will not fail and the job will continue."

 

This error will come when some of the records sent have some issue, e.g. required fields missing.

 

But in case some records of a batch gets failed due to Sales Force Internal Reason in that scenario not only that batch but all the batches of that job cycle will get failed. And in that scenario either the job will get failed or we will get error response for all records of all batches in "Reject" Flow.

 

Please confirm if I am wrong or right?

Six Stars

Re: Difference between Sales Force OutPut Components

There appears to be an issue between the actual output rows from a tSalesforce component based on the commit level.  See this bug: https://jira.talendforge.org/browse/TDI-40145 for details.