Salesforce - populating relationship fields using external ids

Four Stars

Salesforce - populating relationship fields using external ids

Talend Version 4.2.3r67267
We are currently struggling with an Proxy/BulkExec issue, therefore I am exploring options & validating our thinking.
One of the benefits of the BulkAPI is it provides an output of all the successes/errors. The architecture of our solution is built around using the success log as a lookup for salesforce Ids when upserting contact records. This approach allows us to quickly load millions of records without the need to query salesforce for the related Account SalesforceIds as we use the salesforce id returned in the success log.
Question 1

I would like to validate my thinking. The salesforce Data Loader tool allows me to use external Ids to populate relationship fields in my case I would want to set the AccountId on the contact object using an external Id as part of the upsert of the contact record
It appears the Talend salesforce components do not support this - is this correct?
Question 2
How use case involves 10M+ record which takes arounds 4-6 hours to write out to CSV file which is then used as a lookup. I'm looking for most efficient approach to looking up salesforce Ids for large data sets - any pointers / experiences would be welcomed. For example filtering by date or running over the dataset one row at a time.

Thanks
One Star

Re: Salesforce - populating relationship fields using external ids

Hi
Q1: Recently a couple of users need external ids which seems to be not supported by salesforce components now. Please report it on BugTracker.
Q2: This issue about job optimization needs to be considered about the entire job. Which part is the bottle neck of this job? How to reduce the consuming of memory? Is it possible to run the job separately with multi threads? A screen shot of your job is preferred.
Regards,
Pedro
Four Stars

Re: Salesforce - populating relationship fields using external ids

Q1. Ok I will raise it on bugtracker
Q2. More background; our initial load is very large, but the daily update volume around 5000 rows. As we know the Account records which we have been upsert as part of the job we should be able to filter the query to salesforce for the account ids. Originally I wasn't thinking like this.
Our main issue is still the running on BulkExec component on unix with a Proxy, as this provides the salesforce ids to us via the output, hence no need to query salesforce for these ids.
Four Stars

Re: Salesforce - populating relationship fields using external ids

I was to checking the logs and noticed that "external ids" are not being returned in the bulkexec logs for the Contact object. They are for the Account object, so it's not across the board.
Q. Is it possible this is a bug for specific objects e.g. Contact
Q. Is this a bug in the Salesforce API itself (v19.0) or the Talend Salesforce component?
Thanks
One Star

Re: Salesforce - populating relationship fields using external ids

Hi
That's why I said above and want you to report it.
Recently a couple of users need external ids which seems to be not supported by salesforce components now. Please report it on BugTracker.

Regards,
Pedro
One Star

Re: Salesforce - populating relationship fields using external ids

Hello,
I am nw in this forum.
I am Salesforce certified.
I would like to know if this basic functionality is now working on Talend.
It seems it is availbale on 5.20M1 version, is it?
Currently, in order to upsert record and lookup relational field on salesforce, I need to the target obhect in salesforce in order to get the Salesforce Id, then match the external id with id salesforce (join) and finally upsert the target ogbje on salesforce.
So for example if I have to upsert Contact that have lookup to Account, i need to read all Account with talent, join Account with my external Account basign on external id in inrder to get Salesforce Account ID based and finally upsert contact lookup accoutn field basing the Account Id.
I would like to be able to use the basic feature that for example is available on SFDC Data Loader where I can upsert lookup filed on external id and not basign on salesforce id. This allow me to simplify the Job.
BR,
Enrico