Salesforce loading related objects with single output

Five Stars

Salesforce loading related objects with single output

I have talend job that works as designed, except that I need to create a parent-child relationship between the two custom objects and to do so i need the first custom object to load before the second custom object.

 

I have read a bunch of documentation, regarding the tFixedInputFlow, but because I need the output of the tmap for both tsalesforceoutput components I don't believe this will work.  I have also tried to use tSleep to pause the second/related custom object, so the first custom object can load completely and this did not work either.  

 

What am I missing?

 

 

I have attached a simple representation of the real job, and all I want to do is load the patient output completely, before the patient_encounter output.

 

Your help is greatly appreciated.

 


Accepted Solutions
Highlighted
Fifteen Stars TRF
Fifteen Stars

Re: Salesforce loading related objects with single output

Remove the tSleep and replace the Add_EhrEncounters by a tHashOutput.
Add a tHashInput connected to the tMSSQLInput by an onSubjobOk trigger.
Connect the Add_EhrEncounters to this tHashInput. Don't forget to refer the tHashOutput from the tHashInput and to to set the schema.

TRF

All Replies
Five Stars

Re: Salesforce loading related objects with single output

It's always the last thing you try that works.  Smiley Wink

 

I got it working by using the tSleep and most importantly I unchecked the "Extend Insert" on the Patient tsalesforceoutput.  I believe by unchecking the "Extend Insert" each patient record is committed one by one, thus allowing the Patient_Encounter to find the related patient record on insert.

Fifteen Stars TRF
Fifteen Stars

Re: Salesforce loading related objects with single output

This is really not a good design option.
Consider to have 2 separate subjobs, 1 for each object and complete the parent object injection before to go with child object. Both subjobs must be linked by an onSubjobOk link.

TRF
Five Stars

Re: Salesforce loading related objects with single output

Thanks for the reply.

 

But how do I utilize the data coming out of the tmap into the two subjobs?

Fifteen Stars TRF
Fifteen Stars

Re: Salesforce loading related objects with single output

For the parent object, you can let as it is.
For the child object, store the records in memory using a tHashOutput.
On next subjob, get the memorized records using a tHashInput linked with the previous tHashOutput.
If you don't find these components in the palette, you need to add them manually (see related posts about this subject).

TRF
Five Stars

Re: Salesforce loading related objects with single output

One clarifications.  The tHashout component only has "On Component Ok" trigger, I assume I use that correct?  The tHashInput has both "On Component Ok" and "On Subjob Ok".   So I use the "On Subjob Ok" trigger to connect to child object flow, correct?

 

Thanks for your help on this.

Fifteen Stars TRF
Fifteen Stars

Re: Salesforce loading related objects with single output

Not really.
Connect your tMSSQLInput to the tHashInput using the onSubjobOk trigger.
This the 1rst subjob will inject records into the parent object and store records for the child object into the tHashOutput.
Then, the 2nd subjob will get child records from the tHashInput and inject them into Salesforce.

TRF
Five Stars

Re: Salesforce loading related objects with single output

I don't think it is exactly what I am after.  I am processing a set of database records and splitting the data via a tmap into two outputs.  Output 1 is used to create a patient record in a salesforce custom object, and Output 2 is used to create a patient encounter in a salesforce custom object (and relates it to the previously created patient records).  When I create the patient encounter I am using the relations lookup and need to make sure all the patient records have all been created before the patient encounter does the lookup.

 

The challenge is I have to process the source record all the same time, so I am trying to make sure the patients are created before the related patient encounters.

 

Hope this helps.  Thanks again for the help.

Fifteen Stars TRF
Fifteen Stars

Re: Salesforce loading related objects with single output

This is a very common use case and the solution I proposed is exactly what you need to do in such a case.
Imagine you have 1,000,000 records to inject for parent object, then to child object. Do you think you can insert records 1 at a time?
No, you will use bulk components and you will cut the job in 2 subjob, exactly as I proposed.
Believe me, I did it for hundreds of jobs.

TRF
Five Stars

Re: Salesforce loading related objects with single output

Ok, you have me sold.  

 

I'm still confused on your suggestion, if I need to trigger the subjobs using the two outputs from the tMap.  Your suggestions was the single output of the TMssqlInput, this won't work in my scenario (or I am missing something).  I have attached an image of the flow.  Where would i insert the thashout and thashinput to get the Add_EhrEncounters to run second?

Highlighted
Fifteen Stars TRF
Fifteen Stars

Re: Salesforce loading related objects with single output

Remove the tSleep and replace the Add_EhrEncounters by a tHashOutput.
Add a tHashInput connected to the tMSSQLInput by an onSubjobOk trigger.
Connect the Add_EhrEncounters to this tHashInput. Don't forget to refer the tHashOutput from the tHashInput and to to set the schema.

TRF

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog