Flow Control

One Star

Flow Control

Hi
1,bill,smith
1,paul,green
1,rob,black
2,simon,bridges
2,garry, martins
2,henry, cowley
3...
I have a tFileInputDelimited with the data above. This is connected to a tjavaflex.
tFileInputDelimited --(main)-->tJavaFlex--OnComponentOk-->tMySQLRow--

The tjavaflex should put all the values for 1 in one treeMap. it should then move to the tmysql component get records from a db that have value 1. Then go to a java component that compares the database values with the tmap values and if there are differences writes to the MYSQL DB.
Then the tJavaflex should put all the values for 2 in a new treeMap. again move to the tmysql component get recrods from a db that hava value 2. Then go to a java component that compares the database values with the treemap and if there are differences writes to the MYSQL DB.
and so on.
I am not sure how to get the tJavaFlex to create the TreeMap with the 1 values then move on to work with these.
Then to return to the tJavaFlex create a new TreeMap with the 2 values then move on to work with these.
and back again for 3.
Thanks for any help.
Community Manager

Re: Flow Control

Hi
Maybe, you don't need to put the data into treeMap, just iterate each data. For example, below is my solution for your request:
Let's say that you have three columns in your delimited file:
id,name1,name2
1,bill,smith
1,paul,green
1,rob,black
2,simon,bridges
2,garry, martins
2,henry, cowley
3...
tFileInputDelimited--main(row1)--tFlowToIterate-->tMysqlInput--main--tJavaRow--runIf--tFixedFlowInput--main--tMysqlOutput
tMysqlInput: select data from database where id equals to current id extract from file, eg:
"select name1, name2 from person where id="+(Integer)globalMap.get("row1.id")
tJavaRow: compare the database data with current data extract from file, eg:
if(input_row.name1.equals((String)globalMap.get("row1.name1")&&input_row.name2.equals((String)globalMap.get("row1.name2")){
globalMap.put("isSame", true);
}else{
globalMap.put("isSame", false);
}
Set the condition of runIf connector as:
!(Boolean)globalMap.get("isSame") //if the data are different, fires next process
tFixedFlowInput, generate the current data extract from file, define the three columns (id, name1, name2) on schema.
column:value
idSmiley SadInteger)globalMap.get("row1.id")
name1Smiley SadString)globalMap.get("row1.name1")
name2Smiley SadString)globalMap.get("row1.name2")
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Flow Control

Hi Shong
Thanks for your response.
My description is a simplified verision of the job I need the TreeMap because the actual data contains dates that need to be put in order.
Also as well as checking if the same record exists it need to check that all records exist and also check if there are any gaps between dates. and if there are to add the gaps to the DB.
Community Manager

Re: Flow Control

hi
OK. The problem is that we can fire next process base on each row or only when all rows finishes, so, it is impossible to design the job as you described, fires next process when all the rows for value 1 finishes. Don't know all the details of your real request, you may process next step after all rows are put into TreeMap on tJavaFlex.
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Flow Control

So how to do what I need to do.
I have records for several thousand customers.
For each customer I have list of different periods. with start and end times for each customer I need to get these in order of end date (previously I was going to use a treemap).
I need to find the gaps that exist in those dates(End date does not neet a start date).
Finally I need to compare the start and end dates with those previously stored for the customer to check if they have changed?