SubJob not working as expected

One Star

SubJob not working as expected

I was working on cleaning up a job I had written by creating subjobs to put in place of elements within the job. I created the jobs I needed, took the pieces out which they were replacing, setup the context var passing and the job executes without error but it is not doing any work when it executes and im not showing any errors in the execution either.
Attached youll find some screen shots that display the old job which works correctly and new job which im having issues with when calling the subJobs.
As you can see I simply took the one part of the original job, stuck it into a new job and im calling that as a subjob rathing than having the elements directly in the main job. I chose a rather simple example here, but there are other parts of the job which deal with multiple look up tables and have multiple mapping elements to them. I wanted to use subjobs so the main job would be simplified and so we would have an easier time working with the individual subjobs through the repository (as we have noticed extreme slow down when trying to work with large jobs through the repository). This also would allow for easier modification of the individual sub jobs as they are separated.
Essentially what should happen with this job, is that it pulls elements that tell it what data needs to be scored, thats the first thing that happens it pulls data from 3 different tables and joins it. It passes this into a tJava row which sets context variables which is the information that gets passed onto the subjobs about how they need to run (context variables are used in queries to pull data and as values of what is getting inserted or updated in other queries). The job with the subjob gets the data and is setting the context variables correctly, but its not actually scoring the data. It is calling the subjob, and I added logging so I could see that the context variables were being passed correctly. However the query doesnt seem to be executing.
Ive executed the job by itself and I get the expected results ( I added a tJava component for it to run by itself so I could set the context variables without it needing to be called by the main job which pulls the needed values from a db). As you can see in that screen shot 108317 rows were processed. When I run the job as subjob calling it through my main job I get no rows processed. This can be seen in the ouput of the job when I execute it, since I turned on the logging of the rows from the output object when I try to run it as a sub job (this will be disabled on normal runs but its a way for me to see what is happening when I run it as a subjob).
Here is the output I get when I run the MainJob:
.------------------------------------------------------+---------+----------+--------+------------+-------.
| tLogRow_3 |
|=-----------------------------------------------------+---------+----------+--------+------------+------=|
|fieldName |elementID|categoryID|maxScore|elementName |batchID|
|=-----------------------------------------------------+---------+----------+--------+------------+------=|
|WORLDSPAN_CODE, AMADEUS_CODE, GALILEO_CODE, SABRE_CODE|20 |5 |35 |BOOKING_CODE|null |
'------------------------------------------------------+---------+----------+--------+------------+-------'
.------------------+-------------------+------------+-------------------+-----.
| tLogRow_2 |
|=-----------------+-------------------+------------+-------------------+----=|
|ELEMENT_IDENTIFIER|CATEGORY_IDENTIFIER|TRX_HOTEL_ID|LKP_TYPE_IDENTIFIER|SCORE|
|=-----------------+-------------------+------------+-------------------+----=|
'------------------+-------------------+------------+-------------------+-----'
Im running java TIS 2.3.3. Why would the job run successfully by itself, but not process correctly when being called as a subjob through another job?
Employee

Re: SubJob not working as expected

To avoid any misunderstanding, don't say "subjob" when you mean "child job". With TOS, subjob is a "reserved word", it means something that a user can see (subjobs are surrounded by colored rectangles since TOS 2.4.0) and it does not mean a child job, that is launched with a tRunJob.
One Star

Re: SubJob not working as expected

Sorry for the wrong nomenclature in my message, Ill start refering from now on to this as a child job.
On a different note I found the issue with what is going on. The job is actually processing correctly, it just returning unexpected results. The reason for the unexpected results appears to be a bug in the handling of passing context variables from parent to child jobs, specifically in the handling of null string values. When a string context var is set to null and passed to a child job, the value is not retained as null. It is actually transformed into the string "null". In my particular job I base the query that is used to gather data based on a context value that is being passed in, specifically I will add a where clause if the value is not null to determine what data needs to be pulled. Since the value was no longer null it caused this to be applied and the query returned no results because there was nothing in the database with "null" as that particular value.
Why does Talend transform the null into "null" in this situation. Shouldn't this simply just be passing variables?
I know that you cannot set the initial value of a context variable to null from a previous message I had posted. But I figured the act of passing a variable would not change the actual value of the data. I know that the value is actually null because right now I am always setting the value to null in a tJava component because we havent implemented its use in this job yet. I have it being set in a tJava component because of the aforementioned inability to set the initial value of a context var to be null. Is null handling off context vars on the radar of things to be fixed?
The previous issue I had with the initial setting of a context variable to null can be found at http://www.talendforge.org/forum/viewtopic.php?id=3093
One Star

Re: SubJob not working as expected

Hi, I create and SQL to extract some records and the results are more than the selection, any idea why?
The SQL is:
"SELECT F4115.IHITM, F4115.IHMCU, F4115.IHCTRY, F4115.IHFY, F4115.IHINVV
FROM F4115
WHERE F4115.IHITM IN ('32004376','32004380','32004381','32004407','32004408','32004409','32004412','32004415','32004417', '32004422','32004437','32004438')"
In the result I have IHITM = "32004416" and "32004436" that were not part of the selection.
Had this ever happens before? Is this a bug?
I am new to this forum any sugestion will be really appreciated.
Thansk!
Community Manager

Re: SubJob not working as expected

Hello EGraveran
Which type of DB do you use? Mysql? Oracle? I see your sql is correct and I test it on mysql and Oracle, all is fine.
Best regards
shong
----------------------------------------------------------
Talend | Data Agility for Modern Business