Four Stars

job works in TOS but breaks while using ETL

Hi!

 

I have a Talend Job that is working well when I execute it on TOS. But when i use the ETL in the company software, it breaks.

I have an error message saying that there is an error in a function I developped (used in tMap).

 

Why the job is working in TOS but not when using the ETL? And how can I know how to fix it?

 

I replaced my function by a number (the function returns a double) and it works. So I can confirm the issue is the function.

 

Does anyone know what to do?

 

Thanks

 

  • Data Integration
1 ACCEPTED SOLUTION

Accepted Solutions
Nine Stars

Re: job works in TOS but breaks while using ETL

Issue not in environment, issue - in the data

 

If Your function return double - for 99,99% some of values in production environment have NULL values and not NULL when You run it locally

 

in this case - You  need add NULL handle to Your function (same for divide by 0)

what logic must be? it depend from what is Your goal:
- You can return null as well
- You can auto-replace for default value

- etc

 

-----------
6 REPLIES
Nine Stars

Re: job works in TOS but breaks while using ETL

Issue not in environment, issue - in the data

 

If Your function return double - for 99,99% some of values in production environment have NULL values and not NULL when You run it locally

 

in this case - You  need add NULL handle to Your function (same for divide by 0)

what logic must be? it depend from what is Your goal:
- You can return null as well
- You can auto-replace for default value

- etc

 

-----------
Four Stars

Re: job works in TOS but breaks while using ETL

Hi,

 

I already considered this case. My function calculate a rate. If one of the entry variables is null, it returns null.

For the division, I use a function that returns 0 if one of the variable is null or equals 0.

And in my tMap (and my schemas), the variables affected are nullable.

 

The error message is not a NullPointerException so I think it must be something else.

 

I try to get my error message again and i'll post it within an hour. It might help.

 

Edit : got an issue independant from Talend. I'll post the error as soon as I can.

Four Stars

Re: job works in TOS but breaks while using ETL

I cut some lines (confidential infos) but this is a large part of the error generated

 

GRAVE : routines/ValueUtil :
at routines.clientUtil.getRate(myRoutine.java:263)
at alimentation.client_myAlim.tFileInputDelimited_2Process(myAlim.java:5435)
at alimentation.client_myAlim.tFileList_2Process(myAlim.java:1847)
at alimentation.client_myAlim.runJobInTOS(myAlim.java:35459)
at alimentation.client_myAlim.runJob(myAlim.java:35284)
at com.client.framework.feed.ExternalFeedPluginManager.runJobTalend(ExternalFeedPluginManager.java:196)
at java.lang.Thread.run(Thread.java:745)

 

routines.ValueUtil from [Module "deployment.0-feed_20170719_162239.ear:main" from Service Module Loader] :
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:205)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:455)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:404)
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:385)
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:130)
at routines.clientUtil.getRate(clientUtil.java:263)
at alimentation.client_myAlim.tFileInputDelimited_2Process(myAlim.java:5435)
at alimentation.client_myAlim.tFileList_2Process(myAlim.java:1847)
at alimentation.client_myAlim.runJobInTOS(myAlim.java:35459)
at alimentation.client_myAlim.runJob(myAlim.java:35284)
at com.client.framework.feed.ExternalFeedPluginManager.runJobTalend(ExternalFeedPluginManager.java:196)
at java.lang.Thread.run(Thread.java:745)

 

 

The datas source in TOS and with the ETL is exactly the same. So, I don't get why it's working in TOS and not with the ETL...

Employee

Re: job works in TOS but breaks while using ETL

Please raise a support ticket on this if it is not working in your subscription version.

Four Stars

Re: job works in TOS but breaks while using ETL

Hi! 

 

I completely forgot to reply!

Well, the error wasn't understandable  but, in the end, the mistake was mine.

I had the case where one of my value was just a blank space... So it wasn't null but it wasn't an expected content so it broke. I used the TRIM function to solve it.

 

So, like almost every time, it wasn't a bug, it was just an uncomplete function.

 

But thanks for your replies and your reactivity, it's cool to know that we have people ready to help newbies like me when we need Smiley Happy

Nine Stars

Re: job works in TOS but breaks while using ETL


EmilieR wrote:

Hi! 

 

I completely forgot to reply!

Well, the error wasn't understandable  but, in the end, the mistake was mine.

I had the case where one of my value was just a blank space... So it wasn't null but it wasn't an expected content so it broke. I used the TRIM function to solve it.

 

So, like almost every time, it wasn't a bug, it was just an uncomplete function.

 

But thanks for your replies and your reactivity, it's cool to know that we have people ready to help newbies like me when we need Smiley Happy


welcome back! :-)

 

it is return us to original error reason:

 

"different data on dev machine and in live environment",

NULL or empty - it just details, but most important - always need have proper test data set

-----------