Spark job in Running in yarn Cluster Mode in MapR Cluster.
On SubJobOK , at the end of job , I am invoking a Standard Job.
I am getting class no found error. Upon looking log , it seems that the jar containing the sub jar is being referred from user cache. Path is something like …
Exact same Job works fine if run the Main Spark Job in Yarn Client mode
Spark Job – Running in Yarn Cluster Mode – Calling Standard Job thru tRunJob - > Invocation of Sub Job in Failing
Spark Job – Running in Yarn Client Mode – Calling Standard Job thru tRunJob - > Invocation of Sub Job is working fine.
Question – Is the subjob invocation from Spark-Yarn-Mode supported in Talend ? if not what is the work around to invoke Standard job from Spark Job running in yarn Cluster Mode.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.