Five Stars

Issues in handling data transfer from HDFS to Azure BLOB (and SQL DW SP Execution)


Hi,

 

We use Talend component to transfer data from HDFS location to Azure BLOB and then we run SP in SQLDW using the Talend component.
But,While running the end-to-end flow we found that when there is an issue/failure in Stored Procedure (SQLDW)  the job is not returning any error details, Its continuing further as if the SP completed fine.
Because of this the Job completes even though there is an Error in Stored Procedure execution in SQLDW.

We run the HDFS to Azure BLOB transfer Job using tRunJob component. Could any one provide any suggestions to handle this issue ?

 

Thanks,

Sakthi

  • Big Data
  • Data Integration
2 REPLIES
Highlighted
Employee

Re: Issues in handling data transfer from HDFS to Azure BLOB (and SQL DW SP Execution)

In which component do you call the SP?  Are you checking the return code of the component calling the SP?  Are the components and the tRunJob "Die on Error" option set?  

Please post screenshots of jobs and component settings.

Five Stars

Re: Issues in handling data transfer from HDFS to Azure BLOB (and SQL DW SP Execution)

Hi All,

 

We manged to resolve the issue. Yes, Die on Error set already. 

Now, we made changes at SP to throw different error, previously SP catches that and throws divide by zero error. It seems talend job couldnt get the divide by zero error. Now, it is fine and able to catch the SP failure.

 

Thanks