Four Stars

Validations in Big Data Batch Job

Hi Team,

 

I have development experience with Talend standard jobs but I am new to developing Big Data Batch Jobs in Talend. I have used tschemacompliance to validate schema data types and lengths in standard jobs in Talend. I am looking for an equivalent component in Big Data Batch job. Can you please help me understand how best we can achieve data type and length validations in a batch job? Looking forward for a timely response. 

 

Thanks and Regards,

Shiv Soni

2 REPLIES
Four Stars

Re: Validations in Big Data Batch Job

Hi,

 

Can anyone help with this please?

Moderator

Re: Validations in Big Data Batch Job

Hello,

The tschemacompliance component is a DI component and will not available in bigdata spark job, so far.

Have you tried to utilize a DI Job to orchestrate your spark job by using tRunJob component?

Best regards

Sabrina

 

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.