Validations in Big Data Batch Job

Highlighted
Four Stars

Validations in Big Data Batch Job

Hi Team,

 

I have development experience with Talend standard jobs but I am new to developing Big Data Batch Jobs in Talend. I have used tschemacompliance to validate schema data types and lengths in standard jobs in Talend. I am looking for an equivalent component in Big Data Batch job. Can you please help me understand how best we can achieve data type and length validations in a batch job? Looking forward for a timely response. 

 

Thanks and Regards,

Shiv Soni

Four Stars

Re: Validations in Big Data Batch Job

Hi,

 

Can anyone help with this please?

Moderator

Re: Validations in Big Data Batch Job

Hello,

The tschemacompliance component is a DI component and will not available in bigdata spark job, so far.

Have you tried to utilize a DI Job to orchestrate your spark job by using tRunJob component?

Best regards

Sabrina

 

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now