What kind of jobs can be scheduled in Oozie?

One Star

What kind of jobs can be scheduled in Oozie?

I have several ETL jobs that are running in Talend. Due to the amount data we would like to schedule these jobs in a Hadoop cluster. What kind of jobs can Talend schedule to Oozie/Hadoop?
Can I just schedule my current job (a combination of MS SQL inputs, joined with tMap and exported to CSV for Google BigQuery with some Java code for transforms) run in the Oozie scheduler? Or do I need to rewrite my joins in Pig Latin?
What would the best strategy be to get data from MS SQL into the Hadoop cluster using Talend?
Highlighted
Employee

Re: What kind of jobs can be scheduled in Oozie?

Any Talend job that uses a big data connector, HDFS, Hive, Pig should be schedulable using the Oozie tab, provided the cluster info has been filled out.
You would need to convert your joins to PigLatin or use the tPigMap component.
The SQOOP components are probably the best for writing RDBMS tables to HDFS or Hive.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now