From Thursday, July, 9, 3:00 PM Pacific,
our Community site will be in
read-only mode
through Sunday, July 12th.
Thank you for your patience.

spark job in talend

Highlighted
Four Stars

spark job in talend

I am trying to create a spark job to iterate through a file placed in hdfs and store it into an ArrayList. That ArrayList will be used in a java code for lookUp. But, the iterate operation is not supported as well as it is not populating the ArrayList too.

Many components are not available in the spark job which are there in a standard job, making it difficult to solve the problem statement.

 

Any suggestions are welcome

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog