I am new to Talend big data job (Spark). I am trying to read data from thiveinput and generating a file but facing below error. The same cluster connections are working fine in data integration job but not working in big data job. In addition, are there any documents available to use Spark components in Talend (like tSQLrow for hive and Spark context,thiveinput, etc)?
[WARN ]: org.apache.spark.util.Utils - Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
[ERROR]: org.apache.spark.SparkContext - Error initializing SparkContext
We are supposing that you are aware that each component has pieces of information indicating the families they belong to and the conditions for their eventual availability.
The search on Talend Help Center can help on this. As you arer using Talend BigData solution, here are the different lists for Big data Jobs on different frameworks in your product
Let us know if it is OK with you.
Screenshots of your bigdata job setting screenshots will be preferred.
Watch the recorded webinar!
Accelerate your data lake projects with an agile approach
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.