Can anyone please advice how to get complete log that comes on console to a file in case of spark batch job?
Any help will be much appreciated.
Could you please have a look at this KB article about:https://community.talend.com/t5/Migration-Configuration-and/Cannot-Get-Talend-Spark-Job-Execution-Lo... to see if Spark History can meet your needs?
Here is online document about:TalendHelpCenter:Logging and checkpointing the activities of your Apache Spark Job.
Try Talend Cloud free for 30 days.
Introduction to Talend Open Studio for Data Integration.
Practical steps to developing your data integration strategy.
Create systems and workflow to manage clean data ingestion and data transformation.