You are facing an issue on TDM on Spark. Here are the environment details to check and collect.
Big data cluster in use: CDH, HDP, MapR...
Component configuration: Signature, Path and details
Spark configuration (Spark version, details and cluster config)
Spark runtime: Local or cluster (Standalone or Yarn Client)