No Spark Components in "Talend Open Studio for Big Data" v6?

One Star

No Spark Components in "Talend Open Studio for Big Data" v6?

Hi,
I was using TOS v5.6 and then upgraded to v6.0 on MAC OSX and there are no components for SPARK in the new version. Has Talend given up on providing components for spark??
Best regards,
Amir.
Seventeen Stars

Re: No Spark Components in "Talend Open Studio for Big Data" v6?

hi ahassan
should be Smiley Happy
This feature is available in Talend big data products, starting from the Community version.


(not working on Mac)
regards
laurent
One Star

Re: No Spark Components in "Talend Open Studio for Big Data" v6?

I also do not see any Spark components in Big Data Open Studio
One Star

Re: No Spark Components in "Talend Open Studio for Big Data" v6?

Same here...  tried looking at marketplace and none there.
_Ben
Community Manager

Re: No Spark Components in "Talend Open Studio for Big Data" v6?

Hi
Sorry for the late answer, I thought I had answered already, but obviously didn't.
Spark components were delivered as Technical Preview in v5.6 but couldn't leverage the full Studio big data capabilities such as turning a standard Job into an M/R Job then into a Streaming Job. The effortless migration of a M/R jobs to Spark was one of the key requirements in our 6.0 (subscription) roadmap.
However the development of this capability involved an overal change of the way we handled Spark so that it becomes natively part of the Code generator. The maintenance issue made it impossible for us to keep two different sets of components generating Spark (given all links to Hadoop distros and versions) and thus we had to drop the initial tSpark** components.
You could give a try to reuse the 5.6 components in the v6 but that's at your own risk, because the API has changed quite a lot and thus to make it work, it will probably require  a rather large amount of work.
I hope this answers your question. Sorry for the inconvenience. Feel free to express any feature request or need in our JIRA.

Elisa

Seven Stars

Re: No Spark Components in "Talend Open Studio for Big Data" v6?

Hi,
We are using enterprise Data Fabric Edition for Talend(6.2.1). but I am not seeing Spark Component in Pallette. Can you please help here how to enable or activate them.