From Thursday, July, 9, 3:00 PM Pacific,
our Community site will be in
read-only mode
through Sunday, July 12th.
Thank you for your patience.

Configure Talend to use ADLS

Highlighted
Five Stars

Configure Talend to use ADLS

How do we correctly configure Talend to use ADLS for storage? The tHDFSConnection requires you to select a big data distro and version otherwise ADLS is not listed as a valid option. In this case ADLS is just a cloud based HDFS compliant file system and the big data distro option makes no sense.

Why do we have to select a distro and which one should we use for this component to work with both ADLS Gen1 and Gen2? Please tell me if I’m completely missing something here?

NB. I’m aware of the separate fact that Cloudera 5.11 introduced support for ADLS but this is only in relation to a Cloudera cluster using ADLS as its storage layer when using Hive, MapReduce, Impala, etc. This is obviously not related to using Talend to read / write to ADLS.
Highlighted
Moderator

Re: Configure Talend to use ADLS

Hello,

Here exists a jira issue on talend bug tracker

https://jira.talendforge.org/browse/TBD-7116

Let us know if it is what you are looking for.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Five Stars

Re: Configure Talend to use ADLS

Hi Sabrina,

No, this is not what I'm referring to. That ticket relates to finding
ADLS components on the palette. My issue relates to a lack of
instructions on how to correctly use those HDFS components to work with
ADLS due to their requirement to specify a specific Hadoop distribution.
Incidentally, that ticket remains as "unresolved" but I notice that when
I search for ADLS that the HDFS components are returned on the palette.
Hence you can probably have somebody test and confirm this and close
that ticket. Not that it helps my issue of course.
Best
Highlighted
Employee

Re: Configure Talend to use ADLS

You can pick Cloudera or Horton (aka HD Insight) obviously for you will be irrelevant, but the component will pick the right libraries for ADLS, in this case the best scenario is to have a unique component for ADLS and this component will come later on but in Parallel you can access to ADLS using this approach.

 

https://help.talend.com/reader/98wTdxOyJjhkll4BXt6Xcw/rzE1cW6_Jsslh~TM5GQ1Uw

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now