Handle AWS Library Dependencies on job

Five Stars

Handle AWS Library Dependencies on job

Hello i have problem with using AWS SDK in my tJavaFlex code get this exception:

 

Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.SDKGlobalConfiguration.isInRegionOptimizedModeEnabled()Z
at com.amazonaws.ClientConfigurationFactory.getConfig(ClientConfigurationFactory.java:35)
at com.amazonaws.client.builder.AwsClientBuilder.resolveClientConfiguration(AwsClientBuilder.java:167)
at com.amazonaws.client.builder.AwsClientBuilder.access$000(AwsClientBuilder.java:54)
at com.amazonaws.client.builder.AwsClientBuilder$SyncBuilderParams.<init>(AwsClientBuilder.java:478)
at com.amazonaws.client.builder.AwsClientBuilder.getSyncClientParams(AwsClientBuilder.java:419)
at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)

 

After doing some googling, I found that this problem is because the AWS SDK dependencies, maybe because we are using sofware that is using old-Talend version (so don't tell to upgrade Talend version) so it still using old AWS SDK version. However, i have try to edit routine libraries in routines and setup routine libraries in code and see the libraries already installed in routines and jobs but in runtime the tJavaFlex still refers to old AWS SDK.

 

How to manage tJavaFlex component to use specific libraries/dependencies correctly?


Accepted Solutions
Highlighted
Five Stars

Re: Handle AWS Library Dependencies on job

Already found solution need to change all the component dependencies in the folder plugins -> xxxlocalprovider -> components -> [Your Components] -> [component_name]_java.xml

 

Then import all the needed dependencies from external modules


All Replies
Highlighted
Five Stars

Re: Handle AWS Library Dependencies on job

Already found solution need to change all the component dependencies in the folder plugins -> xxxlocalprovider -> components -> [Your Components] -> [component_name]_java.xml

 

Then import all the needed dependencies from external modules

Five Stars

Re: Handle AWS Library Dependencies on job

Hi ,

 

Thanks for your post. It is helpful!

 

Could you please help me in knowing how you are handling AWS encryption through talend.

If you give me any sample talend job then that would be very useful.

Also would like to know about how you are using jars/libraries of AWS SDK in talend?

 

Six Stars

Re: Handle AWS Library Dependencies on job

Could you please help me in knowing how you are handling AWS encryption through talend.

In there are two types of encryption

1) clientside

check encrypt box. give kms key.

2)serverside

check serverside encryption box give kms key

 

do not use both encryption at a time.

use JCE policies by replacing those jars in C:\Program Files\Java\jre1.8.0_77\lib\security

1)local_policy.jar

2)US_export_policy.jar

 

 

Also would like to know about how you are using jars/libraries of AWS SDK in talend?

download AWS SDK from aws website. size will be more than 300 MB

extract and rename aws-java-sdk-1.11.507.jar to requested sdk jar version name from component.

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach

Watch