Three ways to set a truststore for an SSL/TLS connection in a Job

Symptom

You want to invoke an HTTPS URL in your Job, and you want to use a tMDMConnection component to access the following MDM URL:

https://localhost:8543/talendmdm/services/soap

But the Job fails to run and throws the error:

Caused by: javax.net.ssl.SSLHandshakeException: SSLHandshakeException invoking https://localhost:8543/talendmdm/services/soap: 
     sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
     unable to find valid certification path to requested target
     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
     at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
     at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1376)
     at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1360)
     at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
     at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:651)
     at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
     ... 10 more

Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

 

Diagnosis

To connect to the HTTPS URL of a non-public server, you must provide the Talend Job or Talend server with the SSL certificate file of the server you are trying to connect to.

 

Connecting to various public servers, such as GitHub or Microsoft, is often easy (that is, you do not have to do any special configuration) because your browser and your Job can check the validity of those servers' SSL certificates with a public certificate authority.

 

However, if you are trying to connect to a non-public server such as your internal Jira server, for example, jira.mycompany.com, that server's SSL certificate is not known to the public certificate authorities, so you need to configure your Job or Talend server to trust that SSL certificate, and therefore trust that Jira server.

 

This means installing that server's SSL certificate, a self-signed certificate, into your environment by storing it into a keystore file that is then used by your Job.

 

Solution

To resolve the issue, perform the following steps:

  1. Download the SSL certificate from the remote server.

  2. Create a keystore file, if necessary, and store the certificate in that keystore file.

  3. Make your Talend Job aware of the location of that keystore file.

Each step is detailed below.

 

Download the SSL certificate from the remote server

 

Using Commandline

If you have the openssl tool, use the appropriate command for your platform:

 

Windows:

openssl s_client -connect {HOSTNAME}:{PORT} 2>NUL <NUL | openssl x509 -outform der > {mykey}.cer

 

Linux/UNIX/Mac OS/X:

openssl s_client -connect {HOSTNAME}:{PORT} 2>/dev/null </dev/null | openssl x509 -outform der > {mykey}.cer

 

Notes about these commands:

  • Certificate files can have various formats and extensions. The DER file format is often stored with either a .der or .cer extension; use .cer here.

  • NUL is the Windows equivalent of Linux/Unix/OSX /dev/null, so 2>NUL and 2>/dev/null specify to send all error output to nowhere, that is, ignore it or get rid of it.

  • <NUL and </dev/null specify to send/read nothing into the command as STDIN so the openssl command is not waiting on input. You may still have to press Enter once or twice for control to return to the console.

 

The .der format is a binary format, so if you try to use type or cat on the .cer file, it will likely be unreadable:

> type microsoft.cer
 0üï10å≈ UUS10U0
Washington10URedmond10U
200116212402Z0üê10tion1UUS10crosUWA10URedmond10ULS CA 40
Microsoft Corporation10UMicrosoft Corporation1

This example connects to https: //www. microsoft. com/ over SSL, so it uses microsoft.cer as the name of the SSL certificate.

 

For a readable version of the file, use the following command; this is useful for troubleshooting:

> openssl x509 -in microsoft.cer -inform der -text -noout

 

For more openssl uses and examples, see the freeCodeCamp OpenSSL Command Cheatsheet web page.

 

Using your browser

If you are accessing an HTTPS service (as opposed to an FTP service, a database service through JDBC, and so on), you can use your browser to export or download the certificate. Most major browsers allow you to export an SSL certificate from a website. Visit the web site with your browser, then export the certificate and store it on your hard drive.

 

  1. Export the SSL certificate using your browser.

    ssl-certificate-export-from-browser-click-the-lock.png

     

  2. Keep default option, DER encoded binary X.509 (.CER).

    export-der-encoded-binary-x509-ssl-certificate.png

     

  3. Specify the path of the exported .cer file, for example, c:\certs\microsoft.cer.

    file-to-export-path.png

     

  4. Click Finish to complete the export of the SSL certificate.

    complete-the-certificate-export-wizard.png

 

Using Google Chrome (on Windows)

  1. Click the Secure text and lock icon in the Location bar.

  2. Click Certificate.

  3. Click Details.

  4. Click Copy to File.

  5. Go through the export wizard:

    1. Click Next.

    2. Keep the default selection, DER encoded binary X.509 (.CER). Click Next.

    3. Name the certificate appropriately, for example, c:\certs\some_site.cer. Click Next.

    4. Click Finish.

    Now you have the remote server's SSL certificate stored at c:\certs\some_site.cer.

 

Create a keystore file, if necessary, and store the certificate in that keystore file

Talend highly recommends using your own separate keystore file, not the default Java keystore file stored at %JAVA_HOME%\jre\lib\security\cacerts. If you use the default Java keystore, your Job cannot run in the Cloud (in a Cloud Engine), but if configured properly it can run through the Cloud on a Remote Engine.

 

The discussion surrounding keystore best practices can be nuanced and complex, but using your own keystore is the simpler option for use with Talend. This article demonstrates both options.

 

The following command creates a keystore at c:\certs\mykeystore.jks, sets its password to changeit (a common convention, but your company may use its own best practices for how to set this password), and imports the microsoft.cer SSL certificate into the new keystore file:

 

keytool -import -alias microsoft -file microsoft.cer -keystore c:/certs/mykeystore.jks -storepass changeit -keypass changeit -noprompt

The -alias microsoft argument means you are giving the name microsoft to the certificate you are importing, so you can refer to this certificate with future keytool commands if necessary.

 

If you want to import the certificate into your default Java keystore, also known as the standard truststore, use the following command:

keytool -import -alias microsoft -keystore "%JAVA_HOME%/jre/lib/security/cacerts" -storepass changeit -keypass changeit
  -file microsoft.der -noprompt

 

If you have any difficulty with %JAVA_HOME%, try using the full path directly instead of using the variable, for example:

keytool -import -alias microsoft -keystore "C:/Program Files/Java/jdk1.8.0_181/jre/lib/security/cacerts" -storepass changeit 
  -keypass changeit -file microsoft.der -noprompt

The quotation marks (") around the "-keystore <cacerts_path>" argument may be necessary because of spaces within the %JAVA_HOME% path.

 

Make your Talend Job aware of the location of your keystore

If you used the default Java keystore location, at %JAVA_HOME%\jre\lib\security\cacerts, you should not have to do anything else to run your Talend Job or Talend server on this machine, your Job is ready to run.

 

However, to run your Job on any other machine, such as your remote JobServer, ESB runtimes, and Remote Engines, you have to do this same certificate-import process on each machine. This is the downside of not creating a separate keystore file to store the private SSL certificate. Also, with this method, your Job cannot run in the Cloud on a Cloud Engine, though it can run on a properly-configured Remote Engine.

 

Every Talend Job, because it is a Java program, automatically looks for certificates in the default Java keystore; that is how this mechanism works. If you chose to create a separate keystore, then you have to explicitly tell your Talend Job where to find the keystore containing the certificate you want it to use.

 

Use any one of the three methods below:

  1. Use the tSetKeystore component:

    tsetkeystore.png

     

  2. Use the tPrejob and tJava components with the following code:

    System.setProperty("javax.net.ssl.trustStore", "C:/Users/username/myKeystore.keystore");
    System.setProperty("javax.net.ssl.trustStorePassword", "password");

    tJava.png

     

  3. Set up the following two JVM arguments for the Job's RUN VM:

    -Djavax.net.ssl.trustStore="C:/Users/username/myKeystore.keystore" 
    -Djavax.net.ssl.trustStorePassword=password

    jvm.png

    The path to your keystore file can be anything you like, as long as the user can read the file. Typically, your Job is executing as talenduser.

Version history
Revision #:
54 of 54
Last update:
‎10-23-2018 11:43 AM
Updated by:
 
Labels (3)
Comments
Employee

The InstallCert Java program is a useful utility for grabbing a SSL certificate from a remote server -- e.g. your Github/Bitbucket/Gitlab server -- and installing it in your local default Java truststore (which will then allow your Talend jobs and servers to trust/connect with that remote server -- after a restart/re-run of your local job/server).

 

High-level instructions:

0) Open a 'cmd' window (or Terminal on *nix systems) in 'Admin' mode

1) Download the InstallCert class attached to this page (or 'git clone' the repository and compile the class)

2) Run it for the remote server in question

3) Copy the generated 'jssecacerts' file to your $JAVA_HOME/jre/lib/security directory 

3a) If you already have a jssecacerts file, then you need to merge this new file into your existing file.
Copy the generated jssecacerts file to your $JAVA_HOME/jre/lib/security directory then run the following keytool command:
(The default password, if you need it, is 'changeit'.)

 

C:\Users\jsmith\dev> copy jssecacerts "%JAVA_HOME%"\jre\lib\security\jssecacerts.tmp
C:\Users\jsmith\dev> keytool -importkeystore -noprompt -srckeystore jssecacerts.tmp -destkeystore jssecacerts

 

4) Re-run your job or Re-start your Talend server (TAC / Jobserver / etc.)

 

To download InstallCert from its repository, compile it, and run it:

 

C:\Users\jsmith\dev> git clone https://github.com/escline/InstallCert
C:\Users\jsmith\dev> cd InstallCert
C:\Users\jsmith\dev\InstallCert> javac InstallCert
C:\Users\jsmith\dev\InstallCert> java -cp . InstallCert

 

This is what a run looks like:

 

C:\Users\psmith\dev\InstallCert>java -cp . InstallCert www.microsoft.com
Loading KeyStore jssecacerts...
Opening connection to www.microsoft.com:443...
Starting SSL handshake...

No errors, certificate is already trusted

Server sent 2 certificate(s):

1 Subject CN=www.microsoft.com, OU=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, ST=WA, C=US
Issuer CN=Microsoft IT TLS CA 4, OU=Microsoft IT, O=Microsoft Corporation, L=Redmond, ST=Washington, C=US
sha1 8f be 50 98 7d 59 f8 c0 23 49 21 62 23 82 50 c2 ed 18 17 6a
md5 dc 23 5d 77 4e b3 66 16 b7 c2 97 53 c6 5b 1b 91

2 Subject CN=Microsoft IT TLS CA 4, OU=Microsoft IT, O=Microsoft Corporation, L=Redmond, ST=Washington, C=US
Issuer CN=Baltimore CyberTrust Root, OU=CyberTrust, O=Baltimore, C=IE
sha1 8a 38 75 5d 09 96 82 3f e8 fa 31 16 a2 77 ce 44 6e ac 4e 99
md5 53 f6 7c 5c c5 29 ff 29 d6 8f c4 e4 dc ab b2 fd

Enter certificate to add to trusted keystore or 'q' to quit: [1]
q
KeyStore not changed

C:\Users\psmith\dev\InstallCert>

The '-cp .' says to put the local directory (.) in the classpath of the Java program you are running.

 


The error message you may see when attempting to get Talend software to talk to a remote SSL-enabled server could be:

Cannot access GIT server. Please check URL and authentication.