[resolved] Talend BigQuery Authorization

One Star

[resolved] Talend BigQuery Authorization

Heya. Does Talend (Big Data) support connections to Google Service Accounts? What I've seen when looking around is the use of Web Service Accounts which require manual copying and pasting of Access tokens.
*Edit: Specifically talking about tBigQueryInput / output components.

Re: [resolved] Talend BigQuery Authorization

Have you already reviewed component reference about:TalendHelpCenter:/tBigQueryInput?
For Authorization code, pasting the authorization code provided by Google for the access you are building.
To obtain the authorization code, you need to execute the Job using this component and when this Job pauses execution to print out an URL address, you navigate to this address to copy the authorization code displayed.
Is there any error when manual copying and pasting of Access tokens?
Best regards
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: [resolved] Talend BigQuery Authorization

Hey Sabrina, thanks for replying.
Yes, I've looked into the component. We're intending to use Talend to automatically connect to the BigQuery endpoints, without manual interference. This would be done by using Google Service Account credentials (for server to server connection) instead of Web Applications credentials. Similar to jlollings tGoogleAnalyticsInput component.
To to specify my question in a bit more detail:
- Does tBigQueryInput support Service Account authorization (pretty sure the answer is no)
- Is there a way to automate the retrieval of an Authorization Code from a Google Web Application, and use it in tBigQueryInput
Edit: Looks like the answer is no to the first question and yes to the second. In fact the default settings of tBigQueryInput generate a refresh token which allows (indefinite?) access for an application or server to the Google API. From Talend's documentation: 
Advanced settings | token properties File Name | 'At the first Job execution using the Authorization code you have obtained from Google BigQuery, the value in this field is the directory and the name of that refresh token file to be created and used; if that token file has been created and you need to reuse it, you have to specify its directory and file name in this field.'
From Google's documentation:
After successfully authorizing access to the BigQuery API using the OAuth 2.0 web server flow with offline access enabled your application will be given two tokens: an access token and a refresh token.

This looks to be handled automatically without configuration by the BigQuery components, even though this was unclear for me. I'm curious to find out (if there are) any expiration for this token. Great experience so far!

Side note: I've tried this using Google's Installed Application credentials, but I'm confident this works with Web Application credentials also.


Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences


Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now