How can I load multiple files from Google Cloud Storage into a single table in BigQuery?

Highlighted
Five Stars

How can I load multiple files from Google Cloud Storage into a single table in BigQuery?

I wonder if there is a way of bulk loading a number of files within a Google Cloud bucket into a BigQuery table since there is no tFileList component for GS in Talend?

 

 

Forteen Stars

Re: How can I load multiple files from Google Cloud Storage into a single table in BigQuery?

You could check Google API

 

https://cloud.google.com/storage/docs/json_api/v1/objects/list

 

in Talend it would be tRESTClient

-----------
Five Stars

Re: How can I load multiple files from Google Cloud Storage into a single table in BigQuery?

Thanks for your helpful response. Are you able to guide me on how to set the tRestClient component up for this? Apart from the URL, I struggle to work out how to setup the parameters e.g authentication token so that it can talk to our google cloud or bigquery.

 

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Enabling Data Governance

Learn how to enable Data Governance

Watch Now

The Definitive Guide to Government Data Quality

Take a peek at the definitive guide to Government Data Quality

Read