tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

Five Stars

tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

I have tried setting up a simple pipeline in Talend Studio that takes Opportunity records from Salesforce, drops them into a CSV, uploads to Google Cloud Storage and then imports into a BigQuery table. Everything works up until the BigQuery import, at which point I get the following error:

 

Exception in component tBigQueryBulkExec_1 (Opps_to_BigQuery)
java.lang.RuntimeException: Job failed: BigQueryError{reason=invalid, location=null, message=You are loading data without specifying data format, data will be treated as CSV format by default. If this is not what you mean, please specify data format by --source_format.}
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.tBigQueryBulkExec_1Process(Opps_to_BigQuery.java:481)
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.runJobInTOS(Opps_to_BigQuery.java:983)
	at sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery.main(Opps_to_BigQuery.java:783)
[FATAL]: sf_opps_to_bigquery.opps_to_bigquery_0_1.Opps_to_BigQuery - tBigQueryBulkExec_1 Job failed: BigQueryError{reason=invalid, location=null, message=You are loading data without specifying data format, data will be treated as CSV format by default. If this is not what you mean, please specify data format by --source_format.}

Has anyone experienced this or know a way to get BigQuery to read from the repository schema?

Screen Shot 2019-11-25 at 2.36.27 pm.png

Community Manager

Re: tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

Does the same thing happen if you untick the "Bulk file already exists in Google Storage" and use that mechanism to send the data?

Five Stars

Re: tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

Good thought. I've just tried that and unfortunately, that gives me the same error. Smiley Sad

Employee

Re: tBigQueryBulkExec importing CSV error: You are loading data without specifying data format.

Hi,

 

   Could you please check whether the previous tBigQueryOutputBulk is correctly pointing to the right area inside Google Storage? Especially the file extension.

 

    There is a solution Template for Salesforce to Google BigQuery in Talend Academy. I would recommend to try the solution with the Template.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

 

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Self-service Talend Migration: Moving from On-Premises to the Cloud

Move from On-Premises to the Cloud by following the advice of experts

Read Now

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog