Four Stars

Issue with Google Bigquery date type

Hi community,

 

With Talend Open Studio for Big Data (Version: 6.4.1), I succeed in loading a file from Google cloud storage  (or flat file) to a Bigquery table with tBigQueryOutput.

Nethertheless, the date fields are loaded in a Bigquery TIMESTAMP type or a STRING type.

 

I do not succeed to load a date field in a Bigquery DATE type.

I tried different ways with SQL instructions (parse_date...),  or with tJava, or with tMap to transform the fields in the correct format.

But at the end, Talend loading in a Bigquery timestamp type or string type, but not in Bigquery date type.

 

Have you ever encountered this problem ?

Do you have a solution, please ?

 

Thanks&regards,

Gilles

 

   

2 REPLIES
Moderator

Re: Issue with Google Bigquery date type

Hi,

Talend supports the same data types than Big Query: https://cloud.google.com/bigquery/preparing-data-for-bigquery#datatypes

Have you tried to insert a tMap into your flow and change the datatypes from the output flow to what BigQuery only allows? After that you have to convert the types with the assignment expressions or you could use the tConvertType component.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Issue with Google Bigquery date type

Hi Sabrina,

Thanks for your answer.

Yes, I tried to insert different components (tMap or tJava tConvertType, ...) and I succeed to change the date type as BigQuery allows.

But at the end of the flow, when Talend loads data, the date type is a timestamp in Bigquery.

In my opinion, it is a Talend issue which not distinguish Bigquery date type and timestamp type.

Best regards,

Gilles