tBigQueryInput - java.lang.NullPointerException

Nine Stars

tBigQueryInput - java.lang.NullPointerException

I am trying to connect to a vendor's Google Big Query using a tBigQueryInput component.

 

I no longer get the 400 connection error, so I think my connection info is correct now.

 

My test query (runs successfully in a web browser and returns 2018-11-30):

SELECT CURRENT_DATE() as myDate;

 

Talend_BigQuery_1.png

 

Talend_BigQuery_2.png 

 

The error I am now getting:

 

Exception in component tBigQueryInput_2 (BigQuery)
java.lang.NullPointerException
    at com.google.api.client.repackaged.com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
    at com.google.api.client.util.Preconditions.checkNotNull(Preconditions.java:127)
    at com.google.api.client.json.jackson2.JacksonFactory.createJsonParser(JacksonFactory.java:92)
    at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:85)
    at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:81)
    at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:88)
    at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:287)
    at com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeTokenRequest.execute(GoogleAuthorizationCodeTokenRequest.java:158)
    at talenddemosjava.bigquery_0_1.BigQuery.tBigQueryInput_2Process(BigQuery.java:714)
    at talenddemosjava.bigquery_0_1.BigQuery.runJobInTOS(BigQuery.java:1337)
    at talenddemosjava.bigquery_0_1.BigQuery.main(BigQuery.java:1186)

Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

Anyone?
Employee

Re: tBigQueryInput - java.lang.NullPointerException

Hi,

 

     It is saying that the not null check is failing. Could you please toggle the not null option in the schema and verify whether it is working? Thought to quickly check whether it is some issue related to it.

 

Warm Regards,

 

Nikhil Thampi

 

 


Warm Regards,
Nikhil Thampi
Please appreciate our members by giving Kudos for spending their time for your query. If your query is answered, please mark the topic as resolved :-)
Thirteen Stars

Re: tBigQueryInput - java.lang.NullPointerException

 

as variant also check work with service account credentials file

Screen Shot 2018-12-04 at 10.38.35 PM.png

 

and it works

Screen Shot 2018-12-04 at 10.38.42 PM.png

-----------
Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

Are you referring to the "Nullable" check box in edit schema?

 

I unchecked it and ran the job again, same error.

 

 

Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

What version are you using?
I am on Version: 7.0.1 and I don't see the Authentication mode drop down.
Thirteen Stars

Re: tBigQueryInput - java.lang.NullPointerException

7.1.1

-----------
Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

I installed 7.1.1 and I was able to get the date successfully when I used the Authentication mode Service Account!

 

When I try to pull data from a single column in a table I am getting this error now:

At least job "BigQuery" has a compile errors, please fix and export again.

 

For the tBigQueryInput component, in the Query, do I need to put extra single quotes next to any single quotes?

 

For example if I am trying to check for a specific date:

WHERE myColumn >= DATE('2018-12-04')

 

This SQL statement runs fine in Google BigQuery web browser and returns a result:

SELECT 'test' as myDate;

 

In my Talend job it gives the error:

org.talend.designer.runprocess.ProcessorException: Job compile errors
At least job "BigQuery" has a compile errors, please fix and export again.
Error Line: 524
Detail Message: jobId_tBigQueryInput_6 cannot be resolved to a variable
There may be some other errors caused by JVM compatibility. Make sure your JVM setup is similar to the studio.

Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

There seems to be a major bug, the issue is Talend always fails when you have more than one tBigQueryInput in a job.

 

I opened this bug report with the steps to always reproduce the issue:

https://jira.talendforge.org/browse/TDI-41501

Highlighted
Thirteen Stars

Re: tBigQueryInput - java.lang.NullPointerException

it is possible.

I have not a time to confirm, but we have similar issue with Kudu components (7.0.1) - when more than 1 Kudu component in SubJob - compilation error "duplicated variable"

 

Kudu -> Kudu not work, but Kudu ->csv + csv to Kudu - work

-----------
Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

Did you get a change to try the steps?

Do you think the issue will be corrected by the next release?

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.