tBigQueryInput - java.lang.NullPointerException

Nine Stars

tBigQueryInput - java.lang.NullPointerException

I am trying to connect to a vendor's Google Big Query using a tBigQueryInput component.

 

I no longer get the 400 connection error, so I think my connection info is correct now.

 

My test query (runs successfully in a web browser and returns 2018-11-30):

SELECT CURRENT_DATE() as myDate;

 

Talend_BigQuery_1.png

 

Talend_BigQuery_2.png 

 

The error I am now getting:

 

Exception in component tBigQueryInput_2 (BigQuery)
java.lang.NullPointerException
    at com.google.api.client.repackaged.com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
    at com.google.api.client.util.Preconditions.checkNotNull(Preconditions.java:127)
    at com.google.api.client.json.jackson2.JacksonFactory.createJsonParser(JacksonFactory.java:92)
    at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:85)
    at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:81)
    at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:88)
    at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:287)
    at com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeTokenRequest.execute(GoogleAuthorizationCodeTokenRequest.java:158)
    at talenddemosjava.bigquery_0_1.BigQuery.tBigQueryInput_2Process(BigQuery.java:714)
    at talenddemosjava.bigquery_0_1.BigQuery.runJobInTOS(BigQuery.java:1337)
    at talenddemosjava.bigquery_0_1.BigQuery.main(BigQuery.java:1186)

Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

Anyone?
Employee

Re: tBigQueryInput - java.lang.NullPointerException

Hi,

 

     It is saying that the not null check is failing. Could you please toggle the not null option in the schema and verify whether it is working? Thought to quickly check whether it is some issue related to it.

 

Warm Regards,

 

Nikhil Thampi

 

 

Forteen Stars

Re: tBigQueryInput - java.lang.NullPointerException

 

as variant also check work with service account credentials file

Screen Shot 2018-12-04 at 10.38.35 PM.png

 

and it works

Screen Shot 2018-12-04 at 10.38.42 PM.png

-----------
Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

Are you referring to the "Nullable" check box in edit schema?

 

I unchecked it and ran the job again, same error.

 

 

Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

What version are you using?
I am on Version: 7.0.1 and I don't see the Authentication mode drop down.
Forteen Stars

Re: tBigQueryInput - java.lang.NullPointerException

7.1.1

-----------
Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

I installed 7.1.1 and I was able to get the date successfully when I used the Authentication mode Service Account!

 

When I try to pull data from a single column in a table I am getting this error now:

At least job "BigQuery" has a compile errors, please fix and export again.

 

For the tBigQueryInput component, in the Query, do I need to put extra single quotes next to any single quotes?

 

For example if I am trying to check for a specific date:

WHERE myColumn >= DATE('2018-12-04')

 

This SQL statement runs fine in Google BigQuery web browser and returns a result:

SELECT 'test' as myDate;

 

In my Talend job it gives the error:

org.talend.designer.runprocess.ProcessorException: Job compile errors
At least job "BigQuery" has a compile errors, please fix and export again.
Error Line: 524
Detail Message: jobId_tBigQueryInput_6 cannot be resolved to a variable
There may be some other errors caused by JVM compatibility. Make sure your JVM setup is similar to the studio.

Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

There seems to be a major bug, the issue is Talend always fails when you have more than one tBigQueryInput in a job.

 

I opened this bug report with the steps to always reproduce the issue:

https://jira.talendforge.org/browse/TDI-41501

Highlighted
Forteen Stars

Re: tBigQueryInput - java.lang.NullPointerException

it is possible.

I have not a time to confirm, but we have similar issue with Kudu components (7.0.1) - when more than 1 Kudu component in SubJob - compilation error "duplicated variable"

 

Kudu -> Kudu not work, but Kudu ->csv + csv to Kudu - work

-----------
Nine Stars

Re: tBigQueryInput - java.lang.NullPointerException

Did you get a change to try the steps?

Do you think the issue will be corrected by the next release?

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog