Error in tBigQueryOutput component

Six Stars

Error in tBigQueryOutput component

Hi All,

 

I have developed a talend job having only three components for insert data into bigquery tables using a csv file. This job is running and inserting data into bigquery table but having below error.

 

Starting job testjob_002 at 13:29 02/11/2018.

[WARN ]: org.jets3t.service.utils.Mimetypes - Unable to find 'mime.types' file in classpath
Starting load job.
[FATAL]: tal_cloud.testjob_002_0_1.testjob_002 - tBigQueryOutput_1_tBQBE 404 Not Found
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F",
"reason" : "notFound"
} ],
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F"

mytalendproject-220908:asia-northeast1.job_4pozTkcPzzMgeoFa4PnHmUz9fK2F
}
com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F",
"reason" : "notFound"
} ],
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:312)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1049)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:410)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
at tal_cloud.testjob_002_0_1.testjob_002.tFileInputDelimited_1Process(testjob_002.java:1474)
at tal_cloud.testjob_002_0_1.testjob_002.runJobInTOS(testjob_002.java:1845)
at tal_cloud.testjob_002_0_1.testjob_002.main(testjob_002.java:1695)
Exception in component tBigQueryOutput_1_tBQBE (testjob_002)
com.google.api.client.googleapis.json.GoogleJsonResponseException: 404 Not Found
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F",
"reason" : "notFound"
} ],
"message" : "Not found: Job mytalendproject-220908:job_4pozTkcPzzMgeoFa4PnHmUz9fK2F"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:312)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1049)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:410)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
at tal_cloud.testjob_002_0_1.testjob_002.tFileInputDelimited_1Process(testjob_002.java:1474)
at tal_cloud.testjob_002_0_1.testjob_002.runJobInTOS(testjob_002.java:1845)
at tal_cloud.testjob_002_0_1.testjob_002.main(testjob_002.java:1695)
Job testjob_002 ended at 13:29 02/11/2018. [exit code=1]

 

 

 

I did not find any solution to remove this error.Please help me.

 

Capture.PNG

 

 

Thanks 

Kailash Yadav 

Moderator

Re: Error in tBigQueryOutput component

Hello Kailash Yadav,

Are you running big data sets? Is it a spark job? Can you please clarify in which Talend version/edition you are?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: Error in tBigQueryOutput component

Hi Sabrina,

 

Thanks for your reply. It's not a spark job its a simple DI Job we are using Talend Big data platform 6.5 version.

 

Thanks

Kailash Yadav  

Six Stars

Re: Error in tBigQueryOutput component

Hi Sabrina,

I am getting the similar error in all the bigquery components for example 

tgooglebigqueryoutput

tgooglebigquerybulkexec

 

throwing job name not found error

{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Not found: Job mytalendproject-220908:job_XpZYr_vRWRVbQONX7gDEGgv8jj9t",
"reason" : "notFound"
} ],
"message" : "Not found: Job mytalendproject-220908:job_XpZYr_vRWRVbQONX7gDEGgv8jj9t"
}


but the job gets executed on the google cloud platform.
the problem here it seems that the job name: "Job mytalendproject-220908:job_XpZYr_vRWRVbQONX7gDEGgv8jj9t"
is incorrect as it does not contain the locale


on the google bigquery the load job that gets executed is : "mytalendproject-220908:asia-northeast1.job_XpZYr_vRWRVbQONX7gDEGgv8jj9t"

 

Please help me it very critical now and having this issue from last two weeks.

 

Thanks

Kailash Yadav

 

One Star

Re: Error in tBigQueryOutput component

Hello I have the same problem. 

 

Seems like it happens when the GCP project is not located either in the US or Europe. Outside those 2 regions you need to add an extra "location" value in the request. For exemple my GCP project is in Asia-east2 so he's looking for a job named : "MyProject_id":asia-east2."Myjobnumber" but Talend is sending a request with only "MyProject_id":"Myjobnumber" as it seems like we can't pass the location value in the tBigQueryOutput component. 

 

I'm still searching for an answer. Anyone can help ? 

 

Thanks

Employee

Re: Error in tBigQueryOutput component

Hi,

 

    If you are using a subscription version of Talend, could you please request support team to create a JIRA ticket to verify this product feature? If you are using a free version of Talend, please use the below link to create the JIRA ticket.

 

https://jira.talendforge.org/secure/Dashboard.jspa

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Download