Four Stars

Bigquery Table Handling Error

Hi,
I am trying to connect to Google Big query table from Talend. While fetching the data from Big query I am getting the below error.
Exception in component tBigQueryInput_3
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
"code" : 403,
"errors" : ,
"message" : "Response too large to return."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:159)
at com.google.api.client.googleapis.json.GoogleJsonResponseException.execute(GoogleJsonResponseException.java:187)
at com.google.api.client.googleapis.services.GoogleClient.executeUnparsed(GoogleClient.java:115)
at com.google.api.client.http.json.JsonHttpRequest.executeUnparsed(JsonHttpRequest.java:112)
at com.google.api.services.bigquery.Bigquery$Jobs$GetQueryResults.execute(Bigquery.java:1755)
at tutorials.mqs_tgt_0_1.MQS_TGT.tBigQueryInput_3Process(MQS_TGT.java:2256)
at tutorials.mqs_tgt_0_1.MQS_TGT.runJobInTOS(MQS_TGT.java:6135)
at tutorials.mqs_tgt_0_1.MQS_TGT.main(MQS_TGT.java:5960)
Is there way to overcome the above issue.
While querying the data directly in Google Biquery we have option to allow larger results. But there is no option available in Talend tBigQueryInput component to allow larger result from Bigquery table.
Thanks
Muthuraj
8 REPLIES
Moderator

Re: Bigquery Table Handling Error

Hi,
Could you indicate the build version you're using? Have you checked the jira issue https://jira.talendforge.org/browse/TDI-28445?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Bigquery Table Handling Error

Hi,
Please find the below details.
Version: 5.3.1
Build id: r104014
Regards
Muthuraj
Moderator

Re: Bigquery Table Handling Error

Hi,
Could you please also show us your tBigQueryInput query?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Bigquery Table Handling Error

Hi Sabrina,
I was in vacation for a month. Please find the query i used for the job.
"select * from as TR1 JOIN each as TH4 on TR1.TRACK_ID = TH4.TRACK_ID where DATE(TR1.INSERT_DATETIME) = DATE(DATE_ADD(TIMESTAMP(CURRENT_DATE()), -1, 'DAY')) ".
The source tables have around 100 million rows per day. So I need to extract the data and load into another table with some additional calculation. But am not able to fetch the complete data from Bigquery table using talend. It is throwing error.
Please let me know if you find anything.
Thanks & Regards
Muthuraj
One Star

Re: Bigquery Table Handling Error

I am getting the following JSON exception while I am executing the query using the java application for Big-Query API. Following is the exception I am getting:
    {
   "error": {
   "errors": ,
"code": 403,
"message": "Access Denied: Job eco-span-847:job_LyHmZIvlY1_0J8JQ3pxThEBf19I: The user does not have permission to run a query in project eco-span-847"
}
}
One Star

Re: Bigquery Table Handling Error

Where can we set "allow-large-results" BQ parameter in tBigQueryInput component? Or how can we export a big table to csv (in GCS) without this issue?
Moderator

Re: Bigquery Table Handling Error

Hi,
Have you already tried it on latest released version 5.6.1 to see if this issue repro?
This issue "tBigQueryInput does not return all rows if result is too big (BigQuery input component)" has been fixed in 5.2.4, 5.3.2, 5.4.2, 5.6.0M0a.
https://jira.talendforge.org/browse/TDI-28445

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Bigquery Table Handling Error

HI All,
Could you please help me how did you resolve this issue. I am using version 5.6.2 but still I am facing the same issue. Where can I set the property Allow_large_results.
Thanks
Manoj