I am getting the following error if there are more than one tBigQueryInput's:
Detail Message: jobId_tBigQueryInput_2 cannot be resolved to a variable
This is my job and the configuration of tBigQueryInput_2. But the config of tBigQueryInput_1 is the same:
In the source code, I am getting the following for tBigQueryInput_1, which is correct:
com.google.cloud.bigquery.JobId jobId_tBigQueryInput_1 = com.google.cloud.bigquery.JobId .of(java.util.UUID.randomUUID().toString()); com.google.cloud.bigquery.Job job_tBigQueryInput_1 = bigquery_tBigQueryInput_1 .create(com.google.cloud.bigquery.JobInfo .newBuilder(queryConfiguration_tBigQueryInput_1) .setJobId(jobId_tBigQueryInput_1).build());
For tBigQueryInput_2, I am getting an error on the last line of the following code:
com.google.cloud.bigquery.JobId jobId_tBigQueryInput_1 = com.google.cloud.bigquery.JobId .of(java.util.UUID.randomUUID().toString()); com.google.cloud.bigquery.Job job_tBigQueryInput_2 = bigquery_tBigQueryInput_2 .create(com.google.cloud.bigquery.JobInfo .newBuilder(queryConfiguration_tBigQueryInput_2) .setJobId(jobId_tBigQueryInput_2).build());
The error is on the first line of the above code. Variable `jobId_tBigQueryInput_1` is beeing initialized, but on the last line it references `jobId_tBigQueryInput_2`, which does not exist.
It works for tBigQueryInput_1, but fails for any other tBigQueryInput's.
Talend Version used: 22.214.171.12481026_1147
The same job has worked on Taled Version 126.96.36.1998.0411_1414
I think it is because of the Group By clause. Since, in the query no aggregation is involved to support a group by. Could you please check and confirm?
Thank dipanjan93 for the reply. But I do not see how that can solve the problem. The job does not compile. So the query is never evaluated.
IMO it is an error in how Talend creates the code for tBigQueryInput processors.
Anyway, I tried your suggestion by simply removing the GROUP BY clause in both queries. The problem persists.
I've created a bug report: https://jira.talendforge.org/browse/TBD-8265
Thank you for the reply dipanjan93. I tried removing the GROUP BY clause in the queries. It did not solve the problem.
But the problem is, that the job does not even build. I have the error in the Code view of the job, so the queries are not evaluated or executed.
I created a bug report: https://jira.talendforge.org/browse/TBD-8265
(my first response has vanished. No idea what happend)
It seems to be having troubles with the jobId for the second and additional tBigQueryInput components.
It is a bug on V 7.1.1 and our developpers are working on this issue. It could be fixed on V 7.2.1.
We will keep you posted.
I have just encountered the exact same issue with my job also - and spotted the same issue with the generated java code re-intantiating the previously defined
jobId_tBigQueryInput_1 instead of jobId_tBigQueryInput_2.
What is the workaround for this whilst we wait for the bugfix?
I have encountered the same issue where in a tExtractxml component its trying to cast datatypes which has not been coded to cast. Like casting a decimal to Date and Date to a Big Decimal.
It is a bug and the fix versions are 7.1.2 and 7.2.1. Could you please clarify in which Talend version/edition you are?
Thanks for this information, I am on version 7.1.1.
also I raised this issue, if you can confirm if this is known issue.
we are in phase of evaluating the ETL as Talend. These are important tests and we are using BQ as our source/target.
thanks in advance
We need more information about your another issue:https://community.talend.com/t5/Design-and-Development/Issue-when-Using-2-tBigQueryOutput-in-1-job/m....
I have also this bug in 7.1.
Can I give you the details you need?
Is there any way to have th bug's link so we can now whats the resolution state?
Here exists a jira issue: TBD-8102 "
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Learn how to make your data more available, reduce costs and cut your build time
Read about OTTO's experiences with Big Data and Personalized Experiences
Take a look at this video about Talend Integration with Databricks