Four Stars

tMap output to tBigQuery validation error

I'm having issues trying to insert the output of a tMap component to a BigQuery table using the tBigQueryOutput component. It appears that the tBigQueryOutput component checks for every field passed by the tMap for null and this comes to an error when a field is of type int in my case. Is there any way to solve/modify this behavior?

 

Thank you.

  • Big Data
  • Data Integration
Tags (1)
7 REPLIES
Moderator

Re: tMap output to tBigQuery validation error

Hello,

Could you please also post your tMap editor screenshot into forum? Which will be helpful for us to understand your setting in tMap.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: tMap output to tBigQuery validation error

Currently I've solved the issue by changing the field type to String, but it's not what I'd like to leave as a solution since I may need to make some other modification in the ETL when receiving this output. I've attached the screenshots you've requested, take in consideration that I've highlighted the changes I've made in order for the job to work, all strings were int type fields.

Ten Stars

Re: tMap output to tBigQuery validation error

I'm not positive, but I think if you check the Nullable box on an int column in a schema, it will use an Integer object instead, which does allow checks for nulls.
Moderator

Re: tMap output to tBigQuery validation error

Hi,

With int, null is not allowed.

Schemas allow you to specify if a column is nullable. This usually determines if it is an int or an Integer. You need to make it nullable.

Best regards

Sabrina

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: tMap output to tBigQuery validation error

Ok, so if I understand correctly I either have to mark the field as nullable which is not something I actually want to allow or i have to assign a default value before the data gets into the tMap component? I'll check either approach and see which one is better for our task. Anyway, thank you all for your help, it's really appreciated.

 

Regards.

Ten Stars

Re: tMap output to tBigQuery validation error

Honestly, this seems like a bug. The code generator should know not to test if a primitive is null.
Four Stars

Re: tMap output to tBigQuery validation error

Yeah, that's what I thought as well, but given the circumstances I think I'll go by trying to feed the data to the tMap component with "null integers" as zero.