tBigQueryOuput ASCII issues

Six Stars

tBigQueryOuput ASCII issues

Morning,

 

I was wondering if anyone can help or advise me on the issue which I am having.

I have checked the forum and tried the solutions provided but have been unsuccessful.

My issue is that when I try to load data from a csv into tBigQuery i get the Error: Bad Character (ASCII 0)

The encoding setting is UTF_8, however, I have tried it with "custom" - "windows 1252" as it did not work with UTF-8 and also with IOS

The csv file data type has been classed as string and then converted using tConvert to ensure the value pass through as string then converted, along with changing the advance setting of encoding and unfortunately still been unsuccessful.

 

Below is the errior message which i receive

Exception in component tBigQueryOutput_1_tBQBE (jo_GCP_ORWHCART_CSV2BQ)

java.lang.Exception: [{"location":"gs://jdis-dl-ods-csv-dev-temp/orwhcart_bulk.txt","message":"Error while reading data, error message: CSV table encountered too many errors, giving up. Rows: 234468; errors: 1. Please look into the errors[] collection for more details.","reason":"invalid"}, {"location":"gs://jdis-dl-ods-csv-dev-temp/orwhcart_bulk.txt","message":"Error while reading data, error message: Error detected while parsing row starting at position: 38484895. Error: Bad character (ASCII 0) encountered.","reason":"invalid"}, {"message":"You are loading data without specifying data format, data will be treated as CSV format by default. If this is not what you mean, please specify data format by --source_format.","reason":"invalid"}]

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.tFileInputDelimited_1Process(jo_GCP_ORWHCART_CSV2BQ.java:6854)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.tJava_2Process(jo_GCP_ORWHCART_CSV2BQ.java:3395)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.tGSList_3Process(jo_GCP_ORWHCART_CSV2BQ.java:3108)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.tGSList_1Process(jo_GCP_ORWHCART_CSV2BQ.java:2616)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.runJobInTOS(jo_GCP_ORWHCART_CSV2BQ.java:14955)

      at jdis_gcp.jo_gcp_orwhcart_csv2bq_0_1.jo_GCP_ORWHCART_CSV2BQ.main(jo_GCP_ORWHCART_CSV2BQ.java:14631)

ORA-12899: value too large for column "TAL_AMC_V7"."TLOGCATCHER"."MESSAGE" (actual: 746, maximum: 255)

 

The ACSII character which is causing the issue is the "NULL" (ASCII code 00)

 

I ma not sure how to overcome this issue.

 

Please any help or advise would be appreciated.

Thanks

 

Highlighted
Five Stars

Re: tBigQueryOuput ASCII issues

Try to change the Encoding to ISO-8859-15. I have succeed using this method.
Please let me know if the issue still exist.

 

encoding.PNG

Six Stars

Re: tBigQueryOuput ASCII issues

@sinuraterickh 

 

Thank you for your respond.

 

I have tried IOS-8859-15 encoding and no look.

I have tried the available encoding and not been successful as yet.

Six Stars

Re: tBigQueryOuput ASCII issues

@nikhilthampi@xdshi

 

Have you come across the below issue?

Tags (4)

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now