[resolved] problem with tMysqlOutput and Encoding

Highlighted
One Star

[resolved] problem with tMysqlOutput and Encoding

Firstly, I would like to say hello. I am new to the forums and new to Talend.
I have been encountering a problem, and I am unsure if something is wrong or if I have missed a step.
I need export data from Excel to MySQL. The part data contain Russian characters.
Tracert in inside TOS - all look well.
Result in table of MySql is wrong.
Does anyone know what the problem is?

Accepted Solutions
One Star

Re: [resolved] problem with tMysqlOutput and Encoding

problem solving - need pass the jdbc parameter to force utf8 encoding "noDatetimeStringSync=true&characterEncoding=utf8"

All Replies
One Star

Re: [resolved] problem with tMysqlOutput and Encoding

problem solving - need pass the jdbc parameter to force utf8 encoding "noDatetimeStringSync=true&characterEncoding=utf8"
One Star

Re: [resolved] problem with tMysqlOutput and Encoding

Vik,
I tried your solution - it does not work for me. I have the same set as yours: data flows from Excel into MySQL table.
I have slight difference - there are couple other components in between.
tFileInputExcel --> tFilterRow --> tMap -->tMysqlOutput
Encoding = UTF8 is set on tFileInputExcel and tMysqlOutput components. I added your suggestion: "noDatetimeStringSync=true&characterEncoding=utf8"
and doesn't seem to work.
Is there anything else you did?
Thank you for your help!
Peter.
One Star

Re: [resolved] problem with tMysqlOutput and Encoding

Hi, Peter.
Sorry for my English.
In the beginning, I have many variants job. The result was identical - bad.
After parameter addition all was normalized.
When the parameter isn't present, Talend send to MySql string "SET NAMES latin1".
After parameter addition "characterEncoding=utf8" - Talend any more doesn't send a line "SET NAMES latin1"
I think that further, the coding established on the server is important.
In my case, I have next MySql serever parameters - DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci.

Vik
One Star

Re: [resolved] problem with tMysqlOutput and Encoding

Hi Once try with this.....this worked for me....
add "useOldUTF8Behavior=true" in JDBC additional parameter....
One Star

Re: [resolved] problem with tMysqlOutput and Encoding

Hi everyone, 
I am having an issue with my job.
I am sending the records to SharePoint using tSoap. My records are having some international characters (like Ö etc.)
When I am running the webservice job through Talend Open Studio I am not getting any error. But when I am running the same job from Windows Scheduler using services it is not supporting those international characters, I am getting below error :
 
SUPPLIER Webservice Job 2015-01-29 14:50:00 Starting Job
0
||<soapenv:Fault xmlns:soapenv=""><faultcode>soapenvSmiley Frustratederver</faultcode><faultstring>com.ctc.wstx.exc.WstxIOException: Invalid UTF-8 middle byte 0x54 (at char #319, byte #127)</faultstring><detail /></soapenv:Fault>
||<soapenv:Fault xmlns:soapenv=""><faultcode>soapenvSmiley Frustratederver</faultcode><faultstring>java.lang.ArrayIndexOutOfBoundsException: read(buf,-380,4380), cbuf</faultstring><detail /></soapenv:Fault>
||<soapenv:Fault xmlns:soapenv=""><faultcode>soapenvSmiley Frustratederver</faultcode><faultstring>com.ctc.wstx.exc.WstxIOException: Invalid UTF-8 middle byte 0x50 (at char #315, byte #127)</faultstring><detail /></soapenv:Fault>
||<soapenv:Fault xmlns:soapenv=""><faultcode>soapenvSmiley Frustratederver</faultcode><faultstring>java.lang.ArrayIndexOutOfBoundsException: read(buf,-380,4380), cbuf</faultstring><detail /></soapenv:Fault>
|<nsSmiley TongueutSUPPLIERResponse xmlns:ns=""><ns:return xmlns:xsi="" xsi:type="ns:Return_Status"><ns:Errors>Error while updating row 0 for DataElement with meta id 213051, Message: Cannot insert the value NULL into column 'PARMA_CODE', table 'PPSS_qa2.dbo.MR_SUPPLIER'; column does not allow nulls. INSERT fails., Error Code: 515, SQL: INSERT INTO MR_SUPPLIER (PARMA_CODE, PARMA_DESC, LAST_UPDATED_BY, LAST_UPDATE_DATE, CREATED_BY, CREATE_DATE, EMAIL, FAX_NUMBER, TELEPHONE_NUMBER, INBOUND_FREIGHT_GROUP, COUNTRY, ZIP_CODE, STATE, CITY, ADDRESS2, ADDRESS1, EXTRACTION_TIME) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)&#xD;
ColumnSmiley TongueARMA_CODE, Value:null&#xD;
ColumnSmiley TongueARMA_DESC, Value:null&#xD;
Column:LAST_UPDATED_BY, Value:null&#xD;
Column:LAST_UPDATE_DATE, Value:null&#xD;
Column:CREATED_BY, Value:null&#xD;
Column:CREATE_DATE, Value:null&#xD;
Column:EMAIL, Value:null&#xD;
Column:FAX_NUMBER, Value:null&#xD;
Column:TELEPHONE_NUMBER, Value:null&#xD;
Column:INBOUND_FREIGHT_GROUP, Value:null&#xD;
Column:COUNTRY, Value:null&#xD;
Column:ZIP_CODE, Value:null&#xD;
ColumnSmiley FrustratedTATE, Value:null&#xD;
Column:CITY, Value:null&#xD;
Column:ADDRESS2, Value:null&#xD;
Column:ADDRESS1, Value:1&#xD;
Column:EXTRACTION_TIME, Value:null</ns:Errors><ns:Num_Deletes>0</ns:Num_Deletes><ns:Num_Inserts>0</ns:Num_Inserts><ns:Num_Updates>0</ns:Num_Updates><ns:Record_Name>SUPPLIER</ns:Record_Name></ns:return></nsSmiley TongueutSUPPLIERResponse>|
SUPPLIER Webservice Job 2015-01-29 14:50:01 Ending Job
 
 
Do I need to check some setting or something to make it work.
 
Any suggestion would be appreciated, Thanks in advance.
 
Cheers,
Shilly
 
One Star

Re: [resolved] problem with tMysqlOutput and Encoding

Thanks, i was try a lot of things and this one work for me. Smiley Happy

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog