Collation for processing? MySQL output results not what expected.

One Star

Collation for processing? MySQL output results not what expected.

Hi All,

New to the forums but not to the product so thanks for the great work so far. I'm running into an issue with Talend on a WindowsXP box. Essentially I have a straight forward job that takes a csv file, uses the tMap component, and then outputs into a table within mysql. One of the csv fields contains a 'name' entry. What I've noticed is that rows with an entry in the name column that contain the é aren't carried over properly into the database. The database is MySQL 5.1 and I'm running InnoDB with utf8 and the correction collation set. My web app that uses the database will gladly throw in non-ascii values. I've set the encoding on the csv file to utf8 and on the database but I'm still getting the bad results. Anybody have any idea what may be causing this? Just for giggles I tied appending some paramters to the end of my JDBC connection string but with no results. Thanks for the help. BTW, this is what I see in the database: Caf
cheers,
Andreas
Highlighted

Re: Collation for processing? MySQL output results not what expected.

You should make sure that you've set all of the encoding types to UTF-8 in your job (input and output)
If you've already done this and are still unsuccessful, maybe you've encountered a bug with mysql 5.1
I ingest non-ascii data with no issues on mysql 5.0.
One Star

Re: Collation for processing? MySQL output results not what expected.

Well, looks like if I change the encoding on the source file to ISO-8859-15 all works as expected. Strange as I thought this encoding was compatible with UTF-8 given that the ISO-8859 code points 'fit' into UTF-8. If anybody who knows more about encoding schemas wants to chime in and correct me that would be great. My knowledge in this area is fairly limited.
thanks,
Andreas

Calling Talend Open Studio Users

The first 100 community members completing the Open Studio survey win a $10 gift voucher.

Start the survey

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now