One Star

Collation for processing? MySQL output results not what expected.

Hi All,

New to the forums but not to the product so thanks for the great work so far. I'm running into an issue with Talend on a WindowsXP box. Essentially I have a straight forward job that takes a csv file, uses the tMap component, and then outputs into a table within mysql. One of the csv fields contains a 'name' entry. What I've noticed is that rows with an entry in the name column that contain the é aren't carried over properly into the database. The database is MySQL 5.1 and I'm running InnoDB with utf8 and the correction collation set. My web app that uses the database will gladly throw in non-ascii values. I've set the encoding on the csv file to utf8 and on the database but I'm still getting the bad results. Anybody have any idea what may be causing this? Just for giggles I tied appending some paramters to the end of my JDBC connection string but with no results. Thanks for the help. BTW, this is what I see in the database: Caf
cheers,
Andreas
2 REPLIES

Re: Collation for processing? MySQL output results not what expected.

You should make sure that you've set all of the encoding types to UTF-8 in your job (input and output)
If you've already done this and are still unsuccessful, maybe you've encountered a bug with mysql 5.1
I ingest non-ascii data with no issues on mysql 5.0.
One Star

Re: Collation for processing? MySQL output results not what expected.

Well, looks like if I change the encoding on the source file to ISO-8859-15 all works as expected. Strange as I thought this encoding was compatible with UTF-8 given that the ISO-8859 code points 'fit' into UTF-8. If anybody who knows more about encoding schemas wants to chime in and correct me that would be great. My knowledge in this area is fairly limited.
thanks,
Andreas