Handling special characters(French letters) in csv to load into DB2 as such

Eight Stars

Handling special characters(French letters) in csv to load into DB2 as such

Hi everyone,

 

I have been struggling to find a solution for this. So I have a csv file in UTF-8 encoding with French letters (ô,â,à etc) that is getting replaced as

bientôt replaced as bientÃŽt

été replaced as Ã©té

and even ' is replaced by â€

But this is not what is expected in the output(DB2). I want these characters to be written as such.

Can someone please suggest me a way to do this?

 

Thanks in advance


Accepted Solutions
Highlighted
Employee

Re: Handling special characters(French letters) in csv to load into DB2 as such

Hi,

 

   I believe VARGRAPHIC will be the right target datatype but please consult your DB2 DBAs for details.

 

https://www.ibm.com/support/knowledgecenter/en/SSEPEK_10.0.0/sqlref/src/tpc/db2z_bif_vargraphic.html

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)


All Replies
Employee

Re: Handling special characters(French letters) in csv to load into DB2 as such

Hi,

 

    After reading from input file, could you please print the data and see whether Talend is able to print it in correct format?

 

    Since you have already selected the language set as UTF-8, it should print properly.

 

    So the next step is to verify the target columns in DB2 whether it can handle the letters other than English. Could you please share these details with relevant screenshots for further analysis?

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

Eight Stars

Re: Handling special characters(French letters) in csv to load into DB2 as such

Hi @nikhilthampi,

 

Thanks for your reply.

When I try to print the csv  data in tlogfile, it print all the characters as required. 

But while loading into db2, with UTF-8 as encoding in advanced settings in tFileinputDelimited component , all characters are fine now except ô.

bientôt is being printed as bientt now.

Please let me know what screenshots are needed from my side

 

 

 

Employee

Re: Handling special characters(French letters) in csv to load into DB2 as such

@nivedhitha,

 

    Since you are able to print the data correctly, Talend is able to read and transmit UNICODE in right manner.

 

    But when Talend is trying to insert the data to DB2, the current setup in DB2 database seems to not allowing to insert the characters in right format.

 

    Could you please refer this issue to your DB2 DBAs and they will help into the internationalization part for the database and tables?

 

I have added the IBM Guide for the same as a starting point.

 

https://www.ibm.com/support/knowledgecenter/SSEPEK/pdf/db2z_10_charbook.pdf

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

 

Highlighted
Employee

Re: Handling special characters(French letters) in csv to load into DB2 as such

Hi,

 

   I believe VARGRAPHIC will be the right target datatype but please consult your DB2 DBAs for details.

 

https://www.ibm.com/support/knowledgecenter/en/SSEPEK_10.0.0/sqlref/src/tpc/db2z_bif_vargraphic.html

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

Eight Stars

Re: Handling special characters(French letters) in csv to load into DB2 as such

Thanks @nikhilthampi for those suggestion and your quick response.

 

Eight Stars

Re: Handling special characters(French letters) in csv to load into DB2 as such

Hi @nikhilthampi,

 

Im now trying to load the data into vargraphic columns in db2 just to see if it would work but couldn't figure out how to cast the string columns in csv to vargraphic or graphic to be able to load into db2.

 

Can you please provide me a solution for this?

 

Thanks !!

Employee

Re: Handling special characters(French letters) in csv to load into DB2 as such

Hi,

 

    Could you please try to change the database data type as below in the tDB2Output Schema? I could not verify it personally as I do not have a DB2 database handy with me.

 

image.png

 

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

Eight Stars

Re: Handling special characters(French letters) in csv to load into DB2 as such

Hi @nikhilthampi,

 

Thanks for getting back.

I solved it by loading it as string but by increasing the bytes I had alloted for each column. Don't understand how it solved my problem but it did.

 

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now