Five Stars MZ
Five Stars

(SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Hi, i have this problem with tMysqlOutput.

tMysqlOutput_3 Data truncation: Data too long for column 'APPLICATION' at row 85

I have set these additional JDBC parameters:

noDatetimeStringSync=true&useUnicode=true&characterEncoding=utf8

and the problem is resolved in the Studio but on server continues to persist.

2017-07-26 11_27_11-Administrator Center.png

1 ACCEPTED SOLUTION

Accepted Solutions
Five Stars MZ
Five Stars

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Hi all, with the support team we realized that the cmdline is executed by default in 32-bit.

We modified the commandline.bat file to run the 64-bit cmdline.

 

Thank you.

Manuel

Robot wink

14 REPLIES
Five Stars MZ
Five Stars

Re: UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Update: I have try to execute the job with Studio on the Server virtual machine and run succesfully.

But the problem when executing in the Talend Administration Console persist.. Smiley Sad

 

2017-07-26 14_26_09-Administrator Center.png

Error:

[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Connection attempts to '172.16.72.245' with the user : 'AOSIS'.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Connection to '172.16.72.245' with the user : 'AOSIS' has succeeded.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Attempt to get bapi function : 'ZDW_MATERIAL'.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Success to get bapi function : 'ZDW_MATERIAL'.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Execute the bapi function : 'ZDW_MATERIAL' has finished.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Closing the connection.
[INFO ]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tSAPBapi_2_TSAPBapi_OUT - Connection has closed.
[FATAL]: cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR - tMysqlOutput_3 Data truncation: Data too long for column 'APPLICATION' at row 85
[statistics] disconnected
Exception in component tMysqlOutput_3
java.sql.BatchUpdateException: Data truncation: Data too long for column 'APPLICATION' at row 85
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
	at com.mysql.jdbc.Util.getInstance(Util.java:387)
	at com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1154)
	at com.mysql.jdbc.PreparedStatement.executeBatchedInserts(PreparedStatement.java:1582)
	at com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1248)
	at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:959)
	at cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR.tFixedFlowInput_1Process(DWH_1_IMP_2_SAP_08_MATERIAL_FR.java:7676)
	at cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR.runJobInTOS(DWH_1_IMP_2_SAP_08_MATERIAL_FR.java:9426)
	at cas_dwh.dwh_1_imp_2_sap_08_material_fr_0_1.DWH_1_IMP_2_SAP_08_MATERIAL_FR.main(DWH_1_IMP_2_SAP_08_MATERIAL_FR.java:9251)
Moderator

Re: UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Hi,

 What's the locale encoding of your TAC server? If the project starts with the -Dfile.encoding=UTF-8 set, try to change the locale encoding to UTF-8.

Let us know if it is OK with you.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars MZ
Five Stars

Re: UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Hi, the studio configuration file is:

 

-vm
C:\Program Files\Java\jdk1.8.0_121\bin\javaw.exe
-vmargs
-Xms1024m
-Xmx2048m
-XX:MaxPermSize=4096m
-Dfile.encoding=UTF-8

 

Where I can find this settings in TAC server?

 

Thanks, Manuel

 

Moderator

Re: UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Hello,

Please add -Dfile.encoding=utf-8 parameter ,in commandLine start script file
start_cmdline.bat like:

cd /D "D:\Products\6.4\cmdline\studio" 
java -Xms64m -Xmx1024m -XX:MaxPermSize=256m -Dfile.encoding=utf-8 -jar plugins/org.eclipse.equinox.launcher_1.1.0.v20100507.jar -nosplash -application org.talend.commandline.CommandLine -consoleLog -data commandline-workspace startServer -p 8002 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars MZ
Five Stars

Re: UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Hi, it's already written in <JETL_621>\cmdline\start_cmdline.bat

 

java -Xms256m -Xmx12288m -Dfile.encoding=utf-8 -jar plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar --disableShellInput -nosplash -application org.talend.commandline.CommandLine -consoleLog -data commandline-workspace startServer -p 8002
Employee

Re: UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

It should be on the Job.  In TAC, in the Job, you should set the same parameters.  You can also set JVM parameters on the job in TAC.  There is a section to set JVM parameters.  Make sure also that the individual components in the job is set to UTF8.

Five Stars MZ
Five Stars

Re: UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

I've tried this way too, but the result is the same:

 

2017-07-27 11_35_16-Administrator Center.png

It's impossible!!! Smiley Sad

Employee

Re: UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

The TAC does not execute the job.  The TAC will send the job to the jobserver, and the jobserver agent will start a java.exe process with the JVM parameters provided.  Not sure why it is not working.  Can you provide the jobs?

Five Stars MZ
Five Stars

Re: UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)


iburtally wrote:

Can you provide the jobs?


It's complicated because the input is a SAP call.

But I have add a tLogRow and I attached the studio log and the tac log.

 

2017-07-27 14_00_02-TIBCO Jaspersoft ETL (6.2.1.20160704_1411) _ admin, admin _ CAS_DWH (Connessione.png

 

You can see the differences, example:

 

Studio:

|303  |TR prel. a div.     |TF rem.fm.stor.to pl|UL Auslagern an Werk|TR Sortie -> div.   |

TAC:

303|TR prel. a div.|TF rem.fm.stor.to pl|UL Auslagern an Werk|TR Sortie -&#62; div.

The database column is a VARCHAR(20).

 

Could the error be due to the sap component as well as the mysql component?

 

Thanks for your help, Manuel

Five Stars MZ
Five Stars

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)


MZ wrote:

Could the error be due to the sap component as well as the mysql component?


OMG Man Frustrated

 

Ok the problem is on the sap component because also eliminating the tMysqlOutput component the encoding is wrong on TAC.

 

Studio:

|303  |TR prel. a div.     |TF rem.fm.stor.to pl|UL Auslagern an Werk|TR Sortie -> div.   |

TAC:

303|TR prel. a div.|TF rem.fm.stor.to pl|UL Auslagern an Werk|TR Sortie -&#62; div.

What happens? Man Frustrated

Tags (1)
Moderator

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Hello,

With your subscription solution, would you mind creating a case on talend support portal so that we can give you a remote assistance(webex) on your issue through support cycle with priority.

Please log on to the Support portal https://login.talend.com/support-login.php

Many thanks.

Best regards

Sabrina

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Employee

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

I think that this Jira may be related: https://jira.talendforge.org/browse/TDI-35499

 

It seems to have been corrected in Studio from 6.2.1, but it's possible we may not have patched the Job Server. We will look into it.

 

Thanks,

Arthur

Five Stars MZ
Five Stars

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)

Hi all, with the support team we realized that the cmdline is executed by default in 32-bit.

We modified the commandline.bat file to run the 64-bit cmdline.

 

Thank you.

Manuel

Robot wink

Moderator

Re: (SAP?) UTF-8 Encoding problem only on Server Console not in Studio (Data too long error)


MZ wrote:

Hi all, with the support team we realized that the cmdline is executed by default in 32-bit.

We modified the commandline.bat file to run the 64-bit cmdline.

 

Thank you.

Manuel


Hello Manuel,

Thanks for your feedback and sharing the solution here.

Best regards

Sabrina

T

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.