Data Profiling Report Generation Failure

Highlighted
Four Stars

Data Profiling Report Generation Failure

Hi,
I performed a Columnar analysis and then I am trying to generate the "Basic" report. Being new to Talend DQ, some of the reports are generated successfully. But for few, I am getting an error message and the report generation fails. The error message that I am getting is "Error during historization of data of report <Report_Name>: could not insert : [org.talend.dataprofiler.datamart.hibernate.TdqValues].

The error message that I am seeing in the Error Log is as under:
org.hibernate.exception.GenericJDBCException: could not insert: [org.talend.dataprofiler.datamart.hibernate.TdqValues]
at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:103)
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:91)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2202)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2595)
at org.hibernate.action.EntityInsertAction.execute(EntityInsertAction.java:51)
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:248)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:232)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:139)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:338)
at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106)
at org.talend.dq.persistence.DatabasePersistence.persist(DatabasePersistence.java:156)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.historizeData(ReportDocGenerator.java:505)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.generate(ReportDocGenerator.java:275)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.generate(LocalReportDocGenerator.java:129)
at org.talend.dataprofiler.core.tdq.ui.action.GenerateReportFileAction$2.run(GenerateReportFileAction.java:342)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: java.sql.SQLException: ORA-12899: value too large for column "DQ_PORTAL"."TDQ_VALUES"."VAL_STRING" (actual: 500, maximum: 255)

at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1044)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3665)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1352)
at org.hibernate.jdbc.NonBatchingBatcher.addToBatch(NonBatchingBatcher.java:23)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2182)
... 16 more

As per my understanding this is because of space. So should I contact with the Admin Team to have the space increased for TDQVALUES tables? Can the DBA's change this size without any impact?

Your help in this regards is highly appreciated.
Moderator

Re: Data Profiling Report Generation Failure

Hi,
Could you please indicate what's the build version you are using? Is there any special characters in your column? Screenshots of data profiling setting will be preferred.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Data Profiling Report Generation Failure

Hi,
I am using Talend Big Data Platform (6.1.1.20151214_1327). There are Chinese characters in columns. We are performing Simple, Advanced, Pattern Frequency.
Moderator

Re: Data Profiling Report Generation Failure

Hi,
This issue is probably caused by Charset(Some special characters get 2 bytes in UTF-8, while they get only one byte in other charset.)
Did you set file delimilted metadata for your column? If so, what does file setting look like? Are you using UFT-8 Encoding?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Data Profiling Report Generation Failure

We are using Oracle Database as our metadata connection. 
Moderator

Re: Data Profiling Report Generation Failure

Hi,
Have you tried to  modify the length of this column in DB side to see if it works?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Data Profiling Report Generation Failure

Modify to what length? 500 as being suggested in the error message?
Four Stars

Re: Data Profiling Report Generation Failure

Hi,

I modified the field length of the column TDQ_VALUES.VAL_STRING to 500 and the report generation was successful for most of the analysis. But it failed for one of the reports with the below error message in the Error_Log:

java.lang.StringIndexOutOfBoundsException: String index out of range: -3
at java.lang.String.substring(Unknown Source)
at org.jfree.text.G2TextMeasurer.getStringWidth(G2TextMeasurer.java:80)
at org.jfree.text.TextUtilities.nextLineBreak(TextUtilities.java:294)
at org.jfree.text.TextUtilities.createTextBlock(TextUtilities.java:235)
at org.jfree.chart.axis.CategoryAxis.createLabel(CategoryAxis.java:1140)
at org.jfree.chart.axis.CategoryAxis.refreshTicks(CategoryAxis.java:1106)
at org.jfree.chart.axis.CategoryAxis.reserveSpace(CategoryAxis.java:823)
at org.jfree.chart.plot.CategoryPlot.calculateDomainAxisSpace(CategoryPlot.java:3144)
at org.jfree.chart.plot.CategoryPlot.calculateAxisSpace(CategoryPlot.java:3211)
at org.jfree.chart.plot.CategoryPlot.draw(CategoryPlot.java:3257)
at org.jfree.chart.JFreeChart.draw(JFreeChart.java:1230)
at org.jfree.chart.JFreeChart.draw(JFreeChart.java:1114)
at net.sf.jasperreports.charts.util.DrawChartRenderer.render(DrawChartRenderer.java:62)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportImage(JRPdfExporter.java:1505)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportElements(JRPdfExporter.java:854)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportPage(JRPdfExporter.java:818)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportReportToStream(JRPdfExporter.java:695)
at net.sf.jasperreports.engine.export.JRPdfExporter.exportReport(JRPdfExporter.java:417)
at org.talend.dataquality.reporting.JasperReportBuilder.run(JasperReportBuilder.java:300)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.generateReport(ReportDocGenerator.java:679)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.generateDoc(ReportDocGenerator.java:413)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.generateDocInternal(LocalReportDocGenerator.java:175)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.access$0(LocalReportDocGenerator.java:174)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator$1.run(LocalReportDocGenerator.java:163)
at org.talend.repository.RepositoryWorkUnit.executeRun(RepositoryWorkUnit.java:99)
at org.talend.core.repository.model.AbstractRepositoryFactory.executeRepositoryWorkUnit(AbstractRepositoryFactory.java:258)
at org.talend.repository.localprovider.model.LocalRepositoryFactory.executeRepositoryWorkUnit(LocalRepositoryFactory.java:3217)
at org.talend.core.repository.model.ProxyRepositoryFactory.executeRepositoryWorkUnit(ProxyRepositoryFactory.java:2024)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.generateDoc(LocalReportDocGenerator.java:168)
at org.talend.dataquality.reporting.engine.ReportDocGenerator.generate(ReportDocGenerator.java:293)
at org.talend.dataprofiler.core.tdq.action.LocalReportDocGenerator.generate(LocalReportDocGenerator.java:129)
at org.talend.dataprofiler.core.tdq.ui.action.GenerateReportFileAction$2.run(GenerateReportFileAction.java:342)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Moderator

Re: Data Profiling Report Generation Failure

Hi,
For your Talend Big Data Platform product, could you please report a ticket on talend support portal so that we can give you a remote assistance on your issue through support cycle with priority?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Enabling Data Governance

Learn how to enable Data Governance

Watch Now

The Definitive Guide to Government Data Quality

Take a peek at the definitive guide to Government Data Quality

Read