Adding columns to output based on input schema/metadata

One Star mtm
One Star

Adding columns to output based on input schema/metadata

I would like to create a custom component that will output the input data as well as some extra columns that will be derived from the input columns. As a proof of concept, in the following code I'm trying to create a duplicate column for every column, but the real example will have more complicated logic. Here's what I tried in the "begin" portion of my .javajet file for the component:
<%@ jet
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();

IConnection conn = node.getOutgoingConnections().get(0);
IMetadataTable outputMetadataTable = conn.getMetadataTable();

List<IMetadataTable> metadatas = node.getMetadataList();
IMetadataTable metadata = metadatas.get(0);

List<IMetadataColumn> columnsToAdd = new java.util.ArrayList<IMetadataColumn>();
for(IMetadataColumn col : outputMetadataTable.getListColumns()){
IMetadataColumn cloned = col.clone();
cloned.setLabel("Clone_of_" + cloned.getLabel());

List<IMetadataColumn> cols = new java.util.ArrayList<IMetadataColumn>(outputMetadataTable.getListColumns());

This seems to almost work, but not quite. If add my component as part of a flow with a single input and single output, the output does indeed show the desired extra columns. However, the generated Java code nevertheless has compile errors. It looks like the "struct" for the output to my component does not have fields to represent the additional columns. That is, if row1 is the input and row2 is the output, the row2Struct class does not have the extra fields that are represented by the change to the metadata. So even though the UI will show me that those columns exist in the output, it seems I am not able to actually put any values into those columns.
Since I am able to change what columns are in the metadata, there must be a way for me to let this be reflected in the structs for the row, right? Thanks in advance for the help.
One Star

Re: Adding columns to output based on input schema/metadata

i liked to import my file excel to mysql workbench then when i clicked on Métadonnées file excel=> new file => download my file (.xls) and when i clicked next; the table that i selected can't overview i don't now what is the problem it makes me this error :
Warning: Cannot read name
Warning: Cannot read name
Warning: Cannot read name
Erreur dans l'aperçu. Des paramètres doivent être modifiés.
: Les erreurs d'aperçu sont généralement dûes à un mauvais encodage.

So what's the solution THANK YOU IN ADVANCE !
One Star mtm
One Star

Re: Adding columns to output based on input schema/metadata

It's been a few weeks now; does anyone have a response to this?
One Star

Re: Adding columns to output based on input schema/metadata

Have you tried to propagate values:
IConnection conn = node.getIncomingConnections().get(0);
IConnection oConn = node.getOutgoingConnections().get(0);
String outputConnectionName = oConn.getName();
<%=outputConnectionName %>.<%=column.getLabel() %>=<%=conn.getName() %>.<%=column.getLabel() %>;
<%=outputConnectionName %>.Clone_of_<%=column.getLabel() %>=<%=conn.getName() %>.<%=column.getLabel() %>;
One Star

Re: Adding columns to output based on input schema/metadata

i love such a rules

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach


Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.



Introduction to Talend Open Studio for Data Integration.