Apply logic to all fields at once?

Five Stars

Apply logic to all fields at once?

Hello everyone,

 

Is there a way to apply some logic to all fields at once?

Like if I have to apply LTRIM to all 200 incoming fields then instead of doing this at individual field level i.e 200 times, is there any not so laborious way to achieve this?

Forteen Stars

Re: Apply logic to all fields at once?

You may need to call manully ,untill unless you call the specific funaction,it will not do the required operation.
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Five Stars

Re: Apply logic to all fields at once?

Thanks Manohar, could you please elaborate?

Eight Stars

Re: Apply logic to all fields at once?

We can reduce our manual intervention by some  tricky way (below is my thought process but there could be some other easy also)

 

 

Create a routine with ltrim functionality with null handling scenario .

 

your input ---> tjava row (sync the columns , generate the code ) and with the help of text editor 's find and replace option apply the newly created routine .. 

 

 

Sample Routine:

 

package routines;

public class ltrimr {
public static String ltrim(String datain) {
String dataout="";
if (datain == null) {
dataout= datain;
}
else if(datain!=null)
{

dataout= datain.replaceAll("^\\s+","");
}
return dataout;

}
}

 

 

tjava row logic : 

 

output_row.newColumn = ltrimr.ltrim(input_row.newColumn);
output_row.newColumn1 = ltrimr.ltrim(input_row.newColumn1);
output_row.newColumn2 = ltrimr.ltrim(input_row.newColumn2);

 

Five Stars

Re: Apply logic to all fields at once?

Thanks @ashif, that's an interesting approach when used with a text editor. I am also thinking of using a database function on each field at SQL level, again a text editor will save a lot time here but am not sure how it will affect the performance.

 

SELECT

A,B,C,D.....

FROM TABLE;

 

REPLACE "," with "),LTRIM(" will do 99% of the job in a jiffy!

 

SELECT
A),LTRIM(B),LTRIM(C),LTRIM(D.....
FROM TABLE;

 

But again, this is handy and efficient only in case of simple operations like LTRIM

Eight Stars

Re: Apply logic to all fields at once?

it could be better , if you explain the scenario with complete information like nature of source (whether it is relational,structured files, semi- structured files ).  Nothing has been specified so community people will try to give you a generic solution (solution at talend level).  Smiley Happy

Five Stars

Re: Apply logic to all fields at once?

yeah.. well the source is MSSQL. Table has 200 fields and the issue is that any string field can have "\n" as a hidden character. This causes a row in target file to break into multiple rows. To overcome this, I have to replace "\n" with "". The problem is that "\n" can appear in any field so the logic needs to applied on all fields.

Highlighted
Eight Stars

Re: Apply logic to all fields at once?

Create a routine with  logic replaceAll("\\r\\n|\\r|\\n", " ") ( include null handling logic) and call the routine in tJavaRow.  Its handy because if any other new characters needs to be handled which can be achieved by simple change and routine and regeneration of code-build

Five Stars

Re: Apply logic to all fields at once?

@ashif I get your point, probably this is the easiest way so far. Thanks!

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog