merge files using unix command in tSystem is not working ?

Highlighted
Five Stars

merge files using unix command in tSystem is not working ?

Hi,

 

I try to merge two csv files line by line using the unix command

 

paste -d ',' file1.csv file2.csv > file3.csv

 

But, I always get the error 

paste: >: No such file or directory

 

But, the files are present in the current directory, when I display the files using "ls -ltr" in another tSystem component, I am able to see the files file1.csv and file2.csv

 

the file names are derived using tjava component and the command is stored in context variable. and I try to execute the command using tSystem component. 

 

Anyone help to resolve this issue ?

Community Manager

Re: merge files using unix command in tSystem is not working ?

Can you show us your configuration of the tSystem (a screenshot), the configuration of the code were you derive the names (in the tJava?) and a screenshot of the job? There are a number of things that may have gone wrong here and that information should give us enough info to help you fix it....or enough info to know to ask for more :-)

Five Stars

Re: merge files using unix command in tSystem is not working ?

Even If I hardcode the command then also it is not working

 

"paste -d ',' out1.csv out2.csv > new.csv"

 

This is the command I use inside the tSystem component. 

And the files out1.csv and out2.csv are there in the current directory. That I can see when I list the files using "ls -ltr" command.

Five Stars

Re: merge files using unix command in tSystem is not working ?

Hi all,

 

Found a solution for this one as well. The tSystem component is designed to run the windows commands and not for unix.

 

We will have use the below string array in order for this to work.

 

new String[]{"/bin/bash","-c","" +context.merge_command}

 

The actual unix command to merge the files is present in the context variable context.merge_command. That is "paste -d ',' out1.csv out2.csv > new.csv"

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now