Processing output of "tSystem"

One Star

Processing output of "tSystem"

I often have to do a query at the command line and processing the result.
Actual I do the following steps:
1. Delete possible existing temp-files
2. Execute the command via tSystem and pipe the output into a temporary file
3. Open the file with TFileInput* and handle the data
4. Delete the tempFile
Ist there better solution for this? I saw that it is possible to fetch the output of tSystem to a global variable. Can I use this variable as Input for TFileInput*?
Alternatively is it possible to create from the global variable a data stream (convert the (string?)-variable into a hash/list(?) with tPerl for example)?
Tags (1)

Re: Processing output of "tSystem"

You current solution (reading the produced file with tFileInputDelimited) is fine in my opinion. But as you suggest, there are many ways to do it.
Can I use this variable as Input for TFileInput*?

No, you can't.
Alternatively is it possible to create from the global variable a data stream (convert the (string?)-variable into a hash/list(?) with tPerl for example)?

Yes, I suppose you can create a list from the tSystem output variable. It depends a lot on your Perl skill level and the output variable format.
To read the list, I use the tArrayIn invisible component (make it visible by checking the option in the TOS preferences). My tPerl contains:
our @lines = (

My job output is:
Starting job TEMP7 at 09:49 10/07/2007.
| tLogRow_1 |
| id | username | age |
| 1 | plegall | 26 |
| 2 | fbonan | 36 |
| 3 | jgottero | 21 |
Job TEMP7 ended at 09:49 10/07/2007.
One Star

Re: Processing output of "tSystem"

Is this component (tArrayIn) available in 2.1.0RC1? I cant find any option to activate ist.
I found a different solution myself (thanks for the "perl-tip"): I tried to pipe the output in the open command like the example shown in the picture below.
But it seems that the generated code adds the fixed mode "<" to the open command.
open($input_FH_tFileInputDelimited_1, '<', $desc_tFileInputDelimited_1{filename})

Is this necessary? Or could it be changed to a option (with mapping list in the gui)?
One Star

Re: Processing output of "tSystem"

Hi plegall,
I found the option for showing the hidden components in Talend 2.1.0 (without RC). I'll try it next time. Are there any special things to know If I use them? Like lower stability, possibly not available in the next release, and so on?
Concerning my suggestion to allow to parse the output of a system command / program. Should I open a feauture request? Additional it may be possible to take the logic to read a file (or system command) into a independent module and connect them. But this may be a bigger change to Talend.
Example: (tFileInput | tCommandExecute | ... ) -> (tInputXml | tInputPositional | ...)
With this solution I could solve another problem: Sometimes I have to change files before I could use them. For example make same changes to well form a file to use it with tFileXmlInput. Actual I read it with tFileInputPositional, transform it with tPerlRow and write it back with tFileOutputPositional into a temporary file. This file is pocessed by tFileInputXml.
Altogether creating, writing, deleting temporary files is one of my main tasks. Maybe creating a module to handle them, could be another way...

Cloud Free Trial

Try Talend Cloud free for 30 days.


Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.