One Star

[resolved] Execute python script using tSystem

Hello,
I'm trying to execute a python script in Talend.
I just put a tSystem on my job and fill the 'command' field by : "cmd /c C:/Python24/python.exe Q:/Python/Test/myScript.py"
The script is supposed to read files, create 2 others one. All the files the script uses (or writes) are in the same folder than the script. Outside Talend, the script works well.
But when I run the job, my script doesn't work. I have the 'prints' on the console inside Talend, but the files are not detected and the others are not written.
Any idea to help me ?
Thank you !
5 REPLIES
One Star dbh
One Star

Re: [resolved] Execute python script using tSystem

Can can you share your Python script? 
It may be writing the output to the Talend job's current directory rather than where you expect it to be . 
It might also not be reading the File based on the way the parameters are passed to the CMD
One Star

Re: [resolved] Execute python script using tSystem

Hi,
Thank you for your answer.
Sadly, I can't share the script.
But here is the structure :
from glob import glob
for file in glob("*.csv"):
infile = open( file, 'r' )
lines = infile.readlines()
infile.close()
for line in lines:
// read the data
// operations on data, result stored in a list

outfile = open( outputFileName , 'w' )
for outputLine in list:
outfile.write(outputLine)
outfile.close()

I checked the Talend job's current directory, nothing inside.
But actually you're right, none of the files are read.
One Star dbh
One Star

Re: [resolved] Execute python script using tSystem

The script appears to make an assumption that the CSV files are in the current directory, based on the glob statement . For an easy mod, just specify an absolute path to glob ex
glob("/data/20160229/*.csv")
 You could also add some debugging output lines  For help in troubleshoouting 
 It's files might sometimes be in a different folder, then you can look at ways to perm parameterize The script or add a configuration file
One Star dbh
One Star

Re: [resolved] Execute python script using tSystem

 Of course,  I should ask… 
Why not use Talend components for your task? It is very easy to list files , read CSV files, transform, and write the files out in a different format
One Star

Re: [resolved] Execute python script using tSystem

I specified the absolute path to glob, and it seems it's working !
Thank you dbh !
To answer you, several things :

I've already written the python script
I'm a beginner in Talend (but I have to work with jobs already built)
The script is looking in different ways in the data already processed (comparison, eliminate duplications, insertions...)
It would be easier to upgrade the script (adding more error types/messages for instance) than modify Talend
But, if I hadn't spent hours on the script before, I certainly would have use Talend components.
Thanks again !