Six Stars

how to access directory situated on different server through TDF ?

Hi,

I have a job in TDF 6.3.1 which contains following design -PublishErrorDesign.jpg

 

So I am fetching parameters from tFileInputdelimited_1and store in Global variable and pushed it into tFileOutputRaw_1,to create a output file.

 

But I want to create this output file to the directory situated on another server,but i have not access to that server by my credentials .

 

That's why I uses the output file path with Username who has the access to that server like this

"file:\\\\Servername\\c$\\Talend\\6.3.1\\cmdline\\studio\\out.txt"

But I am unable to create or access that direcotry.

 

Can any one suggest me what i have missed? Suggest me the solution pls..

Any how I want to access that directory otherwise I will not able to call  commandline.bat  file reside on that Server through tSystem_1 component.

 

Kindly reply soon....!

 

Thanks,

Vaibhav

 

  • Data Integration
1 ACCEPTED SOLUTION

Accepted Solutions
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi,

Finally issue resolved .I build my local talend job and include home directory with the path as same as the path on remote server where commandline.bat file resides.

 

Building talend job and execute .bat file of that job on remote server after copying zip file of job on the server. This Works fine for me. I refer following topics- 1) http://etladvisors.com/2014/05/06/windows-deployment-and-scheduling-of-talend-jobs/

2)https://www.talendforge.org/forum/viewtopic.php?id=43435

 

Thanks,
Vaibhav

25 REPLIES
Seven Stars

Re: how to access directory situated on different server through TDF ?

where You want execute this commandline.bat file?

on Your machine or on remote?

-----------
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi,

thanks for your quick response..!

 

I want to execute .bat file through my machine but the problem is it is resides on different server. Commandline.bat file requires its all related files they are also resides on that server.

That's why I want to push my output file on that server and call it from Commandline.bat  file by specifying its name in command in Commandline.bat.

 

Thanks,

Vaibhav

Seven Stars

Re: how to access directory situated on different server through TDF ?

Still not 100% clean

 

1) if this .bat file could be executed from remote disk, but on local machine (did You test this?), You can:

- map network drive

- just make a local copy of files

 

2) But if as usual - dependencies more serious, like dll or other libraries - file will not work properly

in this case remote disk not help 

 

in any case - tSystem - call file for execute only on local machine with all local folders, so ev en if You put file on remote server and run bat file - it will look for Disk C: as local Disk C: on Your machine, but not on remote sever as You expect (same for all dependencies - it will search all path on Your machine)

 

Normal way, if You want call remote Job - install on remote server ssh and sftp services, in this case Talend will and upload file and call remote command 

 

 

-----------
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi Vap,

Thanks for your Prompt response....!

 

I mapped the network drive as you suggested,it  works for me.

Now I can Push my output file on that server ,but I am worried about tSystem component to call Commandline.bat file . 

 

I get Following Error in tSystem-

'\\Servername\c$\Talend\6.3.1\cmdline\studio'
CMD.EXE was started with the above path as the current directory.
UNC paths are not supported. Defaulting to Windows directory.
'commandline_test.bat' is not recognized as an internal or external command,
operable program or batch file.

 

 

I am very close to achieve my target if tSystem works fine for me. Is there any different way to do this?

Kindly reply to solve this query if possible Smiley Happy ....!

 

Thanks,

Vaibhav

Seven Stars

Re: how to access directory situated on different server through TDF ?

may be (but not sure) it could be done as You want ...

I can't suggest nothing, because always go by other way (from my point of view more proper)

 

I suggest on my projects (when windows) for clients - https://www.bitvise.com/ssh-server, or analogs from IBM (part of some product) or cygwin and etc

Some clients prefer native windows

 

just install one of this .. .and use standard ssh tools like - ssh, scp, sftp ... all what Talend need for do all what You need

 

 

 

-----------
Nine Stars

Re: how to access directory situated on different server through TDF ?

Windows is telling you the problem. You can't use a UNC path (i.e. paths in the form \\server\sharename\directory) when invoking CMD. If you mapped the share to a drive letter, refer to the drive letter instead.

Seven Stars

Re: how to access directory situated on different server through TDF ?


cterenzi wrote:

Windows is telling you the problem. You can't use a UNC path (i.e. paths in the form \\server\sharename\directory) when invoking CMD. If you mapped the share to a drive letter, refer to the drive letter instead.


I agree with You about UNC error (and when I mean map - I mean map as local drive T:\ for example), just personal opinion - this is wrong way

even if You map network drive and try for example run excel - it will not work

-----------
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi,

I get following Error while using tSSH component-

java.io.IOException: There was a problem while connecting <ServerName> 22

 

SSH_pic.jpg

 

I am confused while set up of SSH2 option in Network tab of Talend Studio-->Windows-->Preferences.

Is it mandatory to install Putty.exe on my local machine to connect remote server by SSH protocol?

 

Kindly reply..!

Seven Stars

Re: how to access directory situated on different server through TDF ?

don't touch Talend :-)

 

for work tSSH - You must have enabled SSH on target server, by default it not installed on windows server, and as well could be closed by firewall

-----------
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi ,

You mean to say I have to install Putty on My local machine so that ,I can connect to Remote server and run the .bat file there?

 

I have downloaded the putty setup successfully installed also but when click "open" gives "Network Error:Connection Refused" . I enter Hostname as my Remote Server Name.

But Tell me Is it required to install openSSH or Tectia SSH on my Remote server.

Even Remote server doesn't have SSHD service in services.msc.

 

Can you please help me?

 

Thanks...!

Nine Stars

Re: how to access directory situated on different server through TDF ?

The destination server has to be configured to accept connections of this type.

Would it not make more sense to set up this batch file on the host running the Talend job? Or else compile the Talend job and set it up to run on the host with the batch file?
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi ,

I want to run my talend job from my local machine but want trigger or call .bat file which is on Remote server.

 

We have tSSH component to do this task but I am unable to connect Remote server even after installing Putty in my local machine and freeSSHd on my Remote server.

I get status on my Remote server as follows-

freesshd_pic.jpg

 

But when I Double click on Putty in My Local machine,I get this error-

putty error.jpg

Even if it recognises my remote Hostname then also it gives above error.

Is there any extra settings I have to Make in freesshd or Putty ?

 

Kindly Suggest...!

 

Thanks,

Vaibhav

Seven Stars

Re: how to access directory situated on different server through TDF ?

@cterenzi - You are could be right, but some time we could have other restrictions - such as we have 2 different network(offices in same organisation) in process

we finish process in one office, upload file, then call remote job.

Of course - there are many other ways how to resolve same problem

 

@vaibhav - do not forget about windows firewall, it close all what not enabled by default

-----------
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi ,

I am not using freeSShd right now to get rid of "Access denied " error in when I enter my Username and after giving password ,it gives me access denied error.

Now I installed KpyM telnet ssh server setup on my Remote server to which I want to connect from my local machine.

 

Now I am able to login by my local machine using Putty,but problem with tSSh component in my talend job .I have provided all necessary info like Host,Username,Password etc. even job executed successfully, no error comes .

 

But changes which I have expected from talend job is not reflected . Don't know why this is happened ?

 

Reply with your valuable suggestions please...!

 

Six Stars

Re: how to access directory situated on different server through TDF ?

I am using following settings in my tSSh component-

 

tssh settings.jpg

kindly reply ...!

 

Seven Stars

Re: how to access directory situated on different server through TDF ?

You can first easy check - just login to ssh and run command

 

You do not need any CALL and exit, just run it as normal command (in tSSH the same - just a command), and see result

this is shh session to windows server:

Screen Shot 2017-06-07 at 10.57.05 PM.pngScreen Shot 2017-06-07 at 10.57.14 PM.png

-----------
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi Vap,

I m trying to run my commandline.bat file through putty like below

 

Idle time out.jpg

 

but before command executes ,it give pop up window like connection closed by remote host. Ideally on remote Server idle timeout is 5 min ,i explicitly set it as 0 i.e. no timeout. I also set keepalive timeout in Putty as 11 minutes. 

This will not serve my problem even though my changes are not reflected whatever I want .

There are two active sessions files appeared in my SSh server folder.

 

I also tried to run .bat file in tSSh but same happens, the job runs for so long and not closed then also I don't get the output that I expected.

 

Is anything I missed because even Event logger also shows I am able to login and excutes shell command as shown in above attached figure. 

 

Kindly reply...!

 

 

 

 

Seven Stars

Re: how to access directory situated on different server through TDF ?

Hi,

 

I don't know (and more important - do not want to know :-) ) all possible tools in a world

 

I use - https://www.bitvise.com/ssh-server, it work (of course by default it also timeout)

We have Jobs which run 3.5Hr month by month - stop and start new iteration, so ask Your sysadmin (or hire one) for setup it to You once properly, and then just use .

-----------
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi,

Yes of course no one is expert in all tools in the world Smiley Happy 

 I really appreciate your help ,and for our discussion of the current  topic ,now I am able to execute my .bat file by putty after some changes in .ini file on server.

 

Now I am trying to execute same .bat file through talend job by tSSh using Target execution as Local but same thing job doesn't response ,stuck somewhere don't know why but same credentials worked in putty .

It not even give error if there is problem in syntax only running continuously.

 

There is another way also in which we start putty from another batch file,by tSystem component but I want to execute this from tSSH if possible.

 

tssh_new.jpg

Does it belong to Port number ,bcz putty uses port number 22 and when I started Talend job It shows - [statistics] connecting to socket on port 3357.

 

Reply if its relevant for you...!

 

Thanks & Regards,

Vaibhav

 

Seven Stars

Re: how to access directory situated on different server through TDF ?

I can't understand - what command You try to use

 

make all simple 

all work, attached same .bat file as on previous example:

 

Screen Shot 2017-06-08 at 11.40.23 PM.pngScreen Shot 2017-06-08 at 11.40.29 PM.png

-----------
Six Stars

Re: how to access directory situated on different server through TDF ?

Sorry...!  I have attached wrong snapshot of tssh component.

I am just running tSSh component to print a message just for testing purpose that tSSh works or not  but I am not getting any result.

 

No action is performed when I run the job ,it simply running for long time for a printing a msg  and job is not terminates.

 

 

Six Stars

Re: how to access directory situated on different server through TDF ?

Hi ,
I am still facing the issue while executing .bat file on remote server by my local machine using tSSh componnent.

Is there any extra setting in tssh component? That I missed bcz I had checked log files of my Kpym Telnet SSh server, I am able to Login there with my provided credential like host name,Username etc.

But After that it hangs for long time .I am not
even able to print msg using echo command Smiley Sad

Kindly suggest if possible...!
Seven Stars

Re: how to access directory situated on different server through TDF ?

it is not real time process, all printed after

 

most of jobs oriented for run from scheduler - so it problem not actual for 99,999% of cases ... just forget

-----------
Six Stars

Re: how to access directory situated on different server through TDF ?

Hi,

Sorry for inconvenience ...! Just last time clear my doubt if possible.

I have changed my approach , I don't know why but in my case tSSh not working.

 

Hence I build my job and copy the zip File to my remote server .In that zip File I get my job's .bat file . I want to run following multiple commands by tSystem in Single command option

tSystem_1 code:new String[]{"cd C:\\Talend\\6.3.1\\cmdline\\studio","CALL commandline_test.bat"} 

PublishErrorDesign.jpg

 

I can now execute my job over my remote server over cmd ,as I get results from tFileOutputRaw_1 in which I set C:\Talend\6.3.1\cmdline\studio path for output File, but I got below error in tsystem Component -->

 

Exception in component tSystem_1
java.io.IOException: Cannot run program "cd C:\Talend\6.3.1\cmdline\studio": CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at avasf.auto_publish_n_deploy_job_0_1.Auto_Publish_N_Deploy_Job.tSystem
_1Process(Auto_Publish_N_Deploy_Job.java:3342)
at avasf.auto_publish_n_deploy_job_0_1.Auto_Publish_N_Deploy_Job.tFixedF
lowInput_1Process(Auto_Publish_N_Deploy_Job.java:3185)
at avasf.auto_publish_n_deploy_job_0_1.Auto_Publish_N_Deploy_Job.tFileIn
putDelimited_1Process(Auto_Publish_N_Deploy_Job.java:2665)
at avasf.auto_publish_n_deploy_job_0_1.Auto_Publish_N_Deploy_Job.tSetGlo
balVar_1Process(Auto_Publish_N_Deploy_Job.java:3632)
at avasf.auto_publish_n_deploy_job_0_1.Auto_Publish_N_Deploy_Job.runJobI
nTOS(Auto_Publish_N_Deploy_Job.java:7934)
at avasf.auto_publish_n_deploy_job_0_1.Auto_Publish_N_Deploy_Job.main(Au

 

Thanks,

Vaibhav

Six Stars

Re: how to access directory situated on different server through TDF ?

Hi,

Finally issue resolved .I build my local talend job and include home directory with the path as same as the path on remote server where commandline.bat file resides.

 

Building talend job and execute .bat file of that job on remote server after copying zip file of job on the server. This Works fine for me. I refer following topics- 1) http://etladvisors.com/2014/05/06/windows-deployment-and-scheduling-of-talend-jobs/

2)https://www.talendforge.org/forum/viewtopic.php?id=43435

 

Thanks,
Vaibhav