Five Stars

How to include a file in Project Resource

Hi,

 

I want to include a file which i am referring in my Talend Job. 

I created a talend Routine which uses a file, I have loaded the file to routine along with jar file.

Now when i run the job in Local mode, Job is able to fetch the data from file, but when i run in job Server, It does not get the file.

 

help needed to resolve the issue

How to include the file in my project resources for remote run?

  • Data Integration
1 ACCEPTED SOLUTION

Accepted Solutions
Employee

Re: How to include a file in Project Resource

Generally Routines require static methods.  Hence, yes, you can pass the path of the file to your routines. It can be an extra parameter to your routine method, which is the path of the file, that you pass when calling the routine.

If you call the routine frequently, you can setup a static private variable, create a method to set that variable, and call that method first thing in your job.  Then every other routine methods can refer to the static variable which contains the path of the file.  And then you load it as usual in your routine.

 

You will need a folder on your jobserver or on a shared drive (locked down so that only the job can access it if the content needs to be secured) where you will put this file.

11 REPLIES
Community Manager

Re: How to include a file in Project Resource

Hi
It does not support relative file path. You can have a shared directory (through NFS or SMB mount) to have all necessary files accessible for all servers where your job runs.

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Five Stars

Re: How to include a file in Project Resource

Hi Shong,

 

How to point my job to look for the files which are needed for Project Resource?

 

In other words, How my job will understand the location of the file.

Five Stars

Re: How to include a file in Project Resource

Please find attached email for detailed explanation on what is the issue i am trying to resolve

Ten Stars

Re: How to include a file in Project Resource

You can change path to files for context variables

 

Spoiler
context.path_to_files+"/file_name1.csv"
context.path_to_files+"/file_name2.xls"

and etc

 

with implicit context loading it allow refer to different path on different servers

-----------
Five Stars

Re: How to include a file in Project Resource

Thanks for the information.

 

I am not looking to share the files between different servers.

 

I want to access a file which is located in Java Project Resource folder. (Same as Eclipse)

See my attached screenshot. (File Name is vmkeys.xml)

 

I want to access this file when i run the job in JobServer. Not sure where should i keep the file.

 

Employee

Re: How to include a file in Project Resource

This functionality exists for ESB services since we have resources there.  In DI, it doesn't.  You will need to put the file somewhere on disk on your jobserver, and read the file as the first operation of your job.

Ten Stars

Re: How to include a file in Project Resource

I also mean - not share file between servers, but have unique location for same file on different environment

for example - when Dev machine windows, and Prod Server Linux it is hard to have same PATH to resources

and context variables - help to resolve this

 

 

-----------
Five Stars

Re: How to include a file in Project Resource

My Java code routine in the job automatically searches for the file "vmkeys.xml" in project resources folder:

I.e. C:\Talend\workspace\.Java\src\main\resources

 

In my local workspace file is located under below path

C:\Talend\workspace\.Java\src\main\resources\vmkeys.xml

I do not have to specifically change anything in my job to read data from this file, for local run.

 

 

Now for remote run, If i keep the file in job server in below location

/ABC/vmkeys.xml

I am not sure how i change my job to look for this file in any other location in job server:

I am not sure how to make this work, I believe in Eclipse as well files are read from "Project Resources" folder.

 

Any suggestions would be really helpful.

 

 

Ten Stars

Re: How to include a file in Project Resource

what is the problem with context variables?

 

Screen Shot 2017-07-20 at 12.31.48 AM.pngScreen Shot 2017-07-20 at 12.31.58 AM.png

 

no one fixed value, in properties:

Screen Shot 2017-07-20 at 12.33.30 AM.png

 

and csv file on each machine:

 

Screen Shot 2017-07-20 at 12.32.21 AM.png

 

that all
when Job start, it read OS variable, it it not NULL - take config file from there, if not from default (for me) place

Configuration could be changed without re-build the Job

-----------
Five Stars

Re: How to include a file in Project Resource

Thanks for detailed explanation,

I think, I am not able to make my issue clear:

 

I have a xml file named vmkeys.xml, which is required for my routine.

My routine code is accessing this file when ever i run my job.

I think my code routine always expects the file to be at Java Project Resources folder

I.e. C:\Talend\workspace\.Java\src\main\resources

 

If i remove the file from above directory my routine will fail

 

Now if i keep my file to any other directory say C:\Talend\workspace\.Java\src\main\resources1

In this case also my routine will fail

 

Question:

I want to know if i can change my job routine to start looking for the file "vmkeys.xml" in any other directory.

The reason i want this is, because in job server i dont have any corresponding project resource directory.

 

Employee

Re: How to include a file in Project Resource

Generally Routines require static methods.  Hence, yes, you can pass the path of the file to your routines. It can be an extra parameter to your routine method, which is the path of the file, that you pass when calling the routine.

If you call the routine frequently, you can setup a static private variable, create a method to set that variable, and call that method first thing in your job.  Then every other routine methods can refer to the static variable which contains the path of the file.  And then you load it as usual in your routine.

 

You will need a folder on your jobserver or on a shared drive (locked down so that only the job can access it if the content needs to be secured) where you will put this file.