Lock file or check if file open

One Star

Lock file or check if file open

Hi,
Are there any component that can support us lock a file or check if that file is open?
Thanks for help!
Moderator

Re: Lock file or check if file open

Hi,
Are there any component that can support us lock a file or check if that file is open?

We don't have such component as you mentioned. As we known that Talend is a code generator ETL which use JAVA as the underline technology generated to perform the Data Extraction, Transformation and Loading.

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Lock file or check if file open

Hi
Thanks for reply,
I'm having problem with file locking. I must run 2 parallel subjob which will access the same file. If subjob 1 run first then subjob 2 will be pause; and if subjob 2 run first, subjob 1 must be pause.
Thus, at each subjob, I must check if the file is opened by another or not.
And because there aren't any supported component, I use tJava with the following code, but it's seem not right. Can u help me solve this problem? Thanks
Here is the code:
java.io.File file = new java.io.File(fileName);
java.nio.channels.FileChannel channel = new java.io.RandomAccessFile(file, "rw").getChannel();
// Get an exclusive lock on the whole file
java.nio.channels.FileLock lock = channel.lock();
try {
lock = channel.tryLock();
System.out.println("lock success");
} catch (java.nio.channels.OverlappingFileLockException e) {
System.out.println("pause");
} finally {
lock.release();
}
Community Manager

Re: Lock file or check if file open

Hi
You can try this job design without adding any Java code to check the file status.
Parent job:
tRunJob_1
tRunJob_2
tRunJob_1: call a child job that run the processing1, uncheck the 'die on error' option on tRunJob.
tRunJob_1: call a child job that run the processing2, uncheck the 'die on error' option on tRunJob.
Open the Job view of parent job, click Extra tab, and check the 'Mutiple thread execution' option.

Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Seven Stars

Re: Lock file or check if file open

Use tWaitForFile with the options "Include present file" and "Wait for file to be released" at the start of each subjob.
One Star

Re: Lock file or check if file open

Hi,
The tWaitForFile is not working for ftp transfer. I have two job. One is getting a file from remote server using tFTPGet.
Other reading the file which is read from remote server. Both job running parallely.

Job 1 : tFTPConnection---->tFTPFileList---->tFTPGet

Job 2: tWaitForFile----on subjob ok----> tFileList--- >tFileInputDelimited--->tMap--->tLogRow.
Problem here, second job is not waiting ftp to complete. I have used "Include present file" and "Wait for file to be released" option in tWaitForFile and checked but still its not working.
As i know there is no file lock component in Talend, how to resolve this problem. Anybody know how to solve it. Please let me know the solution.
Regards,
Akki
One Star

Re: Lock file or check if file open

Hi,
I forgot to mention. I'm using talend TOS 5.2.2 in linux machine. Whether tWaitForFile support only in windows??
Regards,
Akshath Hegde
One Star

Re: Lock file or check if file open

Hi,
I want to know whther there is any component for this scenario in TIS?? if yes please suggest me the component and its use. And please reply for above questions. i hope Most of people are expecting the answer for these questions.
Regards,
Akki
Community Manager

Re: Lock file or check if file open

Hi
There is a job design problem in your job, you should use iterate instead of onSubjobOk link from tWaitForFile, eg:
tWaitForFile----iterate---> tFileList--- >tFileInputDelimited--->tMap--->tLogRow.
If you select 'continue loop' otion in the Then list, tWaitForFile keeps working forever without end.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Lock file or check if file open

Hi shong,
Thank you for reply shong. i have checked with iterate in tWaitForFile. But it will not wait file transfer to complete.
"tWaitForFile should wait file transfer to complete, after complete transfer only it should take transferred file".
Please help to resolve this problem. If you reply quick it will be very helpful for us.

Regards,
Akki
One Star

Re: Lock file or check if file open

Hi Shong,
Even i have tried with iterate in tWaitForFile, But tWaitForFile will not wait file transfer to complete. Please suggest a appropriate solution for this.
Thanks and Regards,
Mithun
Community Manager

Re: Lock file or check if file open

Hi
Both job running parallely.

Why do you run the two jobs parallely? Run the job 2 first, tWaitForFile will monitor the specified directory and trigger the next action once a new file is created, and then run any other jobs.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Lock file or check if file open

Hi shong,
Thanks for reply shong. That's our requirement. In our project, file will be transferring from remote server. So other job should wait completion of file transfer.
"Is this possible?? there is any way to achieve this". Please suggest any possibilities for this scenario.
Regards,
Akki
One Star

Re: Lock file or check if file open

Hi,
As u have mentioned tWaitForFile will trigger on next action, But i want other job to wait until FTP completes its transmission.
"For ex: When one process is writing to file other process should not be able to read that file. "
Regards,
Mithun
One Star

Re: Lock file or check if file open

Hi,
Please advice me because i am running out of time.
Regards,
Akki
One Star

Re: Lock file or check if file open

Hi,
Can Anybody suggest answer please...
Thanks and regards,
Mithun
Community Manager

Re: Lock file or check if file open

Hi Mithun
Job1 and Job2 are two separated jobs in your case, right? As I suggested, run Job2 (contains tWaitForFile component) first to make tWaitForFile keep monitor the specified folder and trigger next action once a new file arrives.
If I don't understand your request well, can you please export the jobs and email them to me?
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Lock file or check if file open

Hi shong,
I have sent the mail to you. Please find the attachment in it.
Regards,
Akshath
One Star

Re: Lock file or check if file open

Hi,
It's very sad that many of questions are still unanswered in talend forge!!! It is not good for talend as well as for the us (talend user). People will be tired of waiting for answer. I have been expecting answer for AKSHATH question but there is no result here. I think it better, not to ask quetion's from talend forge. If there is no answer then why we have to waste our time in posting questions.

-----------------
From,
Talend User
Community Manager

Re: Lock file or check if file open

Hi Akshath
I have received your email and your job, but we have so many topics/questions to read and answer each day, please notify your reply to me if I miss your topic next time. I will try to take a look at your job and reply you today.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Lock file or check if file open

Hi shong,

Thanks for replying. I can understand your work pressure. But whoever posted previous post its true. If anybody answer for our post we will not get any notification, so we have to keep on monitoring this page. Anyway i will be waiting for your reply.
Regards,
Akshath
Community Manager

Re: Lock file or check if file open

Hi Akshath
I have took a look at your job and found that there are some job design problem, I have redesigned it and email it to you.
I test the 'Wait for the file to be released' option on tWaitForFile on v5.3.1 and it works as expected, tWaitForFile starts to trigger next action only when the file is released (all the transfer finishes). I am not sure it was a bug on v5.2.2 the version you are using, if you still get problem on v5.2.2, try to download the latest version 5.3.1 and test it again.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Lock file or check if file open

Hi shong,
Just wanted to give feedback.. WaitForFile we tested all possibilities. But tWaitForFile is not working as expected. And stackOverFolw they have mentioned that tWaitForFile wont work in linux.. So may be this is reason we are not getting over expected output from tWaitForFile.
I suggest you to test it, If its not working then please in next version try to release it. J
Regards,
Akshath Hegde