[resolved] File sync

One Star

[resolved] File sync

Hi I want to keep the files at two locations in sync with each other.The two file locations will be passed as the context variables.
e.g.
let one location is C:/source
and the other is C:/target
When my job runs for the first time it should first check the source folder and match them with the contents of the target folder. If there is a difference in the two locations the files need to be copied to the other location.Basically the two folders need to be in sync with each other. If there are new files in the target folder they need to be copied to the source and vice versa.

Any help on the above will be highly appreciated.
Thanks in Advance
Abhineet

Accepted Solutions
Community Manager

Re: [resolved] File sync

Hi
Using tFileList to iterate each file in source folder, check the file if it exits in the target using tFileExist, copy it to target folder with tFileCopy if not. Foe example:
tFileList--iterate--tJava--componentOK--tFileExist--runIf--tFileCopy
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business

All Replies
Community Manager

Re: [resolved] File sync

Hi
Using tFileList to iterate each file in source folder, check the file if it exits in the target using tFileExist, copy it to target folder with tFileCopy if not. Foe example:
tFileList--iterate--tJava--componentOK--tFileExist--runIf--tFileCopy
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] File sync

Thanks Shong!
Seems good..Do I need to follow the same steps in another flow while I check the target folder?
Because there may be a scenario that there is a new file in the target folder that also needs to be copied in the source folder.
Regards,
Abhineet
Community Manager

Re: [resolved] File sync

Hi
Yes, use the same way to check the target folder in next subjob.
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] File sync

Hi
Using tFileList to iterate each file in source folder, check the file if it exits in the target using tFileExist, copy it to target folder with tFileCopy if not. Foe example:
tFileList--iterate--tJava--componentOK--tFileExist--runIf--tFileCopy
Best regards
Shong

Hi I trying the same solution but I am bit unfamiliar with Java.Can u help me out with the sample Java code to be used in tJava component.
Regards,
Abhineet
One Star

Re: [resolved] File sync

Hi,
Don't think you need to use a tJava (shong correct me if i'm wrong Smiley Happy). Just link your tfileList to the tFileExist.
tFileList--iterate--tfileExist---runif---tfilecopy
One Star

Re: [resolved] File sync

Hi,
Don't think you need to use a tJava (shong correct me if i'm wrong Smiley Happy). Just link your tfileList to the tFileExist.
tFileList--iterate--tfileExist---runif---tfilecopy

I'm not sure if tfileExist is a standalone component because when I am using the combination
tFileList--iterate--tfileExist---runif---tfilecopy
it gives me a syntax error in the tfileExist component.
Regards,
Abhineet
One Star

Re: [resolved] File sync

show me your job and what you have put in your tfileexist component
One Star

Re: [resolved] File sync

My Bad!
seems to be working now...guess there was some mistake in the "if" condition.

Regards,
Abhineet
One Star

Re: [resolved] File sync

Sometimes it's good to switch in the source view, it's easier to detect the errors
Community Manager

Re: [resolved] File sync

Hi
@Or3l: I use tJava to get the current file name and initialize variables, such as the target file path, which will be used on tFileExist.
Best regards
Shong.
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] File sync

Finally it worked that too without a tJava Smiley Happy
Thanks Guys!!
Regards,
Abhineet
One Star

Re: [resolved] File sync

Hi,
appologies to reopen the resolved item.
As a part of the same requirement i need to copy only those files which are created not more the a minute before.
I was thinking of using a tfileProperties along with the below flow

tFileList--iterate--tfileExist---runif---tfilecopy
|
|
iterate(order2)
|
tfileProperties
and give the condition in runIf , something like (systemTime - filemodifiedTime < 1).
please suggest if it is the correct approach or can be achieved in some other way.

Regards,
Abhineet
One Star

Re: [resolved] File sync

Hi,
appologies to reopen the resolved item.
As a part of the same requirement i need to copy only those files which are created not more the a minute before.
I was thinking of using a tfileProperties along with the below flow

tFileList--iterate--tfileExist---runif---tfilecopy
|
|
iterate(order2)
|
tfileProperties
and give the condition in runIf , something like (systemTime - filemodifiedTime < 1).
please suggest if it is the correct approach or can be achieved in some other way.

Also , if the above approach is correct how to achieve that.
Regards,
Abhineet
One Star

Re: [resolved] File sync

Hi Team,
Can somebody help me on that?
Regards,
Abhineet
One Star

Re: [resolved] File sync

Hi,
I think I would first get the file that have been created not more than a minute before, then check if there are already in the target folder and then copy them. So It would give sthing like this
tFileList--iterate---tFileProperties----runif----tfileexist---runif---tfilecopy
One thing toknow is that the mtime property is based on the datetime modification of the file, so it's not really the date of creation of the file.
Furthermore I don't know if you have already managed that but you should probably "tag" the files you have already synchronized (by adding something in their name for example). Unless your job will be quite long.
One Star

Re: [resolved] File sync

Hi,
I tried the above connection but it seems that the connection "tFileProperties----runif----tfileexist" is not feasible. It gives me a warning that tfileProperties must have an output attached.
Regards,
Abhineet
One Star

Re: [resolved] File sync

Hi,
put a tflowtoiterate between tfileproperties an the runif
One Star

Re: [resolved] File sync

Use tfilelist---iterate-->tFilecopy 
 in tfilecopy you can use 
filename ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))
Replace Existing files checked