Moving files within the bucket after deleting source files

Four Stars

Moving files within the bucket after deleting source files

scenario : moving the files to destination folder (s3 within the bucket) then removing the source files 

 

job flow

ts3 connection

 

ts3 list

bucket name  :-    "bucket_src"

Key prefix :-     "Main/Source/"

 

ts3 copy

source configuration bucket :- "bucket_src"

Key :- ((String)globalMap.get("tS3List_1_CURRENT_KEY"))

Destination configuration bucket :- "bucket_src"

Key :- "Main/Dest/" + ((String)globalMap.get("tS3List_1_CURRENT_KEY")).substring( ((String)globalMap.get("tS3List_1_CURRENT_KEY")).lastIndexOf("/") + 1 )

 

ts3 Delete

Bucket :- "bucket_src"

Key :- ((String)globalMap.get("tS3List_1_CURRENT_KEY"))

(Deleting entire source folder )

my requirement I want to delete files after loading to destination folder 

Please suggest to deleting only files.

 

Thanks 

MRA

 

 

 

 

 

 

 

 

 

 

Tags (1)
Community Manager

Re: Moving files within the bucket after deleting source files

Hello
The key includes the folder path, you need to exclude the the folder path and only iterate the files.
eg:
tS3list--iterate--tjava--runIf--tS3Copy--oncomponent--tS3Delete

on tjava:
String key = ((String)globalMap.get("tS3List_1_CURRENT_KEY"));
int length=key.length();

if(length>12){
globalMap.put("isContinue",true);
}else{
globalMap.put("isContinue",false);
}

Set the condition of runIf link as:
(Boolean)globalMap.get("isContinue")

Hope it helps you!

Regards
Shong

----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: Moving files within the bucket after deleting source files

The Name "(Boolean)globalMap.get("isContinue")" is not valid or already exists.

Four Stars

Re: Moving files within the bucket after deleting source files

Run if condition error 

 

The Name "(Boolean)globalMap.get("isContinue")" is not valid or already exists.

Community Manager

Re: Moving files within the bucket after deleting source files

No double quotes enclose the expression, it is (Boolean)globalMap.get("isContinue").
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: Moving files within the bucket after deleting source files

Hi Shong,

 

even without double quotation also getting same error 

 

 

 

 

Thanks,

MRA

Community Manager

Re: Moving files within the bucket after deleting source files

Can you please capture a screenshot to show your job design and where you write the expression?
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: Moving files within the bucket after deleting source files

Hi Shong,

thank a lot for quick replay

 

 

Thanks,

MRA


This is the job flow. I achived

EX:-job flow
ts3 connection---ts3 list----ts3 copy
                             |----on subjob ok
                             |
                        ts3 list----ts3 Delete


ts3 connection

 

ts3 list

bucket name :- "bucket_src"

Key prefix :- "Main/Source/"

 

ts3 copy

source configuration bucket :- "bucket_src"

Key :- ((String)globalMap.get("tS3List_1_CURRENT_KEY"))

Destination configuration bucket :- "bucket_src"

Key :- "Main/Dest/" + ((String)globalMap.get("tS3List_1_CURRENT_KEY")).substring( ((String)globalMap.get("tS3List_1_CURRENT_KEY")).lastIndexOf("/") + 1 )

 

ts3 list

bucket name :- "bucket_src"

Key prefix(multiple key prefix) :- "Main/Source/output"  ('output' is the file name which we are getting in source directory)
                                                     "Main/Source/temp"  ('temp' is the file name which we are getting in source directory)

ts3 Delete

Bucket :- "bucket_src"

Key :- ((String)globalMap.get("tS3List_1_CURRENT_KEY"))