Delete HDFS File Based on Creation Date

Delete HDFS File Based on Creation Date

Good morning,
I'm attempting to delete files from a HDFS directory based on creation date.  I'm using something similar to the solution found here: 
My problem is rather than using tFileList/tFileDelete, I need to use tHDFSList/tHDFSDelete.  However tFilterRow does not connect to tHDFSDelete.
I'm somewhat of a novice in Talend, teaching myself little by little.  Any help or ideas on how I can change this flow to work is greatly appreciated!
Moderator

Re: Delete HDFS File Based on Creation Date

Hi,
The tfilterrow component requires an output component. Could you please try this work flow to see if it is Ok with you?
tHDFSConnection-->tHDFSList-->tHDFSProperties-->tMap-->tFilterRow-->tfileoutputxxx-->tHDFSDelete
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Ten Stars

Re: Delete HDFS File Based on Creation Date

If the output of tFilterRow is the list of files to delete, inserting a tFlowToIterate between it and tHDFSDelete should let you loop over the files and delete each one.

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog