tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

Seven Stars

tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

I have a tFileList which iterates through a directory of excel files and a tMap to append the CurrentPath, Directory to the flow. 

For each file I read the contents but I want to include the modified date as well so I have the tFileProperties as lookup into a 2nd tmap and I use ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")) in the tFileInputExcel and tFileProperties 

 

I tried matching the CurrentPath [set in tMap1 using ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")) ] to the tFileProperties basepath and abspath but my output doesn't match for any,

 

I also changed the tMap lookupModel to reload at each row.

 

I guess in a nutshell how do I match the tFileProperties to the flow of data out of the same File?

Twelve Stars

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

did you use reload for each row for lockup on properties?
I do not know if you can retrieve properties on an open file!

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Seven Stars

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

I did use reload for each row for lookup.

 

My job is outlined below. If its not possible to get properties on an open file how do I re-engineer the job so I don't end up getting the properties but not the data rows (assuming I can't ReadExcel if the file is open for tFileProperties 

 

ReadFolder -> ReadExcel->tMap(addSourceFileDataToRow) ->tMap(GetModifiedDate) ->

                                                                                                             |

                                                                                                             |

                                                                                                     tFileProperties

Community Manager

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

Hi mobmsc2
Read the file properties and cache them into memory for used later, change the job as below:
ReadFolder -> tfileProperties--tHashoutput--oncomponentok-
ReadExcel->tMap(addSourceFileDataToRow) ->tMap(GetModifiedDate) ->

                                                                                       |

                                                                                       |

                                                                             tHashInput
Let me know if it works.

Regards
Shong

----------------------------------------------------------
Talend | Data Agility for Modern Business
Seven Stars

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

That didn't work. If I run in debug mode there are no rows coming in from the lookup hash flow so nothing to match against the flow from the tExcel. Thats even though there there is content read into the hashOutput

Community Manager

Re: tFileProperties as Lookup in tMap how to match the fields when reading from tFileList

Change the job design as below:
main job:
tfilelist--iterate--tRunJob

on tRunjob: call the child job and pass the current file path to child job, refer to this page to know how to pass a value from main job to child job.
https://help.talend.com/reader/Wi~~2Q0v6A43uBlbET9gQg/MHs41sIsaV91cFGXj4DVMQ

child job:
tfileProperties--tHashoutput
|
onsubjobok
|
ReadExcel->tMap(addSourceFileDataToRow) ->tMap(GetModifiedDate) ->

|

|

tHashInput
This way should work.

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog