tFileOutput, get filename used

Four Stars

tFileOutput, get filename used

Hi, after having created a file with tFileOutputPositional, how do i access the filename used? When writing the file i use a unique name, 'C:/in/work/" + (String)globalMap.get("tFileList_1_CURRENT_FILE") + "_" + java.util.UUID.randomUUID()', and the plan is to OnSubJubOK rename the file with tFileCopy. But i cannot find the filename used by tFileOutputPositional anywhere. In the 6.3 documentation it seems that it was added to the globalMap but I'm using 7.1 and it no longer seems to be apart of it.

Nine Stars

Re: tFileOutput, get filename used

Before passing the data to tFileOutputPositional you could process it through tJavarow like - 

//Code generated according to input schema and output schema
output_row.newColumn = input_row.newColumn;
output_row.newColumn1 = input_row.newColumn1;
output_row.newColumn2 = input_row.newColumn2;

and after that you could use your logic to derive the filename and store it in context or globalMap which you could use in the tFileOutputPositional - "File Name" property as well in the tFileCopy in a different subjob.

String uuid = UUID.randomUUID().toString();
String filename = "C:/Talend/"+ uuid +"/out.txt";
globalMap.put("filename", filename);
System.out.println("Filename: "+ (String)globalMap.get("filename"));
Four Stars

Re: tFileOutput, get filename used

Have tried some similar to that, but couldn't get it to work as the filename part of tFileOutputPositional is initialized on subjob start. So directly after my tFileList component has run, the filename of tFileOutputPositional is set :/.

Eight Stars

Re: tFileOutput, get filename used

Hi @anpe ,

 

Instead of onSubjob Ok from tFileLIst you should put OnComponent Ok from the component just befor tFileCopy.

 

As you are using OnSubjobOk from tFileList, the CURRENT_FILE globalMap will be flushed, hence you will get null value in that.

 

On Component Ok link from the component before tFileCopy will still keep the globalMap of tFileList till the execution is finished from tFileCopy and the second iteration will start from tFileList then only.

 

Thanks and Regards,

Subhadip

Four Stars

Re: tFileOutput, get filename used

Hi subhadip13, think you have misunderstood my problem. The problem is not the CURRENT_FILE variable. Put the filename that tFileOutputPositional use. My job can be seen below. It polls a folder and then does some mapping changes before writing a new file to disk. After that I want to move the newly created file, but need to know what the filename used was, as it is randomly set.

Capture.PNG

Eight Stars

Re: tFileOutput, get filename used

Hi @anpe ,

 

Sorry if I have misunderstood the problem.

 

You can do an  "OnComponentOk" from tFileOutputPositional and use that link to tFileCopy.

 

Thanks and Regards,

Subhadip

 

Nine Stars

Re: tFileOutput, get filename used

Based on the screenshot of the job shared, I think you could complete the flow in a single sub job where you could use tFileCopy after using tFileOutputPositional in that case you would be able to get/store the file names for every iteration.
Highlighted
Four Stars

Re: tFileOutput, get filename used

Ï don't see how that helps, tFileOutputPositional still doesn't output the filename it uses. So why would it help to connect the tFileCopy to it directly?

Four Stars

Re: tFileOutput, get filename used

For now i ended up in below solution, adding a tJava after tFileList to set the filename and add it to the globalMap (thus before tFileOutputPositional is initialized). It works, but is not exactly what I want. The current solution doesn't allow me to use any data inside the file when setting the filename, which I might want in the future.

 

 

Capture.PNG

Nine Stars

Re: tFileOutput, get filename used

That's exactly what I had suggested at the beginning to use tJavarow to derive the filename and use it later for your tFileOutputPositional task and also to have the tFileCopy immediately after and not in separate subjob.
Regarding your "use any data inside the file when setting the filename" - this could be done after your file is written, by reading the same file not sure though what you had meant apart from this.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog