From Thursday, July, 9, 3:00 PM Pacific,
our Community site will be in
read-only mode
through Sunday, July 12th.
Thank you for your patience.

tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Highlighted
Five Stars

tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

When upgrading project from 6.4 to 7.2.1, a job started to fail due to arrayoutofbounds error. I isolated this to the tFTPFileList component.  When reviewing the generation code and comparing it to the prior version it appears there is a bug in the logic.

 

Is this the proper venue to report this or is there a SOP for doing so?

 

6.4.0 code: The loop for pulling the data from the FTP dir method was slightly different.

for (String filemask_tFTPFileList_1 : maskList_tFTPFileList_1) {
java.util.regex.Pattern fileNamePattern_tFTPFileList_1 = java.util.regex.Pattern
.compile(filemask_tFTPFileList_1
.replaceAll("\\.", "\\\\.")
.replaceAll("\\*", ".*"));

for (String filemaskTemp_tFTPFileList_1 : fileList_tFTPFileList_1) {
if (fileNamePattern_tFTPFileList_1
.matcher(
filemaskTemp_tFTPFileList_1)
.matches()) {
fileListTemp_tFTPFileList_1
.add(filemaskTemp_tFTPFileList_1);
}
}
}

 

 

7.1.1 code: Issue is that the ftp_tFTPFileList_1.dir(null, true) and ftp_tFTPFileList_1.dir(null, false) bring back different sized arrays, and the looping logic is based off of the larger array. 

 

String[] fileList_tFTPFileList_1 = ftp_tFTPFileList_1.dir(null, true);
String[] fileListwithNames_tFTPFileList_1 = ftp_tFTPFileList_1.dir(null, false);
ftp_tFTPFileList_1.chdir(rootDir_tFTPFileList_1);
List<String> fileListTemp_tFTPFileList_1 = new java.util.ArrayList<String>();
List<String> fileListTempWithNames_tFTPFileList_1 = new java.util.ArrayList<String>();

for (String filemask_tFTPFileList_1 : maskList_tFTPFileList_1) {
java.util.regex.Pattern fileNamePattern_tFTPFileList_1 = java.util.regex.Pattern
.compile(filemask_tFTPFileList_1.replaceAll("\\.", "\\\\.")
.replaceAll("\\*", ".*"));
for (int i_tFTPFileList_1 = 0; i_tFTPFileList_1 < fileList_tFTPFileList_1.length; i_tFTPFileList_1++) {
String filemaskTemp_tFTPFileList_1 = fileList_tFTPFileList_1[i_tFTPFileList_1];
String filemaskTempName_tFTPFileList_1 = fileListwithNames_tFTPFileList_1[i_tFTPFileList_1];

if (fileNamePattern_tFTPFileList_1.matcher(filemaskTempName_tFTPFileList_1)
.matches()) {
fileListTemp_tFTPFileList_1.add(filemaskTemp_tFTPFileList_1);
fileListTempWithNames_tFTPFileList_1
.add(fileListwithNames_tFTPFileList_1[i_tFTPFileList_1]);
}
}
}

Highlighted
Employee

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Hi,

 

    Please report the bug t Talend JIRA forum. The link is as shown below.

 

https://jira.talendforge.org

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Highlighted
Four Stars

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Talend Team, 

 

Can you please let us know, if there is any patch available for the bug explained in this thread related to tFTPFileList in v7.2.1?

 

Highlighted
Four Stars

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Didn't see JIRA so created ticket:  TDI-43662 tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Highlighted
Five Stars

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

So I had put a ticket in with Talend support since the issue was in their enterprise version. Talend support created bug report and then a patch for the issue. I guess I expected that they would have been pushed back into the TOS repo as well. The relationship between the TOS and the for profit side of things is a bit confusing at times.  

Highlighted
Employee
Employee

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Please attach the job which exists bugs.

Here is the tutorial https://help.talend.com/reader/HqPypUO_cLctKv_dAWcgFw/fYXMbAVE56gH~wvSDjnxTw

Highlighted
Five Stars

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

The internal case number was 00151288. Patch was Patch_20191015_TPS-3447.  I don't have the test job any longer that reproduces. Here are some notes from the interaction with support.

 

Some details:

  • Our internal server is a windows based server. A public server I hit was Linux based, and my test job at the time does not fail. The FTP server is Windows 2016 running ipswitch wsftp server 8.5.2.
  • I suspect there MAY be a change in behavior in the ftp library that this component uses, in how it handles dir() (using LIST versus NLIST), but regardless, the component should handle it.
  • My test/job uses a file mask (*.*). However the error occurs with or without the mask.

 

Added notes about the folder I am trying to list:

  • happens o be one level of folder off of the root.
  • there multiple folders off of the root.
  • number of files in the folder does not change the behavior.  

My work around was taking the 6.4 code and creating a custom object and I used that. 

 

I put together a new test....

Highlighted
Employee
Employee

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Thank for your information. But sorry, I still couldn't find the problem. 

It would be helpful to have the original job.

Highlighted
Four Stars

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

@vyu  If you're testing with the patch installed, you won't be able to reproduce the error.  The patch is only available to Enterprise Edition users.  As I'm using the community edition, the patch is unavailable to me.  

Highlighted
Five Stars

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

That job I attached is the same that was submitted to Enterprise ticket and used for debugging. I want to emphasize that you need to connect to a FTP server that I mentioned above. I ran the job attached against an internal windows server, and was able to reproduce the error.

 

I think the enterprise engineer who worked the case installed the server mentioned above and was able to reproduce. 

Highlighted
Employee
Employee

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

This issue will be fixed in 7.3.1 Release. Please wait for the new version.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now