tPOP3 too much slow!

tPOP3 too much slow!

Hi all,
in a job of mine I use tPOP3 component to retrieve mail having some excel files in attachment.
The email box contains (at today) about 1000 email and each day the job retrieves 100 email and filters them as showed in the attached images.

But, tPOP3 is too much slow to retrieve email!
In fact, it requires about 20 minutes to complete.
I perfomed a lot of ping against the pop3 server and it is ok (response time is about 40-50 ms)
Do you think I need to edit any settings or is this the normal behavior of the component?
Moderator

Re: tPOP3 too much slow!

Hi,
Is there any error message printed on console when retrieve mail from your pop3 server? What's the purpose of second filter "A"? 
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

Re: tPOP3 too much slow!

Hi Sabrina,
no errors are displayed in console.
The second filter considers only email sent "TO" (=A in italian language) a specific email address.
Thanks for any advice.
Moderator

Re: tPOP3 too much slow!

Hi,
Could you please try to retrieve your email from pop server by using tPop component without any filter to see if it is normal?(make a simple job and set the number emails retrieved as 10)?
We don't see any problem on your component setting. Is it the whole job design? tPop? Did you read these emails through tfileinputmail?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

Re: tPOP3 too much slow!

Hi,
I performed the following tests:
1. I create a job with only a tPop3 component (nothing else!)
2. I set the number of the retrieved mails equal to 1---> Result: too slow
3. I removed a filter ---> Result: too slow
4. I removed the other filter (so without filters) ---> Fast
5.  I applied the first removed filter ---> too slow
6. I removed the previous filter and set the number of the retrived mails equal to 100 --> Fast
So, it's clear that the problem is due to filters which cause a very slow execution of the job.
Do you have any advice to improve performance?
Many thanks!
Moderator

Re: tPOP3 too much slow!

Hi, 
The "To" is working well with you? Do you want to get emails from yesterday?
Could you please also indicate the build version you are using? Open source or subscription version?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

Re: tPOP3 too much slow!

Hi Sabrina,
"To" options works well but it causes very slow performance as other filters.
As you wrote correctly, I would like consider emails sent yesterday to a specific recipient. In particular, I have to read emails sent to different mail groups. My email account is subscribed to different mail groups. So, I want to consider emails sent yesterday to a particular email group.
I'm using TOS DI 6.1.0.
Thank you very much for your help!

Re: tPOP3 too much slow!

Hi,
can anybody give me any advice?
Can anybody share their exprience?
Thanks a lot
Moderator

Re: tPOP3 too much slow!

Hi,
We don't see any error in your job setting. Does this issue repro on other build version?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

Re: tPOP3 too much slow!

Hi,
there aren't errors but perfomance are very very slow when a filter is applied as described in my previous posts.
I don't know if it happens in other versions.

Re: tPOP3 too much slow!

Moreover, I found this following (old) post:
https://www.talendforge.org/forum/viewtopic.php?id=9303
where at the last the user wvaibhav  describes just what I wrote above.
I'm still trying different configuration to understand the problem related to slow performance, at the moment, without success.
Four Stars

Re: tPOP3 too much slow!

Today, after about 3 months I haven't found a solution able to speed up my job in which I use tPOP component with filters as described in my previous posts.
Can anybody who use tPOP component with filters can share its own configuration and experience, please?
This strange behavior is killing me...Smiley Happy
Many thanks in advance for your contribution
Sixteen Stars

Re: tPOP3 too much slow!

I have to admit that I have never had this issue, so can't give you any advice on this. However, if you want to prove this is a Talend issue and not something related to your email account (it is possible), you can do a comparison with Talend of using their component and using some example Java to do the same thing.
1) Create a new email account where you can share the credentials.
2) Create a job using the Talend component.
3) Create a similar job using some example Java code for retrieving emails (there is loads on the net) in a tJava or tJavaFlex
Then compare. If both examples perform in the same way, then there is clearly something else wrong (your network, your computer, your email service provider, something Java related). However, if you can clearly see a performance improvement when using the job that uses the example Java, then you can send it to Talend to prove the issue.
The other advantage of doing this is that if the Java version does work better, you can use that until Talend fix this (.....if it is their fault). 
Four Stars

Re: tPOP3 too much slow!

Hi,
i verify the email retrieving behavior using  custom code (using tJava and in a main method in Eclipse) 
I also read the code generated in Talend and I noticed that, because POP3 limitations, all INBOX folder are loaded each job execution.
So, at the beginning of the "mailbox life" all is fast. After some mail (my inbox folder contains about 3500 mail)  and the code
javax.mail.Message[] msgs_tPOP_4 = folder_tPOP_4.getMessages();

cause an array filled with 3500 messages
Each element of this array is processed by a loop as follows
for (int counter_tPOP_4 = 0; counter_tPOP_4 < msgs_tPOP_4.length; counter_tPOP_4++) {

where filters are checked.
Once the array is filled all mail server are equal, so I put my attention to the loop because the array loading is fast
I noticed that this loop are executed differently in different servers!
In a server, in fact, a loop over about 800 messages is completed immediately, while a loop over about 3500 messages is completed after 15 minutes!
what do you think? Shall I check JVM options?