Can't connect to mongo: pool is closed

One Star

Can't connect to mongo: pool is closed

Hi,
We are migrating from MySQL to MongoDB, using the exact same jobs, changing only connections/inputs/outputs components.
When running in our production cluster we randomly get this exception from the tMongoDBConnection component:
java.lang.IllegalStateException:The pool is closed

How can we handle this error?
Thanks,
Maxime.
Moderator

Re: Can't connect to mongo: pool is closed

Hi,
Could you please post your current job design screeshots into forum(migrating from MySQL to MongoDB)? Have you tried to check out "Die on error" option on tMongoDBOutPut component to see if there is any more error message printed on console?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Can't connect to mongo: pool is closed

Hi Sabrina, thanks for the answer,
Actually the problem is not occuring within talend, if I run any job within talend it runs fine. It looks like more a scalability issue:
If I run various jobs in parallel in our cluster then the connection problems appear, never in the Talend console. They always appear during the connection (tMongoDBConnection ), which is the first component of each job.
This is what our log tables looks like:
Moderator

Re: Can't connect to mongo: pool is closed

Hi,
Could you please indicate the build version you are using? subscription solution?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Can't connect to mongo: pool is closed

Yes sure, we are using Talend Open Studio Big Data 5.6.2 
Moderator

Re: Can't connect to mongo: pool is closed

Hi,
Have you tried to use tMongoDBClose in your workflow to close a connection to the MongoDB database?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Can't connect to mongo: pool is closed

Hi Sabrina,
Yes, every job starts with a tMongoDBConnection and ends with a tMongoDBClose.
Moderator

Re: Can't connect to mongo: pool is closed

Hi,
Would you mind uploading your current job design screenshot into forum?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Can't connect to mongo: pool is closed

Yes of course, please find attached one example. All our jobs are designed the same way, regarding the connections. When we were working with MySQL the jobs were identicals and we didn't have this problem.
One Star

Re: Can't connect to mongo: pool is closed

Hi Sabrina,
Do you have any thoughts about this that could help me find the root of the problem?
Thanks a lot,
Maxime
One Star

Re: Can't connect to mongo: pool is closed

Hi Sabrina,
I just got an answer from MongoDB support regarding this exception:
The exception you are seeing is from the Java driver and usually occurs if the application has disposed or otherwise closed the connections in the pool, and then the application attempts to use said pool. You do not get this exception from having a connection pool that is too small.
This is really weird because as you can see in this screenshot the exception occurs during the connection, at the very first step of each job:

It happends randomly and under some load, but never more than 30 concurrent connections so it should be handled correctly.
Thanks for any help,
Maxime.