Employee

Database polling best pratice

Hi,
I've a use case where I need to push information to partners when something change
in the source database.
We usually recommend to use a DI Job for the polling part and I would like to know if it's
the best practice or if there is a more "direct" way to do that with Camel.
By the way, I've seen on Camel JDBC page the following piece of code :
from("timer://foo?period=60000").setBody(constant("select * from customer")).to("jdbc:testdb").to("activemq:queue:customers");

but it doesn't seem to be the best solution...
Thank you for your help,
Bahaaldine
3 REPLIES
Employee

Re: Database polling best pratice

The code above is the easiest way to start. Of course it has some drawbacks. So it uses a fixed interval to poll. That means if you set the interval short you risk to waste resources. If you set it too high you may have too much latency. So for some cases it may be interesting to use a trigger in the database and trigger the poll by e.g. a http call. In general though polling is a good idea.
The other drawback is that every time all rows are loaded. If the table is big this may be a big issue. Again triggers may help in this case. So you could use a trigger on the original table to put each new row into a second table. Then the camel route could be on the second table and delete all rows after the poll.
One Star

Re: Database polling best pratice

Hi:
I'm one of the firts developer of EAI tools in Mexico. I have almost 20 years expertise on this field.
I was working with Vitria, WebMethods y Tibco.
The solution tha you propose as the secondary aproach:
"The other drawback is that every time all rows are loaded. If the table is big this may be a big issue. Again triggers may help in this case. So you could use a trigger on the original table to put each new row into a second table. Then the camel route could be on the second table and delete all rows after the poll."
Is exactly the solution used by Vitria and Tibco (Vitria and Tibco are cousing applications, they were created by the same team on the beggining), bute the deletion is done only after the 2 Face commint is completed, in order to keep the integrity.
Which one should it be the best aproach using Talend OS ESB 5.0?
Also I found in the documentation for 5.0 thta the EIP is included now, but I cannot found the message palete mentioned on the documentation.
Can you explain me how to do this?
I'll be waiting for your answer.
Best Regards.
One Star

Re: Database polling best pratice

hi ,
I am new to the Talend in the past i used webMethods, so anybody is there to help me in this regard.
Present i want to publish the data from Oracle to Stellent so how can i implement code for this operation in Talend.
Thanks in advance.....
Regards,
Ramakrishna