I don't know if this topic is the correct one to my question but were we go:
I want to migrate a solution implemented using pentaho to talend and i want to know what is the best approach to replicate the pentaho scenario in talend. The scenario is the following one:
- N file Feeds with files each 2, 5 or 15 minutes that need to be:
I hope that this description provides enough information in order you can guide me on the best path to the architecture needed in talend to respond to this scenario.
Could you please elaborate your scenario with an example with input and expected output values?
My scenario is the following one:
- N files placed in a remote folder by external processes.
- My application must acquire those files, register them in a control table, and process then. This process includes enrichment of the file by processing the values contained in it and by consulting database tables and after that produce a file that will be loaded to one database via external table.
In a very simple description the scenario is the one described above but as I try to specify in the first post we have those requirements because this is a very intensive use scenario.
Just for the record in Pentaho we have 5 Data Integrations in cluster mode to ensure the processing of the files but even with this configuration the client is not satisfied with the solution because isn't fastest enough and have a lot of crashes on those DI's.