TOS DI - tsqlite output slow performance

Four Stars

TOS DI - tsqlite output slow performance

Hello everyone! I have a problem with tsqlite output component, i tried the same simple job on my Windows (i7 2,20 GHz - 16 GB RAM -  64 bit) :


As you see it's a very simple JOB where I am loading a 3'000'000 rows csv table, in a sqlite db. 

The problem is that this job run on average 5000 rows/s on my windows. But I tried it on the macbook air of my colleagues and the same job run on average 75 000 rows/s, and moreover the macbook has only 8GB of RAM .

I tried to increase Xmx and Xms but nothing changed. Someone can give me a suggestion?


Re: TOS DI - tsqlite output slow performance


The average speed was getting slow down when you load your data into tsqliteoutput component?

Here is "Use Batch" check box in advanced settings of  tsqliteoutput component which is used  to activate the batch mode for data processing.

Let us know if it is OK with you.

Best regards


Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.


Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.