Four Stars

Performance testing

Hello community,

I'm a performance testing expert and completely new with Talend.

My company has implemented many jobs using Talend and they asked me to valide its performance under a load test.

So far, I worked only with load testing tools such as Jmeter or Loadrunner but it seems it's not suitable for this case.

I heard about a bulk load mode but don't know much.

When I test a web based app, I validate metrics such  as response time, CPU, hits per sec, throughput, transaction ok/ko etc...

But what about a Talend job?

What relevant metrics or information should I highlight in my analysis?

And what would be the best way to load test a Talend job? Any good tools for it?

Any help would be greatly appreciated.

I may have more information later about the technical architecture.

Antoine.

 

2 REPLIES
Five Stars

Re: Performance testing

Hello Antoine,

 

Bulk load is way how to import data to table directly from a file (e.g. CSV). Read about it for MySQL here:

https://dev.mysql.com/doc/refman/5.7/en/load-data.html

Similar operations exist also for other databases.

 

Metrics which can be important:

  • Number of records per second processed
  • How long the job runs (seconds, milliseconds...)
  • Resources consumed (can be merged via OS monitoring and PID provided by tStatCatcher)

One of the ways is to run the job in Talend Studio and see performance counters.

Another way is to collect statistics using tStatCatcher. It will return you the information you can merge with e.g. OS statistics (via PID number).

These statistics can be exported to a CSV file, database... The schema on the screenshot is generated the tStatCatcher. 

Výstřižek.PNG

Hope it helps.

 

Regards

lojdr

Four Stars

Re: Performance testing

Thank you Lojdr, I really appreciate your help!

I gonna try it soon but I'm almost sure it will solve my problem.

I'll keep you updated.

Thank you again.

Antoine.