Java job is 50% slower when run in Linux compared to Windows
A coworker and I were doing sample runs of a mildly complex Java job. We each took turns 3 times so that any unusual DB/network latency would be factored out between runs (all other coworkers had gone home for the night anyway). I consistently clocked 15 minutes and he consistently got 10 minutes. The job is a parent job that kicks off 10 child jobs of varying complexity. Traces and statistics were disabled. Note that the 50% speed decrease was seen very evenly across all child jobs. Perhaps most surprisingly, the CPU usage on my workstation was often very low, typically in the 10% range. Not a single one of the jobs writes files to disk (either with tFile* components or tMap temporary disk buffer usage), so HDD usage is non-existent on my machine. None of the jobs have any multi-threaded components. Still, I suspect that Java is not utilizing my CPU as well as it could. There probably wont be any magic bullet solution to this, but can anyone think of any settings we might check in either Talend, Java, or our operating systems which might contribute to this significant difference? We were both launching our jobs via TOS, but I don't see any reason why exporting a JAR and running it from the shell would make a difference. We have verified that our projects are bit-for-bit identical via SVN STAT. Is this almost certainly a Java issue?
My system specs (took 15 minutes): Intel Core 2 Duo E6550 (2333 MHz) Ubuntu 9.04 x64 4GB RAM (Java Xms and Xmx both set to 2500 MB) Java 1.6.0_13 TOS 3.1.1 All x64 version His system specs (job takes 10 minutes): Intel Mobile Core 2 Duo (needs verification, but is at least as old as the E6550) Windows XP 32-bit 4GB RAM (Java Xmx set to the default 1024 MB) Java 1.6.0_13 TOS 3.1.1 Windows x86 version
Let me know if any other information might be useful and I will provide it. Thanks, Brian
Re: Java job is 50% slower when run in Linux compared to Windows
Hi Brian, I don't think that your systems are comparable. First: The generated code is the same on both systems. And it shouldn't matter if you run your script from TOS or as an exported jar. Second: The main impact on performance will now have your system. This means: Your system may have different hardware, which will behave different (especially on load). (CPU-Speed, CPU-Architecture, Memory-Speed, ...) Your operating system will handle such things like memory-management, process- and thread-management, ... different The implementation of the JVM will be different (even if you use the same version). So again memory-management could be different, other functions could be realized in native code, the JIT-compiler will work different and much more. If you have problem with the execution time of your job on your workstation (Ubuntu) you should things like: - Memory settings (seems you allocate the double amount as on windows) - Process priority - Will your job do anything else than cpu usage? May be there a bottleneck? Bye Volker