Andy Grove's Blog

I've been researching some ideas in my spare time over the past couple of weeks where performance is critical so I decided to put together a framework for easily running performance and scalability tests. The idea is to provide a framework that is as easy to use as JUnit and that encourages performance testing as an integral part of the development process, rather than as an afterthought before a product release or deployment.

I've called the framework JPerf and I have made the code available on sourceforge - it's just a few classes at the moment but I will be evolving this as and when I need the functionality myself.

The key interface is PerfTest which provides setUp(), test() and tearDown() methods. The test runners call setUp() once and then repeatedly call test() for the specified period of time and then call tearDown() at the end to free up resources, such as database connections.

There are currently two test runners - PerfTestRunner and ScaleTestRunner, which measure performance and scalability, respectively. Here's some sample output from the example that ships with JPerf.

PerfTestRunner:

[java] org.jperf.example.SimpleDateFormatTest: 243,971 iterations in 500ms (486,968 iterations per second)
[java] org.jperf.example.MyDateFormatTest: 371,968 iterations in 500ms (742,451 iterations per second)

ScaleTestRunner:

[java] With 1 client threads, throughput is 100.0 tps and memory usage is 5137656
[java] With 2 client threads, throughput is 202.0 tps and memory usage is 5175144
[java] With 3 client threads, throughput is 300.0 tps and memory usage is 5223616
[java] With 4 client threads, throughput is 402.0 tps and memory usage is 5271592
[java] With 5 client threads, throughput is 504.0 tps and memory usage is 5330056
[java] With 6 client threads, throughput is 602.0 tps and memory usage is 4888336
[java] With 7 client threads, throughput is 702.0 tps and memory usage is 4979288
[java] With 8 client threads, throughput is 798.0 tps and memory usage is 5057864
[java] With 9 client threads, throughput is 900.0 tps and memory usage is 5137568
[java] With 10 client threads, throughput is 1000.0 tps and memory usage is 5215200


I'd be interested to hear ideas on what features people would like to see in this framework to make it useful for their projects. If you have any requirements please leave a comment here or post a feature request on sourceforge.

Labels: java, performance