Tuesday, June 19, 2012

Running a simple performance test

Now that we have our first recorded script, let's run a simple performance test of the Mongoose HTTP server.  I've provided a sample Grinder properties file in C:\Grinder\scripts\grinder.properties. The Grinder Agent knows how to find this file from the DOS environment variable %GRINDERPROPERTIES% set in C:\Grinder\setGrinderEnv.cmd.

In addition, theTCPProxy recorder was configured in C:\Grinder\startProxy.cmd to output its generated Grinder script to C:\Grinder\scripts\grinder_recorded.py. Rename grinder_recorded.py to grinder.py so it matches the name specified in grinder.properties file. The reason for renaming the file is to prevent accidentally overwriting of grinder.py if we decide to do a future re-recording.

Now start the Grinder Console using C:\Grinder\startConsole.cmd:

And then start the Grinder Agent using C:\Grinder\startAgent.cmd. You should see a DOS window showing the Agent starting up and attempted to connect to the Console:

The Agent is now waiting for instruction from the Console, so go to the Action menu and select Start processes. A pop-up window warning that a grinder.properties file has not been selected. You can ignore this for now, and click OK. Your performance test should start and if all is well you should see the TPS counter in the Console displaying the number of requests The Grinder is making to the HTTP server:

After a short while go to Action / Stop Processes to end the performance test. If the Console is reporting errors similar to the following:

then it might be the case that the HTTP server is not running. To confirm if this is the case, click here again to confirm the simple "Hello World" HTML page is being served. If you get an error in your browser, restart the HTTP server by clicking on C:\Grinder\startHttpd.cmd and then check again you can browse to the page. Restart the Agent which was stopped when you chose Stop Processes above, and then rerun the test. You can find out more about the other features of the Grinder Console in the Grinder Console documentation.

Now lets look at the log files that The Grinder generated in C:\Grinder\logs:

There are three log files that the Grinder could possibly create. The Grinder always should create a data file computername-0-data.log, which includes a line of statistics for each HTTP request that was made, such as response time and any client or server errors. If there were any errors during the test you should also see a file computername-0-error.log. This is where any errors are logged, should there have been any during the test.  Finally, the main log file should be created called computername-0.log which contains a log of actions each Grinder thread performed. You can also write custom messages to this log file using the grinder.logger.output facility. You can have a look at the very useful Script Gallery documentation for more examples on how to use this logging facility. The most interesting part of this file is the final statistics summary computed and logged at the end:

             Tests        Errors       Mean Test    Test Time    TPS          Mean         Response     Response     Mean time to Mean time to Mean time to 
                                       Time (ms)    Standard                  response     bytes per    errors       resolve host establish    first byte   
                                                    Deviation                 length       second                                 connection                

(Test 100    10163        0            15.45        5.32         313.82       0.00         0.00         0            0.00         0.00         0.00)         "Page 1"
Test 101     10163        0            15.20        5.16         313.82       45.00        14121.82     0            0.02         1.50         13.87         "GET /"

Totals       10163        0            15.20        5.16         313.82       45.00        14121.82     0            0.02         1.50         13.87        
             (10163)      (0)                                                                                                                               

These statistics are computed from the computername-0-data.log file and provide a summary of which tests were run, any errors, the Mean Test Time (also known as Response Time), average TPS, and lots of other useful summary statistics.

I hope that the last two posts provided a brief overview of recording and execution of a Grinder test script, and how to examine the results of the test run. Next time I plan to show how to develop a RESTful test script that can be used to test an API. This will not use the TCPProxy recording function to generate a Jython script, but instead be coded from scratch and incorporate some third-party Java libraries for parsing the REST responses.

No comments:

Post a Comment