asperfwb Wiki Rss Feed

Updated Wiki: Home

robkerr — Wed, 05 Oct 2011 10:20:52 GMT

Project Description
AS Performance Workbench is a scalability testing application for Analysis Services Cubes. It puts a GUI around monitoring counters, uses multiple threads to simulate high loads, and produces beautiful HTML output documenting your test results.

The goal for AS Performance Workbench is to provide a user-friendly, highly efficient scalability testing and documentation tool for SSAS.

This application has the following features:

The ability to use multiple threads to simultaneously generate query workload in a way that simulates multiple simultaneous application users.
A function to generate query workload in random order across threads to produce a random workload profile consistent with users taking their own path through the application.
Random “think time” intervals when performing “simulated user population” testing.
Functions to collect detailed performance counter statistics and present this information graphically, including annotations on graphs to indicate query failures.
Automatic production of HTML reports suitable for use in interpreting test results and briefing business users on the test outcome.

Additional Documentation: http://asperfwb.codeplex.com/documentation

Watch a demo on YouTube: http://www.youtube.com/watch?v=Kn4-3JTyDXM

Questions: Rob Kerr (http://blog.robkerr.com)

Application Screen Shot

Report Output (HTML)

Updated Wiki: Documentation

robkerr — Wed, 05 Oct 2011 10:20:03 GMT

Key elements in AS Performance Workbench (refer to the screen capture below):

SSAS Server\Instance. Before testing, enter the server\instance to connect to, and select the database for testing from the drop-down list.
The Product Version (SSAS 2005 or SSAS 2008) should be selected, so that correct counter names are used during testing.
The Start Counters / Stop Counters buttons begin and end the counter data collection, which proceeds at a rate of 1 sample/second.
Place all MDX queries to run in the load test in a folder, and enter or select that folder in the Path for Queries field before pressing the Start Test button
The Test Profile controls how queries are submitted to the server
1. All queries on each thread (filename order) - runs all queries on each thread, in order of filename alphabetically. Use to provide a predictable workload for precise comparisons.
2. All queries on each thread (random order) - runs all queries on each thread, in random order on each thread. Use to simulate "real world" scenarios where query order is unpredicatable.
3. Each query on its own thread - runs each query on a different thread in parallel. Use to examin the effect of specific queries running simultaneously.
Threads. This tells the workbench how many simultaneous connections to make to the SSAS server. Depending on the intent of the test, threads can be used to simulate concurrent queries (no think time) or concurrent users (random think time).
Think Time. If think time is entered, each thread will pause for a random number of seconds between the range given before submitting its next query.
Clear Cache. This button will clear SSAS and Windows File Caches immediately. This function requires the ASSP assembly to be installed on the server. See http://asstoredprocedures.codeplex.com/
Counter Stop. Counter data collection will automatically stop this many seconds after all queries have completed.
Start Test/Stop Test. When Start Test is pressed, the test is started. First, Workbench starts the number of threads specified, and opens a database connection on each thread. Next it begins submitting queries on each thread. The test is complete when all queries scheduled for all threads completes (or when the Stop Test button is pressed).
As the test proceeds, counters are monitored and plotted on the Performance Chart, which is updated every second. If a thread detects a query that has failed to complete successfully, a vertical red line is overlaid on the chart.
The lower grid displays the current value of counters, and updates statistics for Average, Max and Min values during the entire test run. Use the Enabled checkbox to select whether to plot a line on the chart.
The Report Path specifies where reports are saved
The Report Title is embedded in the HTML report.
Save Report generates an HTML report for documentation purposes.

The typical workflow to run a stress test is:

Connect to the SSAS instance
Specify the Path for Queries, Test Profile, Threads, and Think Time
Clear Cache
Start Counters (then wait a few seconds to allow them to stabilize)
Press the Start Test button

NOTE: If you're running AS Performance Workbench on the server you're testing, run the application as Administrator. If run without the Administrator SID, you may not be able to read SSAS counters on some systems (such as Windows 7)

New Comment on "Documentation"

aashishthakker — Wed, 06 Apr 2011 22:16:53 GMT

Hi Rob, We have a sample query for one of our cubes that we wish to measure performance for and it keeps getting "timed out". The error message we get is Exception: XML for Analysis parser: The XML for Analysis request timed out before it was completed. Any ideas how/where we can change a timeout setting? NOTE: When we run this mdx directly in BIDS it does work/execute correctly. regards, Aashish

Updated Wiki: Home

robkerr — Tue, 22 Mar 2011 15:38:18 GMT

The ability to use multiple threads to simultaneously generate query workload in a way that simulates multiple simultaneous application users.
A function to generate query workload in random order across threads to produce a random workload profile consistent with users taking their own path through the application.
Random “think time” intervals when performing “simulated user population” testing.
Functions to collect detailed performance counter statistics and present this information graphically, including annotations on graphs to indicate query failures.
Automatic production of HTML reports suitable for use in interpreting test results and briefing business users on the test outcome.

Updated Wiki: Home

robkerr — Tue, 22 Mar 2011 15:38:09 GMT

The ability to use multiple threads to simultaneously generate query workload in a way that simulates multiple simultaneous application users.
A function to generate query workload in random order across threads to produce a random workload profile consistent with users taking their own path through the application.
Random “think time” intervals when performing “simulated user population” testing.
Functions to collect detailed performance counter statistics and present this information graphically, including annotations on graphs to indicate query failures.
Automatic production of HTML reports suitable for use in interpreting test results and briefing business users on the test outcome.

Updated Wiki: Home

robkerr — Mon, 21 Mar 2011 04:57:31 GMT

The ability to use multiple threads to simultaneously generate query workload in a way that simulates multiple simultaneous application users.
A function to generate query workload in random order across threads to produce a random workload profile consistent with users taking their own path through the application.
Random “think time” intervals when performing “simulated user population” testing.
Functions to collect detailed performance counter statistics and present this information graphically, including annotations on graphs to indicate query failures.
Automatic production of HTML reports suitable for use in interpreting test results and briefing business users on the test outcome.

Watch a demo on YouTube: http://www.youtube.com/watch?v=Kn4-3JTyDXM

Questions: Rob Kerr (http://blog.robkerr.com)

Application Screen Shot

Report Output (HTML)