AS Performance Workbench

Created Issue: MDX Query multiple execution issue [1498]

SivaKrishna — Sat, 16 Mar 2013 14:55:43 GMT

HI All,

Thanks for such great tool....Currently we are facing issue related to Query multiple executions and informed on this codeflex forum...but no one is look into this to resolve

Guys if you already have solution for this...Could you please share us/post the code....

Thanks

New Post: Multiple Executions Issue

SivaKrishna — Tue, 12 Feb 2013 07:33:51 GMT

Hi Rob,

Thanks for providing the such a great tool...

But we had some issues with the tool (Single MDX query triggered 2 times on the server), first this issue was raised on October 5 2012 and we hope that has been solved.

Please let me know the solution or any Patch.

Commented Issue: Multiple executions issue [1103]

SivaKrishna — Tue, 12 Feb 2013 06:41:48 GMT

Great tool however is seems to execute the MDX query 2x number of threads.
E.g. Threads = 1; Test Profile = All queries on each thread (filename order)
executes each MDX file 2 times

(the thread index may be starting from 0 and we are passing 1?)

Thanks,
Rob
Comments: ** Comment from web user: SivaKrishna **

Hope fix has been found for this issue?

Commented Issue: The XML for Analysis request timed out [1370]

Minnie — Mon, 10 Dec 2012 02:37:38 GMT

Hi,

Can someone tell me what could have caused these exception in the Result Report. e.g. below is one of my MDX report output in the workbench, and it can be run in SSMS; the profiler shows it run 52572 (duration column, millisecond); all my 9 queries show run successfuly in sql profiler; but each one of them shows the "timed out exception" in the performance Workbench.

"Exception: XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. Server: The operation has been cancelled. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed."

when I run the same group of test for 2nd time (did not clear the cache), some queries did not have the exception. and I noticed all of them run below 30,000; so I think this is related to "ForceCommitTimeout" SSAS property because I noticed the value for the property is 30,000; but I am not certain.

How the result report shall be interpreted in this error exception? Shall we change the property to be a lot larger value if it is related to it?

Thank you.
Comments: ** Comment from web user: Minnie **

As mentioned in the discussion (TAB), I've uploaded the solution for "time-out" issue and the new solution also includes SSAS 2012 counters.

http://asperfwb.codeplex.com/SourceControl/list/patches

New Post: Time out exception

Minnie — Mon, 10 Dec 2012 02:36:01 GMT

Hi,

In case anybody encountered the same issue, my colleague had made changes on the source code. please see the patch uploaded below; it also allows SSAS 2012 performance counters

http://asperfwb.codeplex.com/SourceControl/list/patches

Regards,

Min

Patch Uploaded: #13423

Minnie — Mon, 10 Dec 2012 02:33:47 GMT

Minnie has uploaded a patch.

Description:
My colleague had made changes to the solution/project to do below:

1. fix up connection time-out error "XML for Analysis parser: The XML for Analysis request timed out before it was completed." it allow 1 hour now by default

2. Additional 20 counters which can be viewed in the report and/or on chart
support monitoring for Analysis Services 2012 (Connect to SSAS Server/Product Version)

Created Issue: The XML for Analysis request timed out [1370]

Minnie — Thu, 29 Nov 2012 03:51:59 GMT

New Post: Time out exception

Minnie — Thu, 29 Nov 2012 03:16:01 GMT

Hi,

Not quite sure this product is still being supported, if yes, can someone tell me what could have caused these error. e.g. below is one of my MDX report output in the workbench, and it can be run in SSMS; the profiler shows it run 52572 (duration column, millisecond)

"Exception: XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. Server: The operation has been cancelled. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed. XML for Analysis parser: The XML for Analysis request timed out before it was completed."

Thank you.

Created Issue: "Think time" settings [1105]

kosteckir — Fri, 05 Oct 2012 14:31:33 GMT

The documentation reads "Think Time. If think time is entered, each thread will pause for a random number of seconds between the range given before submitting its next query."

However it is set by default and does not allow value 0 = "don't think", which could come quite useful with testing a small query run multiple times.

Thank you for the fantastic work on this and my apologies for the little contribution (just raising rather than fixing the challenges) unfortunately I am flat out on another assignment at the moment.

Many thanks,
Rob

Created Issue: Error handling with the Clear Cache [1104]

kosteckir — Fri, 05 Oct 2012 14:20:06 GMT

We have had an issue on the server the ASSP library used for the Clear Cache. The dll was not installed properly and was not clearing the cache (error message was appearing in the profiler during the operation) however the information from the app reads "ASSP.ClearAllCaches() completed" which is a bit misleading.

Not a major issue as long as you are aware.

Many thanks,
Rob

Created Issue: Multiple executions issue [1103]

kosteckir — Fri, 05 Oct 2012 14:13:24 GMT

Patch Uploaded: #12964

samdani — Tue, 02 Oct 2012 20:52:04 GMT

samdani has uploaded a patch.

Description:
bjjb

Source code checked in, #22735

Project Collection Service Accounts — Mon, 01 Oct 2012 21:40:24 GMT

Upgrade: New Version of LabDefaultTemplate.xaml. To upgrade your build definitions, please visit the following link: http://go.microsoft.com/fwlink/?LinkId=254563

Source code checked in, #22734

Project Collection Service Accounts — Mon, 01 Oct 2012 21:38:54 GMT

Checked in by server upgrade

New Post: SSAS 2012

Acius — Thu, 16 Aug 2012 04:31:59 GMT

Hi,

Any thoughts of a SSAS 2012 version of AS Performance Workbench?

Regards

New Post: Time out exception

samdani — Mon, 02 Jul 2012 06:22:47 GMT

Hi Rob,

I too have concern about this exception, could you please let us know how to enhance this setting and where.

Or let us know if there is any patch available for customising request time out setting.

Thanks in Advance

-Samdani

Updated Wiki: Home

robkerr — Wed, 05 Oct 2011 10:20:52 GMT

Project Description
AS Performance Workbench is a scalability testing application for Analysis Services Cubes. It puts a GUI around monitoring counters, uses multiple threads to simulate high loads, and produces beautiful HTML output documenting your test results.

The goal for AS Performance Workbench is to provide a user-friendly, highly efficient scalability testing and documentation tool for SSAS.

This application has the following features:

The ability to use multiple threads to simultaneously generate query workload in a way that simulates multiple simultaneous application users.
A function to generate query workload in random order across threads to produce a random workload profile consistent with users taking their own path through the application.
Random “think time” intervals when performing “simulated user population” testing.
Functions to collect detailed performance counter statistics and present this information graphically, including annotations on graphs to indicate query failures.
Automatic production of HTML reports suitable for use in interpreting test results and briefing business users on the test outcome.

Additional Documentation: http://asperfwb.codeplex.com/documentation

Watch a demo on YouTube: http://www.youtube.com/watch?v=Kn4-3JTyDXM

Questions: Rob Kerr (http://blog.robkerr.com)

Application Screen Shot

Report Output (HTML)

Updated Wiki: Documentation

robkerr — Wed, 05 Oct 2011 10:20:03 GMT

Key elements in AS Performance Workbench (refer to the screen capture below):

SSAS Server\Instance. Before testing, enter the server\instance to connect to, and select the database for testing from the drop-down list.
The Product Version (SSAS 2005 or SSAS 2008) should be selected, so that correct counter names are used during testing.
The Start Counters / Stop Counters buttons begin and end the counter data collection, which proceeds at a rate of 1 sample/second.
Place all MDX queries to run in the load test in a folder, and enter or select that folder in the Path for Queries field before pressing the Start Test button
The Test Profile controls how queries are submitted to the server
1. All queries on each thread (filename order) - runs all queries on each thread, in order of filename alphabetically. Use to provide a predictable workload for precise comparisons.
2. All queries on each thread (random order) - runs all queries on each thread, in random order on each thread. Use to simulate "real world" scenarios where query order is unpredicatable.
3. Each query on its own thread - runs each query on a different thread in parallel. Use to examin the effect of specific queries running simultaneously.
Threads. This tells the workbench how many simultaneous connections to make to the SSAS server. Depending on the intent of the test, threads can be used to simulate concurrent queries (no think time) or concurrent users (random think time).
Think Time. If think time is entered, each thread will pause for a random number of seconds between the range given before submitting its next query.
Clear Cache. This button will clear SSAS and Windows File Caches immediately. This function requires the ASSP assembly to be installed on the server. See http://asstoredprocedures.codeplex.com/
Counter Stop. Counter data collection will automatically stop this many seconds after all queries have completed.
Start Test/Stop Test. When Start Test is pressed, the test is started. First, Workbench starts the number of threads specified, and opens a database connection on each thread. Next it begins submitting queries on each thread. The test is complete when all queries scheduled for all threads completes (or when the Stop Test button is pressed).
As the test proceeds, counters are monitored and plotted on the Performance Chart, which is updated every second. If a thread detects a query that has failed to complete successfully, a vertical red line is overlaid on the chart.
The lower grid displays the current value of counters, and updates statistics for Average, Max and Min values during the entire test run. Use the Enabled checkbox to select whether to plot a line on the chart.
The Report Path specifies where reports are saved
The Report Title is embedded in the HTML report.
Save Report generates an HTML report for documentation purposes.

The typical workflow to run a stress test is:

Connect to the SSAS instance
Specify the Path for Queries, Test Profile, Threads, and Think Time
Clear Cache
Start Counters (then wait a few seconds to allow them to stabilize)
Press the Start Test button

NOTE: If you're running AS Performance Workbench on the server you're testing, run the application as Administrator. If run without the Administrator SID, you may not be able to read SSAS counters on some systems (such as Windows 7)

New Post: Time out exception

aaditya2000 — Thu, 22 Sep 2011 22:13:42 GMT

Hello:

My MDX query is runing fine in SSMS, but when i run in AS Performance Workbench, i am getting following error\exception.

Exception: XML for Analysis parser: The XML for Analysis request timed out before it was completed.

Any help is greatly appreciated. thanking in advance.

Thanks

New Post: Connection String for Server

dowcampb — Tue, 06 Sep 2011 14:12:48 GMT

Would be possible to change the connection string to the server? For example, I would like to include "Roles=SomeSecurityRole" into the connection string. This would enable me to test performance with different security roles.