How Many Performance Scenarios?
It’s a simple question, but the answer is that “it depends” (upon the context).
To start to answer the question we need to understand something about the risk:
- What would the impact of performance issues be?
- How likely are these issues to occur?
- What is the risk appetite?
- How much performance risk do you want to mitigate? (i.e. how much time do you want to spend writing, running, analysing, and maintaining scripts)
Understanding Performance Risk
To start understanding and building a model there are more questions to help:
- How complex is your application?
- What is the overall volume of transactions?
- What is the size and “shape” of the data?
- How will the system grow in 1, 2, 5 years?
Some rules of thumb:
- Not more than 20 “simple” scenarios
- Not more than 10 “complex” scenarios
- More than 2
These are just there to try to get the most value from the effort, as performance scripts are generally1 quite brittle and can require a lot of re-work if the application changes frequently.
- depending on the application under test, the tools you use, the time allowed and the person writing the tests ↩