How Many Performance Scenarios?

It’s a simple question, but the answer is that “it depends” (upon the context).

Risk

To start to answer the question we need to understand something about the specific risks for your project/product:

• What would the impact of performance issues be?
• How likely are these issues to occur?
• What is the risk appetite?
  • How much performance risk do you want to mitigate? (i.e. how much time do you want to spend writing, running, analysing, and maintaining scripts)

Understanding Performance Risk

To start understanding and building a model there are more questions to help:

• How complex is your application?
• What is the overall volume of transactions?
• What is the size and “shape” of the data?
• How will the system grow in 1, 2, 5 years?

Some “Rules of Thumb”:

• More than 2
• Fewer than 20 “simple” scenarios
• Fewer than 10 “complex” scenarios

The rational for these are to try to balance the effort and benefit. This also assumes that the performance scripts you will create, like most¹, are quite brittle and will likely require a lot of re-work if the application changes frequently.