Monday, October 6, 2008

Five Steps to Improved Performance

With the limited time available for tuning, and the extent of improvement that is typically needed, it is vital to focus on those areas that have the highest potential for payoff, rather than to expend effort on improvements that have a negligible overall effect. You may find yourself spending weeks or even months tuning the software without seeing significant improvements. It is usually caused by speculation on the cause of performance problems and a random approach to improvements. The steps below allow you to quickly identify problems, focus your effort on the areas of the software that have the greatest potential for improvement and estimate the relative payoff for potential improvements.

Step 1: Figure Out Where You Need to Be

You should define precise, quantitative, measurable performance objectives. You can express performance objectives in several ways, including response time, throughput, or constraints on resource usage. Some examples are: “The response time for a transaction should be one second or less with up to 1,000 users.” or “CPU utilization should be less than 65% for a peak load of 2,000 events per second.”

When defining performance objectives, don’t forget that your needs may change over the product’s lifetime. For example, your current performance objective may be to process 10,000 events per second. However, in two years, you may need to be able to process 30,000 events per second. It is a good idea to consider future uses of your software so that you can anticipate these changes and build in the necessary scalability.

Step 2: Determine Where You Are Now

Begin by quantifying the problems and identifying their causes. Which uses of the software are causing problems? Typically, these are the most frequent uses. If you don’t have good documentation (like UML sequence diagrams) for these uses, it’s a good idea to do this now. Then, identify the workloads (transaction or customer-call volumes, event arrival rates, and so on) that are causing problems.

Focusing on the architecture provides more and potentially greater options for performance improvement than tuning. By focusing on the architecture, you can identify opportunities for changing what the software does.

Now you will later need to take measurements of the scenarios under operational conditions. This will allow you to understand the system performance parameters such as CPU utilization, I/O rates and average service time, network utilization, message size, and so on. This, in turn, will allow you to identify the bottleneck(s)—the device(s) with the highest utilization. Then, profile the processing steps in the performance scenarios to identify the “hot spots.” The overall impact of improvements gained from addressing these hot spots will be far greater than that of random improvements.

Step 3: Decide Whether You Can Achieve Your Objectives

Before you dive into tuning your software, it’s a good idea to stop and see if you can actually achieve your objectives by tuning. If the difference between where you are now and where you need to be is small, then tuning will probably help. You may even be able to achieve your performance objectives without making changes to the code by tuning operating system parameters, network configuration, file placements, and so on. If not, you’ll need to tune the software.

Performance remedies range from low-cost (and, usually, low-payoff) automatic optimization options (e.g., compiler optimizations) to high-cost (and, potentially, high- payoff) software refactoring or re-creation. Intermediate options are to optimize algorithms and data structures and/or modify program code to use more efficient constructs.

Once you’ve identified the hot spots, some simple calculations will help you decide if you can meet your performance objectives by tuning the software. If not, you may need to explore other alternatives, such as adding more processors. If it is, then you can proceed with confidence as you tune the software. You might also create software models that depict the end-to-end processing steps for performance scenarios. The models will help you quantify the effects of more complex solutions, particularly those that involve contention for resources.

Step 4: Develop a Plan for Achieving Your Objectives

Once you have identified potential remedies, you can rank them based on their payoff. Then apply them starting with those that have the highest payoff. You determine the quantitative performance improvement by estimating the new resource requirements.

Create a list of options that compares the performance improvement versus the costs to determine the best candidates. Note that you may not always choose the option with the best performance improvement because other factors may make it less desirable. For example, the option with the best performance improvement may be risky because it uses an unfamiliar new technology. Or, a change with a high performance payoff may negatively impact another important quality such as maintainability or reliability. You may elect to correct problems with hardware. Be sure to use the models to evaluate this option too. Sometimes the problem is caused by software and no amount of hardware will correct it.

Step 5: Conduct an Economic Analysis of the Project

Upon completion, gather data on the time and cost for the performance analysis, time and for software changes (include coding and testing), hardware costs if applicable, software distribution costs when applicable, and all other costs of the project. Then gather data on the effect of the improvements. These may include savings due to deferred hardware upgrades, staff savings (ie. If fewer staff members are required to use the software), revenue increases (ie. when you can increase the number of orders processed), and so on. It is also a good idea to review the changes made to see if they could have been predicted earlier in the software’s life. If your problems could have been prevented, compare the costs and benefits of detecting and correctingproblems earlier versus detecting them after problems occurred. Then use this data on your next project to determine whether it is worthwhile to spend a little more effort during development to prevent problems.

Cost-Benefit Analysis

The process involves, whether explicitly or implicitly, weighing the total expected costs against the total expected benefits of one or more actions in order to choose the best or most profitable option. Using this process can provide a high payoff for a relatively small cost.

An Ounce of Prevention

Once you run into trouble, tuning the software is likely to be your only choice. However, it’s important to realize that a tuned system will rarely, if ever, exhibit the level of performance that you could have achieved by designing in performance from the beginning. The key to achieving optimum performance is to adopt a proactive approach to performance management that anticipates potential performance problems and includes techniques for identifying and responding to those problems early in the process. With a proactive approach, you produce software that meets performance objectives and is delivered on time and within budget, and avoid the project crises brought about by the need for tuning at the end of the project.

Note: This content is cut down version of the following:
Five Steps to Improved Performance by Lloyd G. Williams and Connie U. Smith

No comments: