Assessing Technology ROI Vendors

thePoint
August 8, 2005

Return on Investment (ROI)—once an economic analysis tool used only by financial managers—is an increasingly critical metric for determining the value of a technology investment. It is no longer sufficient for technology vendors to differentiate themselves solely on features, packaging, performance, and price.

ROI justification has become a crucial part of the sales process, as customers now demand a compelling economic argument before making a purchasing decision.

In a white paper aimed primarily at technology vendors, The Gantry Group LLC provides guidelines for credible ROI calculations. The paper also helps to demystify the ROI process from a customer perspective. It includes:

• basic building blocks of ROI profiling
• explanations of certain terms and algorithms
• data collection methods
• pitfalls to avoid
• guidelines for sound ROI methodology

Changing Times

In "the good old days" customers were hungry for exciting new technologies and anything Internet. Capital investments were made on the mere insertion of an important keyword like Java, Internet, or CRM in the business plan. Many of these investments were made without forecasting whether the value created by the purchase exceeded the total cost of ownership. In many purchasing situations, not enough value was created to offset—or even rationalize—the expenditure.

Now it's a different ball game. Potential customers responsible for the success of a business need good reasons to buy a product. Good reasons are those that quantify:

• how the offering will positively effect a set of business metrics essential to judging the performance of your enterprise
• how long after purchasing the product will you recoup your investment

Vendors used to be able to close sales based on competitive differences, positioning, and functionality. But these are no longer the primary decision criteria used to approve a capital outlay. Instead customers are asking:

• What is the quantitative value of what you're proposing to me?
• How does my business benefit from this in terms of ROI?
• How does your offering compare with alternative solutions in terms of ROI?
• How long will it take my business to recoup the investment?

The high-tech sector is getting the message. Gantry Group recently conducted an online study to benchmark the state of the Massachusetts High Tech Economy, which showed that 60% of technology vendors indicated that selling products or services based on ROI is the most important step they are taking to combat the effects of economic downturn on their business.

As one IT vendor put it, "I don't think we can take [value claims] at face value any more. We have to be very precise about how to show that value and prove that value and then treat it as a true investment proposition."

The Tower of ROI Babble

With acronyms like "ROI", "TCO", and "Payback Period" being bandied about by buyers and sellers alike, it's important to develop a robust analytical approach to determine the value of a vendor offering. In doing so, you will gain a clear understanding of the best methods for measuring their offering's economic value.

Return on Investment (ROI). This is a financial term for the economic gain directly associated with a particular capital outlay. It's a calculation of how much money comes back (or is not spent) for every dollar invested. Essentially ROI is nothing more than the sum of all the cash flows associated with a specific project or initiative, over some defined period of time. Simply stated:

Dollar ROI = Total Benefit – Total Investment

where:

Benefits = Increased Revenues + Cost Savings + Avoided Costs

Total Investment = Custom Development/Integration Costs + Equipment/Connectivity Infrastructure Costs + Ongoing Maintenance/Support/Training

Increased Revenues. These are dollars directly attributable to the technology deployment. Such increases, for example, can be derived from increased sales leads, higher close rates, shortened sales cycles, and higher repeat sales.

Cost Savings. This money is derived from savings associated with reduced outsourcing requirements and lower administrative costs that are the direct result of the application deployment. Labor reductions that do not result in staff eliminations are not included in the ROI calculations as the affected employees are simply redeployed within the organization.

Avoided Costs. These are derived from elimination of staff, shipping, printing, transportation costs, and other components of the business workflow resulting from the technology deployment.

Investment. This term is just another way of expressing Total Cost of Ownership (TCO).

Total Cost of Ownership (TCO). TCO is a vital ingredient to any rigorous ROI calculation. Customers wants to know what the economic benefits are AFTER they subtract out the cost of the offering, which is not just the initial purchase price.

TCO calculation requires a vendor to work closely with customers to discover underlying and ongoing cost drivers that may not be apparent on the surface. A technology product for example, may require new infrastructure investments and the hiring of new skills that its operation may require. On the other hand, infrastructure requirements that are already in place should not be included in the TCO.

Consider also new business processes that must be put in place to accommodate a new system may require training and support. Finally, the lifetime of some technologies must be factored into TCO to reflect the replacement and repair cost of new units when old ones fail or break.

While the mathematics behind ROI are seemingly straightforward, there's nothing simple about identifying all the different ways that costs and revenues are determined within an enterprise. Cost savings can arise out of myriad categories peculiar to the industry and business application. Costs can be one-time and ongoing, and all of these categories have both tangible and intangible components. Once the business metrics surrounding benefits and investments have been framed, the ROI calculation can be performed.

There are also several important metrics for reporting ROI:

Percent ROI is another way to express the financial return. When you see an ROI expressed as a percentage, this is simply the ratio of Total Benefits divided by Total Investment over a given time period. (Don't assume that percent ROI is equal to the benefits divided by only initial costs. Usually a technology purchase has associated ongoing costs that must be factored in.)

Payback Period computes the time period required, beginning at time of investment, to recoup the invested dollars as a direct benefit of deployment. Some companies use a start date of solution deployment, but this does not take into account the amount of planning and pre-installation work that usually needs to be done prior to installation of high-ticket technology. The costs of employee training, installation and testing should also be considered as a factor.

Net Present Value is a financial term for the value of a series of yearly cash flows discounted back to today's dollars by removing the effects of inflation. NPV is useful when calculating ROI, as it can be applied to the net yearly benefits associated with the project.

ROI Risk and Probability must be assigned to an ROI figure for it to be meaningful. One way to deal with risk profiles is to require anticipated ROI for the best, worst, and most likely scenarios.

Data Hunting and Gathering

Sound ROI models are not something that vendors can knock off in a couple of hours. From the vendor standpoint, the model must be analytical, explainable, and defensible. It must be built on a strong foundation of business metrics that are realistic, reasonable, and relevant to your business and industry. The major flaw in most ROI and TCO modeling exercises is lack of accurate, quantifiable data from the customer. Vendor analysis based on interviewing one or two customers for anecdotal impressions about product impact is insufficient.

Vendors who present compelling ROI cases must invest the time and capital resources to query, confirm, and validate their customer's needs around payback and value calculations. Don't accept figures based on sketchy calculations, laced with a measure of secondary data complied by an industry "authority.” Instead, require specific payback estimates validated against a statistically representative set of real customer installations running in specific business environments similar to your own.

In order to develop a strong ROI case vendors must ask questions such as:

• What is the minimally acceptable payback?
• What is the acceptable payback period?
• What are the expected business goals?
• What directly measurable metrics are required?
• What indirect metrics and associated assumptions are required?

ROI Pitfalls

Any ROI calculation that vendors perform on your behalf should rightly be met with skepticism. After all, would the vendor show them to you if the ROI put their offering in a neutral or bad light?

Developing an ROI selling strategy is only as good as the tools vendors use. Their ROI calculator must stand up under your scrutiny, with each assumption and algorithm being defensible. Here are some of the areas where ROI tools can run astray:

Objectivity. Does the vendor hire an independent third party to conduct the research and development of their ROI tool? A truly impartial third-party firm will insist on performing systematic, objective research with customers to identify the correct business metrics used in the tool and will tell vendors up front if the ROI is not what they had hoped for. As a customer, you will only find the vendor credible if you believe their ROI tool is measuring the right things and that it hasn't been "fixed."

One size doesn't fit all. Another common pitfall to watch for is the Website ROI calculator that measures ROI for all companies – no matter what industry. This is flawed since the business performance metrics will be different in different industries. And even if the business metrics are the same, they are generally not defined similarly. Vendors need a separate ROI calculator for each unique industry.

Measuring the unmeasurable. It's a scary thought, but some well-known analyst firms claim to be able to quantify intangible costs and benefits. These parameterizations are generally supported by "domain expertise" and the firm's unique ability to make accurate assessments. Vendors and customers alike need to reject this approach. If a cost or benefit cannot be directly measured as an attribute of the product, don't try to force it.

Choosing the payback period. The payback period, or time horizon implicit in an ROI calculator, is critically important to the overall results. While the general rule is to use a three-year time horizon, this depends on a variety of factors that should be considered for each situation. A vendor offering is all the more valuable if it shows attractive ROI using conservative assumptions.

Double-counting benefits. This is an easy pitfall to inadvertently trip over. One example is calculating the hypothetical ROI impact to include savings from staff reductions plus increased revenues, when it's actually one or the other. Double-counting a benefit can result in inflated ROI results.

Reality testing. Vendors need to analyze the results of ROI modeling as part of their ongoing quality assurance. ROI is more often overstated than understated. Better for you, the customer, to work with vendors who adequately test the ROI tool.

Conclusions

ROI modeling requires systematic application of sound methodology. The following steps will help ensure that ROI analysis produces credible numbers. Vendors should:

• develop a solid modeling tool that can be used on a customer-by-customer basis to calculate tailored ROI
• revisit ROI at the end of the implementation period to measure actual performance vs. expected performance (using consistent metrics)
• employ reasonable cause-and-effect analysis to quantify intangible benefits and costs
• base models on thorough analysis of each major customer segment to understand all aspects of costs and benefits
• employ primary market research, not secondary generalizations or assumptions

In the art and science of projecting ROI, it's to everyone's advantage to remain objective. Any vendor who gains a reputation for promoting misleading numbers will find it tough to recover.

This article was prepared by the staff at the Point for Credit Union Research and Advice and is published online at http://thepoint.cuna.org/. Reprinted with permission.