Do you use an RSS reader? Do you have a MyYahoo! or personalized Google page?

For these and more you can have SCDigest delivered right to your personal pages, all week long.

You can subscribe to our RSS feeds in two ways:

1. Copy our RSS link into your RSS reader - it's easy! www.scdigest.com/rssfeeds.xml
   
   
2. Click on a button below to quickly add it to your favorite readers.

We received quite a few letters on our First Thoughts piece on "The Probability of Supply Chain ROI," which described a much different approach, using probability analysis, for estimating the financial attractiveness of supply chain projects.

But we only publish one this week, because it involves an interesting exchange between Richard Cushing, a business executive and blogger, and SCDigest Editor Dan Gilmore. We publish it in its entirety below - and we think you will enjoy it.

Feedback of the Week - On the Probability of Supply Chain ROI:

I wrote a response to your article on ROI “probabilities” and you’ll find it below.

I’d appreciate hearing your commentary on it.  Thanks.

Richard D. Cushing

Dan Gilmore wrote in “The ‘Probability’ of Supply Chain ROI” propounds properly and rationally the fact that any “forecast,” including forecasts of ROI (return on investment) should not be a single number. Rather, as anyone properly trained in statistical methods will tell you, it should be a range of numbers. The range of numbers would generally be calculated based on a single calculated value plus and minus values that represent the confidence intervals or, simply put, how likely the statistician believes his estimates the calculates will approximate reality. A larger range indicates lower levels of confidence and a smaller range higher confidence levels.

Now, while Gilmore is mathematically correct, the fact remains that most small-to-mid-sized businesses (SMBs) simply do not have anyone trained in statistics on their payroll and they are not likely to go out and hire a statistician to produce ROI forecasts for their IT projects – since this would, by definition, automatically reduce the ROI of the enterprise as a whole in the short term.

Gilmore makes another comment in his article with which I wholeheartedly agree: “[T]here is some evidence that companies are in fact looking at investments that can help them to get back on a growth trajectory (read: increasing Throughput) without having to add much in the way of head count (read: Operating Expenses) by achieving productivity gains.” Given the world-wide economic malaise that is showing some signs of lessening (for the moment, at least), Gilmore’s description probably suits the vast majority of SMBs across the U.S. and beyond.

Furthermore, many others besides me have written that a firm stand on return on investment will be the hallmark of technology spending in the 2010 and beyond. So, I can hardly fault Gilmore for suggesting that SMB executives and managers need to become increasingly sensitive to and realistic about ROI for every kind of investment in their firms’ futures.

Despite my agreement with Gilmore on theoretical grounds regarding forecasts – including ROI forecasts; and despite my agreement with him regarding the goal of companies to get back on a growth trajectory through wise investment of capital resources, I must disagree with him on the matter of adding useless complexity to the return on investment forecasting process.

Allow me to explain why I use the harsh term “useless” to describe such an effort in the development of a ROI forecast for an IT project.

First  of all, let me say that statistical methods ought to be applied where they make sense. Statisticians generally agree that a valid statistical sample must contain at least 30 members. This works great where you have 30 dogs, 30 cows, 30 houses, 30 automobile, 30 miles of roadway, and so forth for comparison. Then, of course, you need to factor for environmental differences. Thirty or more cows all in the same pasture, eating the same foods, and enjoying the same climate would make a pretty good statistical sample for some studies of cows. On the other hand, three Holstein cows in northern Minnesota, two long-horns in west Texas, 15 black whiteface cows in eastern South Dakota, and ten mixed-breed cows in central Florida are not likely to constitute a good “sample” for cow studies.

Why?

Simply because there are too many environmental dissimilarities surrounding the cattle. By the time these factors were accounted for, (generally speaking) any results would have such a large confidence interval as to make any prediction almost meaningless.

When considered as a whole, a typical SMB has tens of thousand of variable at work within the enterprise. Any number of those variables are likely to dramatically separate it any “sister” enterprises in a sample group used to forecast ROI outcomes.

Of course, the fact that traditional ERP – Everything Replacement Projects – are going to affect the whole enterprise is a big part of the problem of predicting ROI outcomes. With tens of thousands of variables at play, picking the winning number is far more challenging than winning the lottery.

First of all, a good many SMBs today have a “pretty good” ERP system in place – regardless of its brand. Unless there is some pressing reason to undertake a traditional ERP – Everything Replacement Project, it is probably a far better idea to consider a New ERP – Extended Readiness for Profit project instead.

Narrowing the scope of the project reduces the complexity. And, reducing the complexity increases the likelihood that your ROI forecast will be more on-target. Allow me to give you a couple of examples:

• Warehouse automation – Suppose your Current Reality Tree (CRT) shows you that, in order to increase Throughput, you need to enable your warehouse to be able process 80 to 100 shipments per hour, rather than the present 30 to 40 per hour. Furthermore, you need to do this while holding the line on Operating Expenses.
  
• Business intelligence – After reviewing their Current Reality Tree and laying out a Transition Tree (TrT), your marketing and sales team has assured you that, if they are provided tools with which to better understand customers and markets, they will be able to segment the market and create UROs (un-refusable offers) and increase Throughput by 130% in two years. No additional staff or other increases in Operating Expenses would be required.

If your executive management team were to elect to pursue either of these projects – or both – the goals are specific and measurable – as would be the expected outcomes. ROI calculations become simple:

Where T = Throughput (Revenues less Truly Variable Costs), OE = Operating Expenses, and I = Investment.

Simple. Elegant. And ROI calculations are far more likely to be right than any calculation around traditional ERP – Everything Replacement Projects.

Dan Gilmore Responds:

I will just say, having been involved in ROI stuff for years, that:

1. It's not just SMBs that lack the stat resources = big companies too

2. I don't agree very much about adding complexity  - you have to see Doug Hubbard in action. But I do agree it does add some complexity - more mental than actual, if that makes sense. It's just not what anyone is used to.

But, it is fundamentally better. Where I think your disagreement errs is that from a practical perspective, ROIs in practive are all over the place, and rarely hit the projected number. In fact, for ERP, the damn solutions and process are so complex the idea that you calculate a single expected number is almost ludicrous - and no one ever goes back to actually calculate an  real ROI. The only attempt I know was when I was at META Group in 1999, before it was bought by Garnter, which found on average a negative ROI from ERP (but you can bet the ROIs all showed the investment well meeting hurdle rates). 

Doug Hubbard has a way of developing confident intervals that I think offer a more more rational way of dealing with this reality. For example, going back to ERP, what is the likelihood that the expected consulting costs will be 50% greater than what is in our ROI projections? Based on a variety of inputs, honing in on those probabilities is actually not that hard.

Big insurance companies now use this model routinely for large IT investments, because it is the natural way for them to think. Dell did or does do it. Once it is baked into how you do analysis, the perceived complexity goes away, and it is for more revealing about the assumptions that go into the analysis.

I am not a statistician, but I can tell you from being involved in many "one number" analysis, mostly as a software vendor, they are largely baloney.

Cushing Responds:

Well, the simple answer is that making your ROI estimate a range rather than a single number increases the likelihood that you may hit – or at least be nearer – your target.  It, in effect, means that you have simply made the target larger.

Great! But like the archer that has trouble hitting the bulls-eye, simply making the bulls-eye three times larger does not make the archer more accurate. It simply increase the likelihood of his hitting the (now much larger) bulls-eye.

Actually, however, my point is NOT the accuracy of the ROI estimate. Most business executives with whom I have worked would be pleased to have assurance of a positive ROI, even if they did not calculate the number in advance, or if they did and missed the mark by some significant factor.

The problem is that far too many technology projects are undertaken with only the vaguest of notions about and worthless rules-of-thumb ROI calculations from the vendor or VAR.  I’d be happy if more executives and managers took time to actually figure out WHAT NEEDS TO CHANGE, WHAT THE CHANGE SHOULD LOOK LIKE, and HOW TO EFFECT THE CHANGE (specifically, how their technology spend will actually contribute to effecting the necessary change). They need to know how the change will affect THROUGHPUT, OPERATING EXPENSES, and what the INVESTMENT will be to do so.

My only point is that adding complexity to the calculation of ROI – like enlarging the bulls-eye – does not necessarily add value to the ultimate goal and that is, I believe, to assure (inasmuch as possible) a positive ROI for every investment and to help assure that the RIGHT projects are undertaken where differing projects are likely to result in differing ROIs.

As I said, I appreciate the mathematician’s desire to have a range for theoretical reasons, but the theory adds little value where the rubber meets the road, in my opinion.

Thank you for the thought-provoking article and conversation.