The "Probability" of Supply Chain ROI 2023

I am on vacation this week, so I am publishing again one of my favorite First Thoughts columns of all-time, on measuring ROI from supply chan investments, especilly around new technology.

It's from all the way back in 2010, but just as relevant today. So here we go:

Here may be a bit of a surprise – the way most of us have thought about ROI calculation is not right, one expert says.

The way most of us in both business and at solution providers have been taught to calculate ROI is to add up the costs of the project and line them up against the financial benefits over some period of time. That leads to any one of a number of methods to determine ROI, whether it is a simple payback period calculation, internal rate of return, discounted cash flow, etc.

There is much to discuss even with these approaches (for example, what should be included in both cost and benefit categories?). But for today's purposes, I will note all these usual methods have something in common: the end result is usually a single number, such as an internal rate of return expected to be 22.5% or something like that. That number must pass a company's given ROI threshold in any particular budget cycle to be funded.

But, as I have learned from a man named Doug Hubbard of Hubbard Decision Research, it is actually somewhat silly to think about ROI as a single number, for all kinds of reasons.

It is instead much better to think about ROI for a project as a probability curve. The absolute reality is that there are a range of potential financial outcomes that are possible for a given initiative.

Understanding those probabilities for potential success or less than success is critical to considering ROI. Some investments, for a variety of reasons, are more risky in terms of achieving the expected return; others are less so.

As just one example of this concept, Hubbard's long years of doing work in IT-related projects have shown that internally developed software has a greater probability of failing to meet expectations than projects using commercial packages. You can agree with that assessment or not, but assuming for a minute this is true, what it says is that for a given project that involves a make or buy decision, you could have options that both have nearly identical "one number" ROIs (say both with internal rates of return of 24%), but which would have very different probabilities of achieving that number. The internally developed systems have more risk.

As shown in one of our Supply Chain Graphics of the Week, what seems to me the real power of this type of analysis is that you can understand the investment and its range of potential outcomes at a much more accurate and interesting level.

For example, there is the obvious analysis of estimating the likelihood that a given investment will achieve its expected return, which we can also think of as "the most likely" result.

Further, by using this approach a company can also estimate the probability that the project will exceed the company's hurdle rate, or the likelihood that it will at least have a positive return. A company may be more likely to approve a project where the likelihood of at least achieving a positive return is high, as another example of how this can work.

These kinds of analyses can help better understand the risk-reward trade-off of potential investments, and even lead to rethinking some aspects of a potential project to reduce risk and shift the curve to the right, if you will.

As an easy example, high levels of supply chain software customization unquestionably add risk. In the past, that risk often involve outright failure; today, it is more often related to problems that lead to delays in getting the software to work right and therefore to start achieving positive cash flow. So, even if the theoretic return is higher with the customizations, the trade-off may look much less favorable when the risk is considered. While we may all understand that intuitively, this approach offers a better way to quantify that.

This really is a better way. Investments do have a probability curve, as both logic and real world outcomes makes clear (meaning the actual ROIs for any type of supply chain investment vary all over the place across companies). The likelihood of well exceeding the most likely ROI is usually small, so the curve usually starts to drop off pretty rapidly to the right of that number. How the curve looks to the left is often the key.

Of course, it takes some skill to develop this approach, but it is not as hard as you may think. Hubbard has a specific method he uses to determine the likelihood of various outcomes, and then uses a Monte Carlo simulation to develop the curve (Monte Carlo simulation is very brief summary is a method that uses a sort of random number generation to simulate hundreds or thousands of occurences).

It would require more space than I have hear to do real justice to this, but for example, let's say in a transportation management system (TMS) project, your confidence in reducing total logistics headcount through efficiency gains may be higher than it is over how much you will save from load optimization, or whatever. This insight is used to measure and build the probability model. One has a confidence factor of 80%; the other, maybe 50%. This kind of info can be used to generate an overall probability curve.

It's unquestionably a more complicated approach than the traditional methods for ROI calculation. Some managers and/or their companies will find a "one number" approach is just more suited to their temperament and liking. This approach requires nuance; the other more "certainty." Clearly, the ROI models and methodologies for ROI calculation used by supply chain technology vendors are almost universally "one number" oriented, and those often provide a basis for the company's own calculations.

Companies sometimes utilize a mini-version of this approach in a sense, such as presenting three scenarios for ROI and/or cost savings; often, this involves presenting "most likely," "best case" and "worst case" scenarios. This is better than just one number, I suppose, but it begs the question: What is the likelihood of each possibility?

I was told many years or so ago by a senior manager at Dell that his company at the time used a methodology much like this. Not sure if that is still the case at Dell. I would be very curious to know if any of our readers out there and/or their companies use anything similar in estimating potential ROI.

I am out of space. Hope I was clear, and more on this soon. To better understand this, it is quite helpful to view the graphic.

At least, I hope you will agree it is fundamentally a better approach, even if it may be too much for many companiesto take on (though I will repeat it is not as hard as you may think). But even then, just thinking this way can improve even traditional approaches to ROI calculation, in my opinion.

What is your reaction to the idea of probability-based ROI analysis? Like it or not? Like the idea, but it seems too hard? Has your company ever used anything similar? Are there other approaches? Let us know your thoughts at the Feedback button below.