Shawn Brayman never felt entirely comfortable with the application of Monte Carlo simulations in financial planning.

The president of Lindsay, Ont.-based PlanPlus Inc. could appreciate the appeal of essentially pushing a button to determine the percentage likelihood of achieving a client’s objectives. But his background in mathematics left him wondering why Monte Carlo’s randomized statistical method was expected to come up with a fundamentally different set of results than a straight algorithmic calculation using statistics.

That initial question led to in-depth analysis, a detailed paper introducing a potential algorithmic replacement for Monte Carlo, called the reliability forecast, and an international Silver Level Financial Frontiers Award, sponsored by the Denver-based Journal of Financial Planning. It’s been a whirlwind journey that has made Brayman a hero to financial planners who question the value of Monte Carlo, and a villain to advocates of a simulation method that has been described as the gold standard.

“It’s not that Monte Carlo is wrong, because Monte Carlo is just math,” says Brayman, who was attending the International Financial Planning Advisors conference in Kuala Lumpur, Malaysia, at which he recently delivered a presentation based on his paper, Beyond Monte Carlo Analysis: An algorithmic replacement for a misunderstood practice. “The difference is the reliability forecast just does it a thousand times faster and, therefore, allows you to slice and dice it a lot more easily.”

The reliability forecast can deliver the same results in 15 to 20 calculations, compared with 10,000 or 100,000 with Monte Carlo, Brayman says. That greater efficiency makes it much easier to analyze individual subgroups of results. And that, in turn, allows advisors to probe issues such as the impact of higher or lower return assumptions (rather than the “mean return”) and sequence of returns on the success of an investment plan — two issues that Brayman doesn’t believe Monte Carlo addresses successfully.

He offers a simple example to explain the consequences of losing those data subsets. “If you gave a math test to a classroom full of students and you find out that the average in the class is 75%, yes, you could assume that there are some people in the class who maybe were gifted and there were some that were slow learners,” he points out. “But you can’t tell me anything about it because the only thing you’ve got is the class average.”

The same, he insists, is true of Monte Carlo when it comes to rates of return and sequence of returns. Averages lose a lot of the information that many advisors assume Monte Carlo simulations are testing for and you end up back where you started — with a straight algorithmic calculation.

Brayman’s reliability forecast uses the same capital market assumptions — the expected return and standard deviation — that Monte Carlo does to drive probability distributions, or the likelihood that a specific outcome will occur. But it doesn’t sink under the weight of randomized data that requires thousands of calculations to avoid introducing errors into the final results.

Instead, his approach creates a probability matrix that reflects the likelihood of reaching a certain age and achieving a certain rate of return. A histogram displays a success/failure boundary line based on when the client’s funds are depleted; a portfolio outlasting a client is defined as a success and a client outliving his or her money is defined as a failure.

“It’s not that the reliability forecast gives a different answer. It should give the same answer as far as the impact of the statistical assumptions you’ve made,” Brayman says. “The difference is, because of the way it works and how fast it goes and the way the information is constructed, it’s a very simple process to, in effect, add up that area of the chart where you have success and failure. It’s a very simple calculation where you can slice and dice the results for the client in a much more meaningful way. You can ask, ‘What is the likelihood of, say, getting a lower return? And if you get the lower return, how will that impact your plan?’”

Brayman has been convincing even die-hard fans of Monte Carlo that there may be ways to improve the way its results are presented to advisors and their clients. He says that one impact his paper may have in the long run is to motivate Monte Carlo advocates to work out ways to isolate subgroups. And he acknowledges that the increasing speed of computers means that they may eventually reach the same end state as the reliability forecast even if it’s a less efficient process.

@page_break@“The advisor is ultimately responsible for the plan and the analysis,” Brayman says. “Monte Carlo is like a big black box and it’s challenging for an advisor. You can’t do all the calculations with a calculator. You have to rely on computers. But you have to have some understanding of what it is you’re using the black box for and be cautious of things that, on the face of it, might appear to make our jobs a lot easier.” IE