How to measure ETF and mutual-fund risk

“Risk has been a hot topic for a long time now in the fund industry,” said Brian Bridger, senior vice-president, analytics and data, with Toronto-based Fundata. “Each of these metrics has their pros and cons.” The more types of metrics that the firm can provide, he added, the more complete the picture of a fund’s risk.

To illustrate how ETFs have fared during the recent tumultuous years, Fundata provided risk-measurement data to Investment Executive for the three years ended July 31. (Risk data can also be calculated for other time periods, Bridger noted.)

Standard deviation — which measures how much a fund or ETF’s returns deviate from its mean returns — has received the blessing of securities regulators as the core methodology for calculating risk ratings. The ratings are based on the most recent 10 years of performance history. Newer funds use a combined history that also includes a proxy such as a market benchmark or an older series of the same or similar fund.

The five-tier risk ratings range from low to high. They’re required to be published in the ETF Facts and Fund Facts disclosure documents. A broadly diversified Canadian equity ETF, for example, can be classified as “medium” risk.

Standard deviation is useful for comparing across asset categories, Bridger said. For example, “you can compare the volatility of an equity fund against a bond fund, and get a sense of the risk of each one.”

As Fundata’s calculations show, investing in a narrow market segment — and leveraging on top of that — is a recipe for off-the-charts volatility. So it is for the BetaPro Marijuana Companies 2x Daily Bull ETF.

The ETF’s three-year standard deviation of returns was a whopping 144%. By comparison, the standard deviation of a broad-market Canadian equity ETF is more than eight times lower.

The BetaPro products sponsored by Toronto-based Horizons ETFs Management (Canada) Inc. employ leverage and inverse leverage, and dominate the lists of the most volatile ETFs. As the Horizons prospectus warns, they’re speculative and may be suitable only for active traders able to assume the risk of losing their entire investment.

Excluding BetaPro, the highest three-year standard deviations were 56% for the Blockchain Technologies ETF, sponsored by Oakville, Ont.-based Harvest Portfolios Group Inc., and 52% for the Horizons Marijuana Life Sciences Index ETF.

One limitation of standard deviation is that it can’t accurately project potential future losses, and may in fact underestimate risk. “The tail risks of funds and investments are usually a lot bigger than what the standard deviation would indicate,” Bridger said, referring to rare but extreme events such as market crashes or natural disasters.

Another drawback of standard deviation is that it’s not expressed in dollar terms.
The average investor, Bridger said, might not fully understand how standard deviation relates to actual dollar loss.

Maximum drawdown is a risk metric that, while expressed in percentage terms, can more readily be translated into dollar amounts. Over a given time frame, it measures the percentage loss from the peak of a fund’s value to its low point.

“An investor can see what has happened in the past and what potentially could happen going forward,” Bridger said. But this measure will be misleading, he added, if the time frame being measured does not include a market correction or a bear market.

Over the past three years, excluding leveraged funds and those specializing in marijuana companies, the largest drawdowns were those of several emerging technology ETFs sponsored by Toronto-based Emerge Canada Inc. The Emerge ARK Fintech Innovation ETF, for example, suffered a 71% peak-to-trough downturn.

A risk measure based on relative performance versus a market benchmark is up-down capture. A down capture greater than 1 means the fund underperformed in a falling market. “If a fund has a very low down capture, then that’s good,” Bridger said. “That means when the market’s going down, they’re not losing as much as the market. They’re mitigating that risk.”

The use of this measure is appropriate when comparing funds in categories like Canadian equity where the constituent funds have broadly similar holdings. But in a diverse category like sector equity, up-down capture comparisons won’t be as meaningful.

Among fixed income ETFs, the BMO Long-Term US Treasury Bond Index ETF had the highest value, 1.7, for downside capture. Like other long-term bond funds, the BMO ETF has sustained steep capital losses because of rising interest rates, which depress bond prices.

Batting average for funds is a measure of month-to-month consistency. A one-month return is considered an “at-bat.” A batting average of more than .500 means the fund has produced a return greater than zero in more than half the monthly performance periods.

Least risky are cash substitutes such as the iShares Premium Money Market ETF and the CI High Interest Savings ETF, with perfect batting averages of 1.000.

At the other extreme were various Horizons BetaPro offerings. The worst was its two-times leveraged marijuana ETF. With a batting average of only .222, it essentially struck out nearly four months out of every five.

Bridger said the batting-average metric can be useful for evaluating alternative strategies funds with a market-neutral mandate. Ideally, “they don’t have big monthly returns but they’re very steady and they don’t incur many down months, so they show a very high batting average.”

During the three-year period, batting averages for these types of ETFs ranged from .472 for the Purpose Multi-Strategy Market Neutral Fund to .639 for the Picton Mahoney Fortified Market Neutral Alternative Fund. As with any risk metric, results will vary depending on the length and dates of the measurement period.