The Seven Deadly Sins of Stress Testing
Imagining your portfolio in different conditions to today is useful. But it’s also dangerous. It’s important to know why.
Stress testing your portfolio is a good idea. If done well, it lets you peep at the future, helping you swerve problems down the road.
But if done badly, which it often is, it can give you a false sense of security; telling you to accelerate when you should have hit the brake.
Firstly, what are the benefits?
By stress testing I mean transporting your portfolio into conditions unlike today’s. You can do this by placing your current holdings into an earlier period when those conditions prevailed.
How do they respond? Does your portfolio carry on motoring? Or spin off the road?
The current inflation-driven growth-to-value rotation market provides a good example. Rising inflation had been AWOL for the preceding 10 years, but within that period there were a few brief episodes when investors fretted over inflation returning.
One of those was the second half of 2016. If you’d transported your portfolio back to then, you’d have had a decent idea of how it might respond if markets panicked about inflation. Which, in 2022, they did. (If you missed it; growth funds and bonds tanked, value funds outperformed).
This means you’d at least be emotionally prepared for such an event or, depending on your process, that you could adapt to account for it happening in the future. And what’s not to like about that?
But that’s the good part.
Stress tests are far from perfect; they’re more art than science. However, they’re increasingly being used for official, regulatory purposes, and official, regulatory people prefer science to art. This means a false sense of precision and data dependence is creeping in. Stress tests are now being treated as factual maps of the future, not loose, vague guides to potential risks (which is basically all they are).
Here are seven ways they commonly go wrong:
1: Cookie-cutter time periods
For a decent stress test, you need to precisely isolate a phase that contains the thing you’re worried about. Unlike the 2016 chart above, most events aren’t so kind as to fall into neat six-month periods. Mostly they’ll begin mid-month and continue for an inconveniently random amount of time.
This is a particular pain for amateur investors, who don’t have access to the same toys as professionals. Relying on publicly available figures, which are often provided only in cookie-cutter lengths, might reveal some hints but is usually so clumsy as to be of no use at all.
2: Snowflakes
If you’re trying to answer a big question like, ‘How will my portfolio perform in a bear market?’, you need to know that bear markets are like snowflakes (as in no two are the same, not that they’re overly sensitive to criticism).
In 2000 growth stocks were hammered while value stocks held up well, but in 2008 it was value that took the beating while growth got off lightly. Blithely testing your portfolio in the previous instance may tell you exactly the wrong thing about the next one.
Likewise, there are different types of inflation. Demand-pull, cost-push, stagflation and hyperinflation, to name a few, will have different impacts on different assets. The 2016 inflation stress test, for example, only covers a change of inflation direction. It doesn’t say what will happen if inflation sticks around for years — that may be different.
To be useful, then, you need to define more precisely than ‘how will my portfolio perform under high inflation?’ or ‘how will it fare in a sell-off?’
3: Tourists
If you’re stress-testing your holdings in funds, you should understand whether they stick to a strict style of investing or whether they shift over time (either deliberately or by drifting towards what’s winning).
If they do shift, then stress-testing won’t work, as you won’t know if, when markets change, they’ll be the same prospect today as they were in the period you’ve chosen for your stress test.
This is a particular problem in multi-asset sectors as many funds actively try to shift between styles, asset classes and regions (badly, in almost all cases), making it impossible to know what to expect from one market phase to the next.
4: Hickam’s dictum
The world of medicine gave us Hickam’s dictum:
“A man can have as many diseases as he damn well pleases.”
It’s a reminder to doctors that patients can have several diseases, not just one at a time. These will collectively present a baffling array of symptoms, which leads to misdiagnosis.
And so it is with markets: A sell-off can have as many causes as it pleases too.
2022, for example, started out as an inflation scare, but these soon morphed into fears over war with Russia. The two then intertwined, and continue to move markets today. As such, using 2022 as a stress test for “inflation” or for “war”, and not “inflation and war”, may throw out some confusing results.
5: What’s water?
As the old joke goes: two young fish pass an older fish swimming the other way. ‘Isn’t the water lovely today?’ calls out the old-timer. The young ones smile and nod, then turn to each other and whisper: ‘What’s water?’
In markets, some trends can persist for so long that we forget they’re trends and assume they’re just what always happens. In the noughties, some investors began to assume that commodity prices only go up and that growth investing was a busted flush, while over the last decade many came to assume tech share prices can only rise and growth always beats value.
If we assume something’s normal, we don’t think to stress test for it as it doesn’t occur to us that it could change. But, as commodity investors found out after 2010, and tech investors in 2022, anything can change, so it’s important to use your imagination or to study further back into history (and ideally both).
6: When the facts change…
Things change, especially in financial markets. Managers leave, mandates change, companies and funds grow and shrink. All of these things, and more besides, need to be accounted for in your stress test. Quite simply, is each holding the same prospect today as when your chosen test conditions occurred? Not to mention the fact that newer holdings in your stable won’t stretch back far enough to cover many instances of different conditions.
7: Black swans
Possibly the biggest problem is that stress tests rely on past data, but future events will likely yield different results from past ones.
Would a pandemic in 2023 spark the same market reaction we saw in 2020? Maybe, but probably not. The fact we’ve now experienced a pandemic changes how we’ll react to another one (Zoom quiz, anyone?).
And that’s for the ‘known-unknown’ category. The past will tell you nothing about black swan events, which are the ‘unknown unknowns’.
In August 2001, for example, no one was stress-testing how their portfolio would react should terrorists crash jets into the World Trade Centre. And even if by some (highly suspicious) miracle they were, there was no past data to use for such a test anyway.
7x1 = One Big Problem
These seven sins add up to one big problem with stress tests: They can give you more confidence about investing than you should have. This causes investors to make costly mistakes, like taking on too much risk or trying to time the market.
So yes, carry out stress tests. They’re a useful way of highlighting risks. But don’t stop there. Remember there are more live risks than there are realised ones from the past, and none will reveal themselves till it’s too late.
In other words: Proceed, but proceed with caution. It’s the best way to avoid stress.
This was adapted from my Citywire column, published in February 2023. Nothing in here should be treated as advice to buy or sell a specific security.
