Superforecasting? It’s all about process!

As readers of 2023’s grey swans will know, we believe that time preparing for tail-risk events – however unlikely they may appear – is time well spent. If one only considers traditional research, it’s difficult to stand out from the crowd.

In this vein, we have long been fans of the Good Judgment Project (GJP) and their 10 commandments of superforecasting. These have proven to be a key input into our ‘prepare, don’t predict’ decision-making process ahead of key events such as the US elections and the aftermath of the COVID-19 pandemic. Last year we enhanced our resources further and subscribed to GJP’s Future First service, giving us direct access to their ‘Superforecaster’ predictions.

So what are the key lessons?

Asking the right question right

Superforecaster rule two is “to break seemingly intractable problems into tractable sub-problems”. This means deconstructing a question and rebuilding it in a way that helps paint a better picture of overall risk. It helps to ask your questions in the right way, just like SMART objectives help individuals perform better in their roles.

To use a specific example, asking ‘will there be stagflation?’ isn’t easily defined, but asking if both the US consumer price index (CPI) of inflation and US unemployment will rise above 5% in at least one quarter in 2023 is specific, measurable and time-bound, and therefore far more helpful in understanding potential risks.

It’s also key that questions are both differentiated and monetisable: asking about expected central bank rates, for instance, won’t provide an answer differentiated from what’s already available from financial market data. Instead, we could ask: is the expected path of rates actually masking the true skew of potential outcomes?

On the surface, markets are pricing in a few rate cuts by the end of 2023. But we believe this masks a large probability of very little movement in rates, merged with a small but nonetheless significant possibility of lots of rate cuts if we hit a deep recession.

So while it appears the most likely outcome is for one or two rate cuts, by asking a better question in the right way, we can ascertain that this specific scenario may not actually be particularly likely.

Use probabilities, be structured and learn from mistakes

In our daily economics and strategy calls, we have learnt to think about forecasting in a very specific way. This begins with using probabilities, not words, to think about the future; it’s far more useful to conclude that there’s a 75% chance of US recession than to say merely that it’s ‘likely’.

We then have a structured approach to monitoring forecasts. This means using morning meetings to absorb newsflow, updating probabilities accordingly and combining every trade with scoring, a write-up and clear signposts for what would merit later reviews. This ties in with rule five: “Each side should list, in advance, the signs that would nudge them toward the other.”

We have also learnt to attribute significant value to the question “How often do things of this sort happen in situations of this sort?” (known in rule three as the ‘outside’ view). Nothing is entirely unique and our researchers will look for a reasonable base rate even when judging seemingly unique events.

Lastly, we dedicate time to carry out unflinching ‘post-mortems’ on our trades with both winners and losers, as summarised in rule eight: “Don’t try to justify or excuse your failures. Own them!” To learn from our mistakes, we aim to promote best-in-class process and understand whether luck has played a role in our winners as well.

If you’d like to learn more about superforecasting, the BBC has done a nice primer video on the subject.