Appearances can be deceiving

Investors should be aware that correlation is not the same thing as causality.

It doesn’t matter how much experience you have as an investor, you should heed the following golden rule of statistics: correlation is not the same thing as causality. People do tend to mix up correlation and causality, not just when investing, but in other situations as well. Just because two lines on a financial chart follow a similar path, it certainly doesn’t mean that there must be a direct connection between them. This is why certain correlations are referred to as spurious correlations – they suggest a relationship between two variables, when in reality there is none, or the correlation arises as a result of a third variable, an additional element that creates a connection between the first two variables that is not obvious from the chart.

Misleading connections

The work of Tyler Vigen is interesting in this context, as it helps explain the potential consequences of such correlations. The Harvard student became a celebrity after writing a computer programme that has already established over 30,000 ridiculous and spurious correlations.

For example, measured over a period of ten years, the programme has determined that the divorce rate in Maine correlates by over 99% with per capita margarine consumption in that US state. Should we therefore conclude that margarine damages the stability of a marriage?

Or another example: over a ten-year period, the number of films actor Nicolas Cage appeared in correlates by over 66% with the number of people who drowned by falling into their swimming pool. Does this mean that the actor’s films are dangerous?

Obviously false connections can be funny. Like the one about the appearance of storks and the birth rate. Statistically, more babies are actually born in regions with more storks. There is correlation, but not really any causality.

If two financial curves follow a similar path, that certainly doesn’t prove that there is a direct connection between them.

Critically assess the situation

We need to take a critical approach when considering correlation and ask whether the apparent connection is in reality caused by a different variable. Even in the financial world, the belief in certain correlations defies all reason and continues to provide the stuff of legends. One of the most famous examples of this is the hemline index. The theory referred to as the hemline index was first described in 1926 by George Taylor; it states that there is a correlation between the average length of skirts worn by women and economic performance.

This theory suggests that as short skirts become more popular, rising markets will follow soon. And on the flip side, the trend towards longer skirts points to a bearish market trend. The basic idea behind the theory is that if sentiment is positive, consumer confidence rises, as does the courage needed to wear short skirts. Some economists are still trying to test the validity of the hemline index theory today. However, the average investor is better advised to focus on factors other than such pseudo-indices when choosing their investment strategy.

Computers are able to carry out a huge number of calculations in mere seconds.

The age of big data

The issue of spurious correlations really isn’t new. Yet as big data, artificial intelligence, machine learning and other similar technologies are gaining ground and opening up whole new areas of opportunity to the financial world, this issue is becoming increasingly topical and complex.

The strength of these technologies lies in the ability of computers to carry out a huge number of tests and calculations in mere seconds. But the more we look for connections, the more of them we find. So a supercomputer is much more likely than a human to establish seemingly meaningful links between variables, which are, in reality, spurious correlations.

Although new technologies offer high levels of innovation and sophistication, we should always ensure that results have been established on a strictly scientific basis and analysed by experts.

Where possible, investors should always take advice from an expert, in order to avoid inadvertently comparing apples and pears or – to use a previous example – establishing a connection between the number of pool drownings and the number of Nicolas Cage films.

Our brains like simple explanations and enjoys making easy connections where none may exist. As the champions of quick conclusions, we may look at a chart and easily convince ourselves that changes in variable A depend on changes in variable B, or vice versa. Yet closer statistical examination may show that the similar developments illustrated in the chart are down to pure coincidence and not to causality at all.

The issue of spurious correlations isn’t new.

We need to be careful, as strong correlation may indicate causality, but there could be other equally good explanations: it may be pure chance that there appears to be a connection between the variables, and that there is no real underlying relationship. There may be a third, hidden variable that makes the relationship appear stronger (or weaker) than it really is.