Psychohistory and finance

In Isaac Asimov’s Foundation books, psychohistory is the scientific study of human behaviour. Psychohistorians are able to predict what large numbers of people will do. However, the theory only works when the people are unaware of the predictions.

In the original Foundation trilogy, the goal of the psychohistorians (the Second Foundation) was to create a society in which they used psychohistory to guide the mass of humanity–essentially as philosopher kings. In Donald Kingsbury’s interesting update of the original trilogy, Psychohistorical Crisis, he observes that this is unstable. People will inevitably become aware of the psychohistorians, and once they are aware of them, their actions become unpredictable. Kingsbury’s solution was for his characters to develop a form of psychohistory which assumed that psychohistory, rather than being secret, was known to everybody. This second-order psychohistory was said to be stable, as everybody predicted everybody predicting everybody.

Kingsbury has to do some handwaving to make this work, and while I enjoyed his novel I think Asimov had it right: predicting human behaviour only works if the subjects are unaware of the prediction. Otherwise it is natural for people to say “they think I will do X, and therefore they will do Y, so it makes sense for me to do Z instead.” You pretty quickly get into a game of “I expect that you expect that she expects….” Not only are people in general fairly good at that sort of game, but actions can differ greatly depending on where they cut off the infinite regression about who expects what. Really clever people will cut it off randomly, by rolling dice, thus becoming deeply unpredictable.

Now let’s think about the stock market. To invest in a stock is to make a prediction that in the future somebody will want to buy that stock at a higher price. To follow an investment strategy is to become predictable in your predictions. If somebody else can predict how you will behave, they can take advantage of that knowledge in their own investments. Of course the amount of money matters; it is only useful if you can predict the behaviour of a significant portion of the investors in some stock.

I think this leads to two interesting conclusions. The first is that it is unwise to follow any popular investing strategy. A popular investing strategy is inherently predictable. If the strategy is popular, then it becomes worth taking advantage of the predictions. The result can be what is known as a contrarian strategy–although of course a popular contrarian strategy can itself become predictable. If you want to be a better than average investor, you should not follow a popular strategy. (Note the key words “better than average;” you can normally get average returns by doing what everybody else is doing–e.g., invest in an index fund.)

The second interesting conclusion is that investment strategies are inherently unstable. Despite Kingsbury’s hand-waving, I don’t believe it is possible to construct an investment strategy which can take second guessing into account. The effect is that any successful investment strategy will turn into an unsuccessful one–that is, if the strategy is predictable.

There are two natural responses. The first is to keep your investment strategy secret so that others can not predict it. This is what most hedge funds do. The second is completely avoid patterns and thus become unpredictable, which is difficult, but may well be what many successful stock traders actually do.

I should observe that in this note I am mostly discussing short-term investors. Long-term investors can rely more on the theoretical underpinnings of the stock market (which as I have argued elsewhere is mainly a hallucination, but at least it’s widely held).