Why Backtesting Always Lies
There is no investor in the world that will not ask you for your past performance results. The human need for assurance is greater than the need for fame of the most insecure Hollywood actor. Yet objectively speaking, there is perhaps no worse indicator of future performance than past results.
All past performance tells you is that you would have made money in market environment of yesterday but it will tell you nothing about how you will perform tomorrow. However, for an investor putting in fresh funds tomorrow is the only thing that matters.
I am sure stock market investors were thrilled to give Bill Miller money. He had, after all, beaten the S&P 15 years in a row! I am also sure that they were not so thrilled when this idiotic line of thinking cost them to lose 58% of their money in just one year as Mr. Miller’s buy-the-dip philosophy suddenly stopped working.
To evaluate any investment idea (options excluded) you need to appreciate that there are only 2 states of price -- momentum and mean reversion. In plain English that means prices will either trend -- moving in one direction for a sustained period of time -- or they will range -- moving around a fixed price level for a sustained period of time. Every single investment strategy, from the most complicated HFT algo to the simplest buy and hold method are simply expressions of these 2 key ideas.
Once you understand this fact you pay much less attention to the past results and give much more weight to trading tactics. The key to a good system is not how much money it makes in a favorable market environment but how little it losses when the state of price behavior changes. This is also the key reason why every single backtest ever devised lies miserably when traded live. And it is the reason why I never care about back test results when I evaluate a system.
As the great Yogi Berra used to say,”In theory there is no difference between theory and practice. In practice there is.” That is because investing in NOT Newtonian physics ( Newton by the way was a horrid investor). The past is not prologue in investing. The future does not repeat itself exactly like the past. That’s why using the past to make decisions about the future is just about the worst way to analyze an investment idea.
Yet we are all subject to recency bias. We all want to be reassured that we are making a smart decision because it worked in the immediate past and that is why most of us are horrible at investing or at assessing investment ideas.