A blog series to contrast the key distinctions between trend following and countertrend strategies during building, testing and trading. In this post we examine the effects of data integrity and simulated trade sample size on backtested performance.
Price Data Integrity
One of the major obstacles for traders looking to research trend following models is data. Since trend following models look to “cut losses short and let winners run”, profitable trades can last for many months or even years. This inherent characteristic has two important implications. First, it results in much longer trade duration’s and consequently fewer simulated trades from a backtest. Second, due to the strong positive skew in trade returns, a small number of highly rewarding trades contribute to the majority of the overall return. These characteristics combined mean that trend following strategies are very sensitive to potential data biases – they cannot tolerate data that has not been fully and properly adjusted for corporate actions and survivorship bias. “Garbage in, garbage out” aptly describes the effect of poor quality data on the backtesting process with respect to trend following. And you’re out of luck if you think that you can simulate the effects of perceived data biases – the concentration of overall return, relatively low number of simulated trades and material impact of survivorship bias makes it near impossible to estimate the effects of known data shortcomings when employing poor quality data for trend following backtesting.
Unfortunately, few retail offerings provide the rigour needed to ensure properly adjusted price datasets. It’s however possible to acquire data that has been professionally prepared for commercial entities in the asset management space, but these are costly and generally out of reach to the private investor.
Successful countertrend strategies on the other-hand are more short-term in nature, with trades lasting days as opposed to months. The shorter holds result in much higher number of simulated trades from a backtest. Another important distinction is that countertrend strategies have relatively low risk/reward ratios but high win rates, so their performance is not dependent on a few highly rewarding outcomes, but rather many small gains. These attributes – large number of historical trades with short duration’s and low past trade return concentration – make countertrend strategies less sensitive to data integrity issues. One additional upside associated with the low trade return concentration (many trades contribute to the overall strategy return, as opposed to few trades as with trend following) is the ability to simulate some of the likely effects of the known data integrity issues on performance. For instance, we could remove the top 10% of most profitable trades from our simulated database to allow for survivorship bias and corporate actions and then rerun the test to determine the effect on overall performance. Essentially, we can emulate a test done on high quality data by massaging the performance numbers downward to allow for perceived data integrity issues.
Many retail offerings provide cheap end-of-day equity price data that are “good enough” to test countertrend strategies. For most retail traders, countertrend strategies are better suited to the data solutions currently available. If you do not have the budget nor understand the intricacies involved in testing long-term strategies, then short-term strategies, such as a countertrend approach, is likely a better place to start.
Simulated Sample Size
As discussed above, countertrend strategies generate a much larger number of simulated trades during a backtest relative to trend following strategies. This is one of the most desirable aspects of a short-term approach because sample size is the single most significant contributor to our confidence in estimating the future – the more simulated trades we have, the higher our confidence in future performance. Smaller samples are more susceptible to the effects of good or bad luck during a backtest, which can over or underestimate the underlying edge that a strategy exploits. Consequently, the expected performance in any given year for a trend following strategy is far less certain relative to a countertrend strategy – our confidence bands are set wider as a direct result of a smaller number of historical trades.
After data integrity, trade sample size from a backtest is the most effective metric to gauge the robustness of a strategy, and oddly enough the least spoken about in trading circles. Sample size is so powerful that it doesn’t matter whether or not we understand why a given strategy works – as the trade sample increases, the probability that the strategy works due to chance alone decreases, and ultimately approaches zero. This fact alone is reason enough for most private investors to abandon research on long-term approaches and instead focus on short-term approaches.
Countertrend strategies, or short-term strategies in general, are much more forgiving when it comes to price data integrity issues. Regardless of whether you have high quality data or not, countertrend strategies always provide for higher levels of confidence in future performance due to the greater number of simulated trades relative to their trend following counterparts. For these reasons, most private investors will be better served by focusing their energies on developing short-term trading strategies as opposed to long-term strategies.
In my next post I’ll explore and discuss the most appropriate markets for each approach. As always, I welcome your thoughts and suggestions.