By Jason Zweig | May 12, 2017 1:03 pm ET
Image credit: William Hogarth, “A Rake’s Progress: The Orgy” (1733), Sir John Soane’s Museum, London
My colleague James Mackintosh’s irresistibly titled column earlier this week, “An Algorithm, an ETF and an Academic Study Walk into a Bar,” highlighted a new study on the surprising fragility of findings that the market can be easily beaten with mechanical methods of stock picking.
The study, by financial economists Kewei Hou and Lu Zhang of Ohio State University and Chen Xue of the University of Cincinnati, sought to replicate earlier research documenting that various factors, or common patterns of risk and return, can predict future outperformance.
Many of these factors have become the basis for so-called smart-beta funds specializing in “value” (stocks trading at below-average multiples of earnings or assets), “low volatility” (stocks whose share prices fluctuate less than average), “momentum” (companies whose share prices have lately been rising fast) or “quality” (companies that are highly profitable relative to their level of assets).
The latest work is in the spirit of the replication movement that has been sweeping through the social sciences since the validity of many psychological studies was thrown into doubt a few years ago. (See here and here, for instance. Or listen to economist Russ Roberts interview statistician Andrew Gelman here – I promise, unlike most interviews between an economist and a statistician, this one is fascinating.)
As the Nobel laureate Daniel Kahneman has written, the common flaws of those psychological findings were that they were “based on small samples, that the effect sizes were perhaps implausibly large, and that no single study was conclusive on its own.”
The same problem persists in medicine, where researcher John Ioannidis’ paper “Why Most Published Research Findings Are False” has been viewed more than 2 million times but has barely stemmed the tide of studies on small groups of subjects with weak results.
The problem of “p-hacking,” or dredging through an ocean of data until you find a pattern you can present as statistically significant, is endemic to research in finance. You should approach all claims of market-beating patterns with extreme skepticism.
In January, Campbell Harvey, a finance professor at Duke University, gave the presidential address at the American Finance Association on that topic. He estimates that at least half of all “discoveries” in investment research (and, thus, the expectations of investors in funds based on them) are false. The video of Prof. Harvey’s talk is well worth watching, or you can read his text here.
As Prof. Harvey has shown, so much data is available in finance that someone searching for a pattern “predicting” outperformance is all but certain to find one — even by statistical fluke alone.
The incentives to find a pattern are huge, and the costs are minimal. Supply a smart young analyst with a decent computer and plenty of pizza, and in a matter of days or weeks you will have a set of data full of patterns – many of them probably spurious — that you can market to investors.
In the 1990s, the investing website the Motley Fool concocted a cockamamie mechanical stock-picking strategy it called The Foolish Four, which took the five stocks in the Dow Jones Industrial Average with the highest ratio of dividend yield to stock price, then eliminated the one with the single highest ratio and invested in the others. The website claimed that history showed investors could have “trashed the market averages” with this approach. Unfortunately, the approach stopped working almost immediately after its so-called discovery, and the Motley Fool has long since abandoned it.
Such weird statistical contortions sound absurd, but many of today’s smart-beta funds are almost as odd. You can buy ETFs that own:
Mind you, some of these might turn out to be great investment ideas. But the odds are, many of them will turn out to be nothing more than patterns that appear to have worked in the past that have no predictive power for the future.
Source: WSJ.com, MoneyBeat blog, http://on.wsj.com/2qdGi6E
Resources:
Mathematicians Against Fraudulent Financial and Investment Advice (MAFFIA)
Campbell Harvey’s presentation (video, text, and slides) on the scientific method in investment research
Darrell Huff, How to Lie with Statistics
Chapter Four, “Prediction,” in Your Money and Your Brain
Definitions of BACKTEST, DATA, MODEL, MUTUAL FUND, QUANT, and SMART BETA in The Devil’s Financial Dictionary
Chasing Hot Returns in ‘Smart-Beta’ Funds Can Be a Dumb Idea
The Difference between an Investment Firm and a Marketing Firm