Market timing. Those two words used to make most professional investors scoff. Now, in the wake of this fall’s mutual-fund trading scandal, they make nearly every investor shudder.

But that’s because today’s headlines have changed the meaning of the term. In its new usage, market timing describes how speculators rapidly trade mutual funds to cash in on small pricing gaps between markets. In the original definition, market timing means shifting your money into cash or bonds when you think stocks are overpriced and moving back into stocks when you think they have gotten cheap.

Most pundits insist that traditional market timing is the investing equivalent of chocolate-covered arsenic — seductive but lethal. Conventional wisdom is so negative that whenever you hear “market timing,” the next two words are almost always “doesn’t work.”

But if the whole point of investing is to buy low and sell high, what’s so bad about refusing to buy stocks when they are overvalued? Isn’t that exactly what Warren Buffett did in 1969 when he liquidated his Buffett Partnership Ltd. — and similar to what he did in the late 1990s when he went into a virtual moratorium on buying stocks? Isn’t it what the great Benjamin Graham advocated in his book The Intelligent Investor when he wrote that “sound procedure would call for reducing the common-stock component below 50% when in the judgment of the investor the market level has become dangerously high”? In short, as is so often the case with conventional wisdom on investing, the truth about market timing is more complex than either its proponents or its opponents care to admit.

What Does “Work” Mean?

Let’s start by asking whether market timing really works. To paraphrase Bill Clinton, the answer depends on what the meaning of the word work is. Professional market timers say they seek to capture most of the upside of stocks while avoiding most of the downside. They are trying to smooth the ride by reducing the volatility of daily returns. In that narrow sense, there’s no doubt that market timing works. Taking your money out of stocks some of the time is virtually guaranteed to make your return less volatile.

Let’s say you time the market by flipping coins: On heads, you sell your stocks and keep the proceeds in cash; on tails, you get back into stocks. Since stocks often produce short-term losses, while cash never does, your market timing by coin flips will ensure that you make (or at least preserve) money during at least some of the stock market’s biggest downswings. So your return will almost certainly be smoother — but it’s unlikely to be higher. After all, your coin flips will lead you to miss many of the market’s biggest upswings as well.

So the intelligent investor should concede that market timing “works” only if it can lower your risk and raise your returns. And in that sense, the question of whether market timing works is much less clear-cut.  A market timer who boasts of having reduced risk with “skill” needs to provide extraordinary amounts of proof for that contention, since you could have done the same thing yourself merely by flipping a coin. 

History in Foresight

The key to market timing is finding a valuation tool that can reliably tell you, not in hindsight but in real time, when the stock market is overvalued. And many of the world’s best investing minds — including Nobel Prize-winning economist William Sharpe, plus finance professors Robert Shiller of Yale, John Campbell of Harvard, Jeremy Siegel of Wharton and Elroy Dimson of the London Business School — have put market history under a microscope, looking for just such a measure.

I spent days plowing through research papers and books bristling with polynomial equations, and by the end I had two results: a bad headache and a clear consensus. Price/earnings ratios, price-to-book value, dividend yields and even the ratio of stock prices to the replacement value of corporate assets (called the Q ratio) can all be used to show, with perfect reliability, when the stock market was overvalued. Unfortunately, it does not appear that any of them can be used to indicate, with reasonable reliability, when the stock market is or will be overvalued.

Price-to-book value, for instance, turns out to have been a great way to forecast what the market was going to do up until about 1960; then it stopped working. For years, dividend yields predicted future returns quite well; then along came the 1990s, when nobody gave a hang about dividends. These valuation ratios never send investors a singing telegram announcing, “I’m not gonna work anymore for you! It’s time to time the market with something new!” Instead, the power of these measures to predict the market’s next turn just withers away.

What about price/earnings ratios? They do have some predictive power. Finance professors Jack Wilson and Charles Jones of North Carolina State University have created a definitive record of P/E ratios on Standard & Poor’s 500-stock index (and its predecessors) all the way back to 1871. Their results enable us, for the first time, to look back at a continuous series of market valuations. When the market’s P/E ratio is below 10, Wilson and Jones’ numbers show, you should definitely buy: From such levels, stocks produce extraordinarily high future returns. But the opposite is not true: According to Wilson and Jones’ data, when the market’s P/E is at 20 or above — up in nosebleed territory, well beyond its long-term average of 16 — stocks may still generate fat gains over the years to come.

So history shows something profoundly strange: Investing in stocks can pay off not just when the market appears to be undervalued but also when it seems [moderately] overvalued.

Because, historically, the U.S. stock market has gone up much more often than it has gone down — and because when it goes up it goes up much more than it goes down — the opportunity cost of getting out of stocks while they are still going up is likely, on average, to exceed the losses from staying in them when they go down. And because markets do go up more often than they go down, most of the “signals” you get of a coming bear market are likely to be noise. At points of extreme overvaluation, it makes sense to become more conservative — but such extremes are rarer than you might think.

If you are near or in retirement, however, you may well want to scale back your exposure to stocks.

History in Hindsight

Yet the temptation to time remains powerful. You’re probably still kicking yourself for not selling your stocks at the end of 1999, when you knew the market was overpriced.

The truth is, if you’re kicking yourself, you’re kidding yourself. You’ve fallen into one of the worst pitfalls for an investor. It’s called hindsight bias. In a series of brilliant experiments, Carnegie Mellon psychologist Baruch Fischhoff has shown that once we discover how things turned out, we not only feel that no other outcome was possible, but that we knew precisely what would happen all along.

In 1972, Fischhoff asked several dozen people to forecast the outcomes of President Richard Nixon’s upcoming trips to China and Russia. Few of them expected the country’s relationship with these two historical enemies to improve. Nixon’s trips, however, went very well. Then Fischhoff went back to the people that he had surveyed and asked them to repeat their earlier predictions. More than three-quarters of them “remembered” having been more optimistic than their original forecasts showed they had actually been.

It’s obvious in late 2003 that in late 1999 the market was about to crash, but it was far from obvious at the time. Back then, most people — probably including you, no matter how you remember it now — were bullish.

How could it be otherwise? After all, if everybody and his brother had known that a crash was coming, then the market would never have been at those record levels in the first place.

Timing by Autopilot

The ideal solution to timing the markets already exists — but so far mainly in theory. Someday all mutual fund companies will offer an automatic rebalancing program that will systematically sell whatever has gone up the most and buy whatever has gone down the most. In a retirement account, auto-rebalancing will have no tax consequences and will keep you on target to hit your goals.

Let’s say you’ve chosen to keep 70% of your money in stocks and 30% in bonds. Now assume that stocks lose a fifth of their value while bonds stand still, leaving you 65% in stocks and 35% in bonds. Auto-rebalancing would sell your bonds down to 30% and put the proceeds in stocks, getting you back to your 70-30 target. You wouldn’t have to touch the money or make a decision.

Graham wrote that because human nature inevitably makes investors hanker to buy in bull markets and sell in bear markets, “we favor some kind of mechanical method for varying the proportion of bonds to stocks in the investor’s portfolio. The chief advantage, perhaps, is that such a formula will give him something to do.”

T. Rowe Price recently introduced quarterly auto-rebalancing for its 401(k) investors. Let’s see if other fund companies have the gumption to implement one of Graham’s most underappreciated brainstorms.

Until auto-rebalancing becomes widely available, you have to rebalance by hand. Every six months, on dates you can easily remember, smooth out your retirement accounts — sell what’s up and buy what’s down until your asset allocation is back to your target level. Do it twice a year, every year; it’s market timing that works, in every good sense of the word.