Ben Carlson at Ritholtz Asset Management reminds us that backtesting offers no shortcuts to investment nirvana. As he correctly points out, there are numerous shortcomings in the art/science of reconstructing the historical results of an investment strategy. But it’s also true that backtesting, if used wisely, can be a powerful tool for sensibly managing expectations with regards to return and risk. In fact, there’s really no reason not to run a backtest, even on strategies that appear to be free of mystery. The tough part is figuring out how, or if, to use the results. Fortunately, clear thinking and planning can boost the odds that backtesting will be a productive exercise.

Even under the best of circumstances, however, it’s essential to remember that backtests are but one tool for researching portfolio design. And like most tools in finance, this one has its share of traps. Meantime, some of the most valuable information in a backtest may counterintuitive because it’s bound up with negative selection. An intelligently designed backtest that shows a strategy to be a dog is usually far more persuasive than a backtest that offers a glowing review.

At its core, this is a tool for data exploration and studying how a given set of portfolio rules interact with securities. But the devil’s firmly in control of the details and you can’t spend too much time thinking about best practices in this corner of quantitative analysis. The first order of business, as Brian Peterson advises in his must-read “Developing & Backtesting Systematic Trading Strategies,” is laying out priorities before you write one line of code.

It is important to understand what you are trying to achieve before you set out to achieve it.

Without a set of clearly defined goals there is great potential to accept a strategy or backtest that is really incompatible with the (previously unstated) business objectives and constraints. Worse, it can lead to adjusting your goal to try to follow the backtest, which can culminate in all sorts of bad decision making, and also increases the probability of erroneously accepting an overfitted backtest.

The sad reality is that backtesting is abused, sometimes naively but intentionally as well. The worst offenders are the charlatans who use flimsy backtests to hawk questionable investment strategies and products.

The good news is that an well-constructed backtest is enormously valuable, if only because it forces you to analyze your assumptions by writing and review the rules via computer code. Nearly every coding project I’ve worked on delivered at least one “ah-ha!” moment of deeper clarity about the pros—and cons—of a strategy under scrutiny. In fact, every strategy deserves to be backtested so that the investor has a clearer understanding of the mechanics. Having coded countless strategies over the years (mostly in R), I’ve found the work to be profoundly enlightening, even when the results more or less fall in line with expectations.

There’s an assumption that only complicated, short-term trading strategies are fair game for backtesting. But relatively conservative portfolio strategies with medium- and long-term horizons can benefit from backtesting analysis by shedding light on the inner workings of interactions between, say, a dozen ETFs that are rebalanced periodically. You may or may not adjust the strategy after this analysis, but the process of quantifying assumptions and reviewing how the risk and return metrics stack up is almost always an edifying effort. The insight may be subtle—recognizing that a fund offers far less diversification benefits than assumed, for instance. Or maybe the results reaffirm your expectations. Another possibility: a backtest reveals a previously overlooked weakness in the strategy.

Backtesting as an educational tool, in other words, is arguably where the most value-added insight lies. By contrast, hitting a home run by uncovering a previously unknown investing system that generates high returns with low risk is the equivalent of winning the lottery—it’s possible, but unlikely.

The simple truth is that there’s no avoiding backtests. Even a buy-and-hold strategy is based on a backtest, albeit a simple one.

We shouldn’t whitewash the caveats in backtesting, but that’s no reason to assume the worst. The potential for abuse is vast, but so is the opportunity to inform and illuminate.