How to Pick the Best Index Fund

A three-step guide for index fund investors.

By + More

At first glance, it seems easy to dismiss the differences between the Vanguard Small-Cap Index and the Nationwide Small Cap Index as purely academic. After all, both are passively managed index funds that promise to track the performance of a group of small-cap stocks. Picking an actively managed fund, you might reason, is one thing—that requires finding the right fund. But when it comes to index funds, there's always a temptation to pick a broad area of the market and go with the first fund you find that tracks it. Bonus points, of course, if it's cheap.

[See Study Shows Increase in Mutual Fund Fees.]

Not so fast. So far this year, Vanguard's fund is up 38.1 percent, compared with Nationwide's 27.6 percent return. Both are index funds. Both track small-capital stocks. And therein lies the problem.

In this case, there is an easy way to explain the performance gap: Vanguard tracks the MSCI U.S. Small Cap 1750 Index, while Nationwide aims to mirror the Russell 2000. But it's not always that simple. Even funds tracking the same index can have disparate results, although the difference shouldn't be that large. In other words, on top of knowing how to pick the right underlying index, investors also need to know how to pick the fund that best replicates the index. Here are some tips for finding the right index fund:

Decide what you want to track. Obviously, the first decision is what area of the market you want to focus on. For instance, do you want to track the performance of small caps? Large caps? The S&P 500? If expenses are your main concern, the general rule is: The bigger the market capitalization of the stocks in the index, the smaller the fees (the same holds true with actively managed funds). Another argument in favor of tracking stocks at the higher end of the market capitalization spectrum is that those areas of the market are more efficient. Practically speaking, that means there is comparatively little that managers can do to beat the market, so investors might as well buy index funds, since they're cheaper than their actively managed peers. On the other hand, when it comes to small caps or micro caps, active management can be particularly important because good managers can take advantage of inefficiencies in ways that indexes can't.

Meanwhile, funds can also track large-cap stocks with more accuracy than they can with small caps. The S&P 500, for example, is a fairly easy index for funds to replicate because they can readily own all 500 stocks. "The S&P 500 [funds] should be pretty much spot-on," says Morningstar analyst Greg Brown. On the other end of the spectrum, bond indexes can be particularly expansive and hard to replicate. To a lesser extent, the same is true with small-cap indexes. "As you move down the market cap ladder and when you move into fixed-income investments, it really does become more tricky," says Brown.

Pick an underlying index. When it comes to selecting indexes, there are two schools of thought. The first is numbers-driven. Advocates of this approach would recommend finding all the indexes that track the area of the market you're interested in and comparing their past performance. You would then go with the index that has posted the best returns. The second approach has a more holistic outlook, with its proponents arguing that solidly constructed indexes are better than high-flying ones. In other words, they point out that the purpose of index funds isn't to outsmart the market but instead to track it.

But what does a solidly constructed index look like? It can be hard for individual investors to sift through the intricacies that separate one index from the next. "These days, it's a little bit like looking at a menu in a foreign language," says Chris Tessin, a portfolio manager for Russell Investments, the company that created the Russell indexes. "I think the thing you want to look for is an index that is comprehensive, that truly covers the entire space," he says. If you're looking for broad large-cap exposure, for instance, the S&P 500 is a more effective index than a handpicked index consisting of only 200 of the 500 companies. "You want to look for an index that's unbiased, that does not just pick a subset from the stocks that you're trying to get exposure to," Tessin says.

Another question to ask is how the indexes are weighted. In stock indexes, the most common way is to give each stock a weighting based on its market capitalization—the bigger the stock's market cap, the more space it gets in the index. But other options, such as weighting each holding equally regardless of its size, also are popular. "One of the complaints of the market cap weighting is if a certain sector really heats up, like energy or industrial materials, investors are sort of buying into that at the peak," says Brown.

Find the right fund. Now it's time to find a fund that tracks the index you've selected. There are two considerations here: sampling methods and costs. In the indexing world, sampling is a necessary evil. On the one hand, it's important for the underlying indexes to be as complete as possible. But on the other hand, it's impractical for funds to own every piece of large indexes. Take the Russell 2000, for example. "The ideal would be obviously to have all 2000 stocks weighted perfectly and trading in unison with the index," says Tessin. "But the fact is there are costs associated with that: There are trading costs associated with it, there are management fees that investors are going to pay."

[Should You Deep-Six Your Mutual Fund?]

In other words, managers must engage in the very biases that the creators of indexes look to avoid. This is important to note because it's all too easy for investors to convince themselves that because index funds are "passively managed," it doesn't really matter who runs them. That's not the case. "They're passively managed in the sense that there's no active manager making market timing calls, but there's definitely some quantitative expertise going in and certainly some statistical methodologies on how they're doing the sampling," says Brown.

With that in mind, it's important to look into each fund's sampling methods and to see how reliably they are replicating their underlying index. Still, that's easier said than done. "Individual investors are going to be challenged because they don't have intimate insight into the trading methodology and into the day-to-day management," says Tessin. Broadly speaking, though, large fund providers can often better mirror indexes because they can own broader chunks of them.

Finally, costs are always important. As a rule, index funds should be cheap. Actively managed funds can charge more because they employ a manager to beat the market. Index funds, on the other hand, merely look to track the market, and if they do their job perfectly, investors should get market performance before expenses. Hefty fees, then, can leave index fund investors faring noticeably worse than the market after all is said and done.