What's the best tool to measure your website?

In a follow up to last week’s look at why Google Analytics always belongs within your measurement consideration set, a reader sent me the following question:

“I’ve messed around with a bunch of the free analytics tools on the web and have found them to be mostly unreliable. Just a few weeks ago, I had a discrepancy of hundreds of thousands of unique visits between compete.com and Google Analytics stats. This was pretty eye-opening and concerning for me.

I wonder if you have any tips on the most reliable tools to use and stats to trust?“

It’s a great question but one that, unfortunately, needs a long, hedged answer.

The simple answer is to check out my 7 keys to successful web metrics.

The long answer, is, well, long…

It’s never a good idea to recommend any one tool without knowing more about what you want to know about your audience. Sites like Compete are great for competitive information, albeit with some caveats, while GA, Omniture and WebTrends are much better to track the content and commerce performance of your site relative to its past performance. Neither, unfortunately, is great at both. The key is using the right tool to answer the right question.

The challenge is that all analytics programs have some set of accuracy issues and, due to different methodologies in use, it’s rare that you’ll get two different analytics tools to line up. Avinash Kaushik has looked at the difference between accuracy and precision in measurement before and as some brilliant commenter noted, “Apples are apples. It doesn’t matter if your apples are rotten as long as you’re comparing ’em to other rotten apples.”

As you evaluate tools, it’s a good idea to understand where its particular strengths and weaknesses are so that you’re choosing the right tool to answer the right question. And, that’s one of the core reasons why I believe you’re better off saving money on the tool and spending your budget on a capable analyst.

The differences in these methodologies will be particularly obvious when comparing something like Compete, which uses a sampled panel of traffic, to GA, which relies upon tags placed on the site and cookies placed on a user’s computer. Neither is “wrong” insofar as their methodology goes. It’s just that they’re designed to track different things. It’s really more a question of whether what they track is what you want to know.

Broadly speaking, tools like Compete are more useful if you’ve got a larger site that appeals to a broad audience in the US. It samples a panel of users, currently about 2,000,000 individuals (or roughly 1% of US based web users) and determines how different sites perform relative to one another based on traffic patterns it sees among its panel of users.

Of course, if your site gets more traffic from outside the US, or appeals to a particular market segment not represented in Compete’s panel (or overrepresented, which is just as bad), it won’t give you a clear picture of how you rank relative to your competitors. The flip side is, any issue in how Compete ranks you may apply equally to your competitors, so it may give you useful competitive intelligence all the same. Check out Douglas Hubbard’s two principles of measurement in my review of his book “How To Measure Anything” for more information on why imperfect data is often better than no data at all. And here’s a link to Compete’s methodology, if you’re a glutton for punishment… um… curious (same thing, really).

Now, GA, and its brethren, on the other hand, track individual web browser sessions (not people), based on cookies placed in a web browser’s cache during a visit. Two visitors using the same computer (say, a married couple, or multiple users of a library computer, for instance), may register as the same visitor in your tracking. By contrast, an individual who visits your site from a work computer and a different home computer would show up as two distinct visitors (that’s true for all tag/cookie based tracking tools). Whether that’s a problem for you or not depends on what you’re trying to measure. For most businesses, it’s not a problem. What matters more is in which direction the data moves. Is bounce rate declining? Are visits and visitors increasing? Are sales? Is conversion rate? GA (and other tag-based tools) do a good job of answering these questions as long as you’re always comparing the data to itself. Comparing GA to Omniture, on the other hand, could confuse the matter due to their differences in the way they count traffic.

I realize this is a longer post than usual – and more of a weighty read. (I also just learned why Avinash’s posts are so long – it’s hard to explain this stuff in a hundred words of less). But, I hope it helps all of you as you evaluate the right tool to measure your marketing. And keep those questions coming.

Did I miss something obvious? Do you have a question? Let me know in the comments.

Are you getting enough value out of your small business website? Want to make sure your business makes the most of the local, mobile, social web? thinks helps you understand how to grow your business via the web, every day. Get more than just news. Get understanding. Add thinks to your feed reader today.

Or subscribe via email.

And while you’re at it, don’t forget to follow Tim on Twitter.

Technorati Tags:
analytics, Avinash Kaushik, douglas hubbard, douglas hubbard review, E-commerce, Google Analytics, how to measure anything, how to measure anything review, measurement, metrics