Analyzing purchase intent on Web 2.0 and lying with data…

Last week, I noted Seth Godin’s post about fudging the numbers. Now Steve Rubel gives a textbook example of how this works in the real world. While I understand Steve’s point – that Web 2.0 sites aren’t designed to drive ad revenues – I have two huge issues with his methodology:

1. As Steve notes, Microsoft adCenter’s Online Commercial Intention tool is, at present, demonstration technology. In fact, I couldn’t repeatedly reproduce Steve’s numbers. While the results were usually close to Steve’s findings, I got different results running the same query more than once. Apparently, the tool must base its answers on live data (kind of cool, actually), but, I wouldn’t put too much emphasis on its data at this point. Microsoft itself states, “some websites that have commercial intent might produce a score of less than 0.5. If you feel that this is the case with your website, we encourage you to contact us to help improve our scoring mechanism.” Not necessarily confidence inspiring. Much worse in the methodology, though, is…
2. The sites that Steve chose to demonstrate the issue. Comparing Twitter with Amazon? Seriously?!? Apples and oranges, I say. I ran some additional sites (commerce and non-commerce focused), using the same tool. Results below…

• Web 2.0 art trading site: Artflock.com – 62%
• Web 2.0 shopping search site: thefind.com – 47%
• Web 1.0 information site: CNN.com – 45%
• Web powerhouse: Google – 25%(!)

The key point is it doesn’t matter if it’s Web 1.0, Web 2.0, or what-have-you. What matters is how effectively you understand your users and meet their needs. That’s the way you monetize your site.