Get Data

Reading an article on marketing in Fast Company I'm reminded of the importance of the stats.

There are two arguments I'm aware of around stats. The first says: if you've got stats figure out what they mean and act on them.

The second argument says: stats are fine, but there are so many variables that your regular stats can't take into account, that they are worthless.

(And there's of course the third argument that says: I know the stats say that, but I want to do this anyway.

We'll ignore this last one as nothing more than bone-headed ignorance, in the knowledge that sooner or later this attitude will die along with it's proponents. Or be fired; whichever comes first.

I've heard this complaint often. The offender is usually described as a pre-Web manager who isn't interested in understanding the stats, but is still in charge.)

Personally, I accept argument two but reject the idea of throwing the baby out with the bathwater.

Big companies, like Amazon, have the time and money to run two parallel site which treat the same content in different ways. By serving them up at the same time you can clearly see which works better according to the test you set up. But small companies don't have the time to run these sort of experiments.

So what is a small company to do?

a) Find the time to run some controlled tests on your most important content.

b) Take section stats and site stats, or page stats and site stats and compare the exit rate, the bounce rate, the time on site or page views per visit for each. That will tell you at least how a page, or section is performing in comparison to the whole site. Not perfect, but helpful.

b) Look for trends over time. Again not perfect, but helpful.

c) Don't ignore the obvious.

There are times, even with the most simple stats package that something will jump right out at you. For example, if one section of your site is getting the majority of user attention, that's clearly the area where your developer time will be best spent. Or if something habitually gets no traffic, it's probably not worth the time you're sinking into it.

Which brings me back to the article in Fast Company, about aqwoman who uses, among other techniques, the Taguchi method.

But the point is simple. If you've got data, use it. If you haven't got data, get data.