I recently read an interesting book based on the following ideas:
- “Incentives are the cornerstone of modern life”
Many websites use this fact of life by offering free products, free trials or free information.
- “The conventional wisdom is often wrong”
That is why we always test our new website ideas. You do test don’t you?
- “Dramatic effects often have distant, even subtle causes”
This is why we test small things like: the design of the call to action button, headlines, promotion code field, etc.
- ” ‘Experts’ from criminologists to real-estate agents – use their informational advantage to serve their own agenda”
This is why we don’t rely on web site experts to decide how to improve the website. Instead, we always test so we know what our customers want.
- “Knowing what to measure and how to measure it makes a complicated world much less so”
By now we all know that looking at website statistics doesn’t help us. Instead, we have to think and decide what to segment and where to drill down, resulting in actionable items that improve our website conversion rate.
Web analytics and economic analysis can be dangerous
The really difficult thing about web analytics is point no. 5. Most people know that this is a challenge but unfortunately they don’t know that the most dangerous problem is how to measure.
Measuring the wrong way is much too common.
Steve Levitt and Stephen Dubner fall into this same trap.
After analyzing what matters in parenting they find out that having been adopted matters. Studies show that a child’s IQ is much more influenced by the biological parents than the adoptive parents (page 171). Now if we left it at that we would think that the adopting parents don’t have much influence on the adoptive child.
website-testing-–-are-website-owners-the-only-ones-who-dont-want-to-test/Luckily, Levitt decided to dig deeper. Maybe because he didn’t like the results. This happens a lot in web analytics—we don’t like the results so we dig deeper until we find what is really going on. This is ok. However you should also do this when we like the results even though it serves our agenda (see point 4 above). You do want to find out the truth, don’t you?
Getting back to the adoptive baby, although the did poorly in school, another study showed that by the time they became adults they “…veered sharply from the destiny that IQ alone might have predicted. Compared to similar children who were not put up for adoption, the adoptees were fare more likely to attend college to have a well-paid job, and to wait until they were out of their teens before getting married. (page 176) ”
So the adoptive parents did matter after all. It is good that Levitt decide to dig deeper. Or maybe his decision of what to measure was wrong (see point 5 above). Maybe instead of measuring success in school he should have been measuring college attendance, jobs and marriage.
It seems like economists make the same mistakes we web analytics people do.
Read the book to get inspired about web analytics.