In the supermarket the other day I was happy to see a sale for low fat cottage cheese in a 3 pack. There were many 3 packs available and I soon found out why.

The expiry date was fast approaching and my son nixed the purchase. Unfortunately there were no single low fat cottage cheese containers for sale–only these fast expiring 3 packs.

Apparently this 3 pack idea didn’t go over well in this store.

All I could think of was the VP Marketing person looking at his graphs which show that the 3 pack idea was increasing revenue and profits. What he forgot to do is segment his analysis. If he would have checked each store type he would have found that some segments didn’t buy the 3 pack.

Segment your analysis to optimize your campaigns

Segment of people who won’t buy the 3 pack:

• Smaller families that don’t need 3 cottage cheese containers
• Or a lower income group that only buys what they need for the near future
• Or people who are trying to gain weight and skip the low fat stuff

What’s good for brick and mortar stores is good for bit and pixel stores

Even when your campaigns are successful make sure your web analysis includes segmentation in order to find out where the campaign did not succeed. Segmenting ideas include:

• Time of day
• Day of week
• Location
• Language
• Products bought

This way you will optimize your campaign even more. Don’t make the same mistake as this supermarket VP. If he had been more proactive I wouldn’t have had to leave the store without my cottage cheese.

Photo credit: stu_spivack

Understanding web analytics and statistics is becoming more important for web marketers. People who read this blog or work with us know that we are data driven. Our decisions are made after testing and analyzing. And we are not the only ones.

The New York Times as well as Wired ran interesting stories on Statistics and Data.  Google figured prominently in both articles.

The world is changing from analog to digital. Data is going to become more important to businesses as time goes on. I agree with Arthur Benjamin that statistics should be a required subject.

Replacing Calculus Education with Statistics and Probability

Maybe this will help the next generation of website analysts to be more successful.

Dashboards are important because they:

• Provide warning signals
• Provide people with an important information overview

However, they can also be very dangerous. If your job requires you to interpret data and create actionable items you must have access to all the data. Don’t rely on someone to prepare a dashboard for you. Here’s why:

• You need to look for trends and patterns in the data. Finding things that should have happened but didn’t. If you rely on someone else’s interpretations - which is what a dashboard is - you will miss things.
• Your expertise will let you know where to find more data. Drilling down or accessing additional information is critical. The right level of detail cannot be decided in advance - which is what a dashboard does.
• You need to build your own mental model - not rely on others
• You need to understand how the data was collected
• You need to be able to dissect the original data as you learn more about the problem

A web analytics professional must convert data into comprehensive explanations in the same way that expert weather forecasters analyze their information

Talented web analytics professionals know that the data is only half the story. Intuition and experience are needed to interpret the data and dashboards can get in the way. To learn more, read Gary Klein’s book, The Power of Intuition. This blog post is based on his analysis of comparing expert vs. average weather forecasters.

Photo credit: Fabiano (LicoSp)

One of the most challenging parts of my job is to convince people of the importance of AB, multivariate, and usability testing on their website. Initially, most companies want to focus on getting traffic through SEO and Google Adwords.

For new sites this is understandable–you can’t do AB testing on sites with no traffic.

However, websites with traffic should start multivariate testing immediately. It doesn’t interfere with increasing traffic through SEO or PPC. And it can improve conversions significantly.

Dan Ariely wishes his nurses would have been open to testing procedures.

It is somewhat comforting to know that Internet marketing is not the only area where people are reluctant to test. At the end of Dan Ariely’s fascinating talk about cheating, he tries to convince everyone of the importance of testing by concluding, “Unless we test those intuitions we won’t make things better.”

I recently read an interesting book based on the following ideas:

1. “Incentives are the cornerstone of modern life”
   Many websites use this fact of life by offering free products, free trials or free information.
2. “The conventional wisdom is often wrong”
   That is why we always test our new website ideas. You do test don’t you?
3. “Dramatic effects often have distant, even subtle causes”
   This is why we test small things like: the design of the call to action button, headlines, promotion code field, etc.
4. ” ‘Experts’ from criminologists to real-estate agents - use their informational advantage to serve their own agenda”
   This is why we don’t rely on web site experts to decide how to improve the website. Instead, we always test so we know what our customers want.
5. “Knowing what to measure and how to measure it makes a complicated world much less so”
   By now we all know that looking at website statistics doesn’t help us. Instead, we have to think and decide what to segment and where to drill down, resulting in actionable items that improve our website conversion rate.

What is surprising is that this is not an Internet or web analytics book. It is an Economics–or what they call Freakonomics–book. Many times we can learn a lot about web analytics from other fields.

Listen to Steven Levitt speak about crack

Web analytics and economic analysis can be dangerous

The really difficult thing about web analytics is point no. 5. Most people know that this is a challenge but unfortunately they don’t know that the most dangerous problem is how to measure.

Measuring the wrong way is much too common.

Steve Levitt and Stephen Dubner fall into this same trap.

After analyzing what matters in parenting they find out that having been adopted matters. Studies show that a child’s IQ is much more influenced by the biological parents than the adoptive parents (page 171). Now if we left it at that we would think that the adopting parents don’t have much influence on the adoptive child.

Luckily, Levitt decided to dig deeper maybe because he didn’t like the results. This happens a lot in web analytics—we don’t like the results so we dig deeper until we find what is really going on. This is ok. However you should also do this when we like the results even though it serves our agenda (see point 4 above). You do want to find out the truth, don’t you?

Getting back to the adopted baby, although he did poorly in school, another study showed that by the time they became adults they “…veered sharply from the destiny that IQ alone might have predicted. Compared to similar children who were not put up for adoption, the adoptees were far more likely to attend college, to have a well-paid job, and to wait until they were out of their teens before getting married. (page 176) ”

So the adoptive parents did matter after all. It is good that Levitt decides to dig deeper. Or maybe his decision of what to measure was wrong (see point 5 above). Maybe instead of measuring success in school he should have been measuring college attendance, jobs and marriage.

It seems like economists make the same mistakes we web analytics people do.

Read the book to get inspired about web analytics.

How Doctors Think by Jerome E. Groopman, M.D., photo credit: nele’s photostream

It seems like I am not the only one to be interested in the way doctors model their thinking. In his book “How Doctor’s Think” Jerome Groopman tells of a hand problem he had. He visited a few doctors but none inspired confidence with their diagnosis or lack thereof. Finally, a young doctor decided to compare both of Doctor Groopman’s hands – an innovative idea — and found the problem. I like that idea and use it frequently in web analytics.

By comparing two things your brain sees things it wouldn’t otherwise think of

Many times when we are looking for new insights on a web site we compare. Comparison examples include

• Conversion paths for different languages
• Conversion paths of PPC vs natural search
• Two keywords that land on the same page

What have you found useful to compare?

I ran across Bounce Rate articles on the web recently and see that many people are wasting their time with this metric. A high bounce rate can be because:

• The referring website is low quality. This happens a lot with social media sites
• The search engine is sending traffic through its images feature
• The keyword is not that relevant to your product offering but your web site ranks high on the search engine
• The traffic is coming directly. This could be from bookmarks but also from hackers, employees or robots that you web analysis software is not filtering out
• Loyal visitors may only come to see what is new (especially from a newsletter) and then leave

A high bounce rate in these cases is not something to worry about.

Bounce rate is almost meaningless unless you drill down

Avinash has put forward good ideas but in most cases we need to go further. Drill down to the:

• Keyword. This is obvious
• Page. This is not enough. When looking at the page drill down to the keywords and traffic sources
• See how the bounce rate changes over time. Map this to previous years to eliminate seasonal influences. Then map changes in bounce rate to changes made on the page, traffic source and keyword changes. This will provide further insights. You may find a change you made to a web page that you forgot to track that either caused much damage or increased success rate dramatically
  
  
  Tracking bounce rate over time can reveal website changes that were overlooked and cause damage. Track by page and by segment for best results.
• Cost of wasted money from Google Adwords and other PPC campaigns. Multiply the bounce rate by the spend for each page but don’t stop there. Analyze by keyword and ad too

After your analysis you will find website pages and/or traffic sources to optimize. Don’t forget to track the improvements you made to make sure your changes bring the desired results.

I know someone who had a dating service before the internet became popular. It was a very personalized service. She always had great stories (anonymous of course) and there was one in particular that has always stuck with me.

Once she was yelled at by an irate customer. She had just come back from a date that my friend and her partner set up. She was furious that she was set up with a smoker when she had checked off non-smokers only on her questionnaire. My friend gently explained to her that she was only human and makes mistakes. This did nothing to placate the woman yelling on the other end of the line.

Does Social media replace the grapevine?

Guess what? This furious woman ended up marrying the smoker! Of course, she didn’t have the courtesy to tell my friend but she is well connected and heard about it through the grapevine (the pre social media way of finding out things in your neighborhood).

This proves that the application of multivariate testing is desperately needed in the online dating scene. Today, a nonsmoker who checks off that they will not tolerate a smoker will not go out with a smoker. Since chemistry and other important items are not something that can be checked off online it takes a back seat to less important traits that are easier to define—making match making more difficult.

This means you are optimizing locally but may be missing out on the best optimized solution. It could be that the smoker is better than the others on every count but smoking. However you never even gave him a chance because he had one of those traits that is easy to check off–online.

If you do not do multivariate testing you may not find the optimal solution

Multivariate testing for the dating scene

The obvious solution is to test all the guys simultaneously for all important traits. Unfortunately this is not possible with dating so we need to do other tactics—a subject for another post and on a different kind of blog.

Website testing and eliminating elements

Sometimes a bad element on the page may work well in combination with other elements. So be careful of eliminating things you may think are bad. For example, I recently read of a case where eliminating the coupon code field increased conversions dramatically.

A promotional field may decrease conversions but that does not mean you should eliminate it

The thinking is that a lot of the people didn’t have coupon codes which made them feel like suckers so they didn’t order. Instead of eliminating the coupon code they could have tested these options of adding a note next to the coupon box:

• After your 1st order we will send you a coupon code
• Sign up for our newsletter and get coupon codes

If you are doing AB testing you must take into account that you may be optimizing for a local peak. Don’t take the results at face value. Think through what the reasons are and prepare your test.

Time on site is frequently used as a conversion proxy when conversions rates are too low to be statistically significant. Avianah Kaushik’s Occam’s Razor blog entry shows how to segment visitors to get meaningful actionable data.

Here is a table showing different keywords. It is clear that keyword 4 has the best time on site. All things being equal, this keyword seems to be the best.

Segmenting traffic by keyword show which one has the highest time on site.

Let’s go further. Lets take out the people who bounce and then segment. Now we can see that keyword 2 has the people with the highest time on site.

By filtering out bounce rates we can see which segment is most important

If we have a B2B lead generation web site with high value products that do not have a lot of sales this segment may be the most important and should be focused on. This is because the quality of the visitors is much more important than the quantity. Using your web analytics solution for drilling down  is great for finding actionable items.

A lot of people enjoyed our SEO Quiz, however we did get one negative response after over a year:

“that is, without a doubt, the most ridiculous quiz i have ever seen.
i took it and followed it only to see what your were pitching.
the quiz proved nothing whatsoever
think about that again.”
BL

(Thank you B.L. Ochman for giving permission to print your reaction.)
I think it illustrates the importance of feedback and the limits of web analysis. We learn the most from negative criticism and it is important to solicit these kinds of comments. Many people wrote that the SEO Quiz was funny or even hilarious. While it is nice to read positive feedback, it doesn’t help as much in a practical way.

This negative response has prompted me to realize that we should be more aggressive in obtaining negative feedback. We need to adjust our website feedback forms to encourage more negative criticism.

As for BL’s comment—it got me thinking. Maybe I should be pitching something at the end of the SEO quiz.