I know someone who had a dating service before the internet became popular. It was a very personalized service. She always had great stories (anonymous of course) and there was one in particular that has always stuck with me.

Once she was yelled at by an irate customer. She had just come back from a date that my friend and her partner set up. She was furious that she was set up with a smoker when she had checked off non-smokers only on her questionnaire. My friend gently explained to her that she was only human and makes mistakes. This did nothing to placate the woman yelling on the other end of the line.

Does Social media replace the grapevine?

Guess what? This furious woman ended up marrying the smoker! Of course, she didn’t have the courtesy to tell my friend but she is well connected and heard about it through the grapevine (the pre social media way of finding out things in your neighborhood).

This proves that the application of multivariate testing is desperately needed in the online dating scene. Today, a nonsmoker who checks off that they will not tolerate a smoker will not go out with a smoker. Since chemistry and other important items are not something that can be checked off online it takes a back seat to less important traits that are easier to define—making match making more difficult.

This means you are optimizing locally but may be missing out on the best optimized solution. It could be that the smoker is better than the others on every count but smoking. However you never even gave him a chance because he had one of those traits that is easy to check off–online.

If you do not do multivariate testing you may not find the optimal solution

Multivariate testing for the dating scene

The obvious solution is to test all the guys simultaneously for all important traits. Unfortunately this is not possible with dating so we need to do other tactics—a subject for another post and on a different kind of blog.

Website testing and eliminating elements

Sometimes a bad element on the page may work well in combination with other elements. So be careful of eliminating things you may think are bad. For example, I recently read of a case where eliminating the coupon code field increased conversions dramatically.

A promotional field may decrease conversions but that does not mean you should eliminate it

The thinking is that a lot of the people didn’t have coupon codes which made them feel like suckers so they didn’t order. Instead of eliminating the coupon code they could have tested these options of adding a note next to the coupon box:

• After your 1st order we will send you a coupon code
• Sign up for our newsletter and get coupon codes

If you are doing AB testing you must take into account that you may be optimizing for a local peak. Don’t take the results at face value. Think through what the reasons are and prepare your test.

Time on site is frequently used as a conversion proxy when conversions rates are too low to be statistically significant. Avianah Kaushik’s Occam’s Razor blog entry shows how to segment visitors to get meaningful actionable data.

Here is a table showing different keywords. It is clear that keyword 4 has the best time on site. All things being equal, this keyword seems to be the best.

Segmenting traffic by keyword show which one has the highest time on site.

Let’s go further. Lets take out the people who bounce and then segment. Now we can see that keyword 2 has the people with the highest time on site.

By filtering out bounce rates we can see which segment is most important

If we have a B2B lead generation web site with high value products that do not have a lot of sales this segment may be the most important and should be focused on. This is because the quality of the visitors is much more important than the quantity. Using your web analytics solution for drilling down  is great for finding actionable items.

A lot of people enjoyed our SEO Quiz, however we did get one negative response after over a year:

“that is, without a doubt, the most ridiculous quiz i have ever seen.
i took it and followed it only to see what your were pitching.
the quiz proved nothing whatsoever
think about that again.”
BL

(Thank you B.L. Ochman for giving permission to print your reaction.)
I think it illustrates the importance of feedback and the limits of web analysis. We learn the most from negative criticism and it is important to solicit these kinds of comments. Many people wrote that the SEO Quiz was funny or even hilarious. While it is nice to read positive feedback, it doesn’t help as much in a practical way.

This negative response has prompted me to realize that we should be more aggressive in obtaining negative feedback. We need to adjust our website feedback forms to encourage more negative criticism.

As for BL’s comment—it got me thinking. Maybe I should be pitching something at the end of the SEO quiz.

When analyzing which ads work better you probably don’t break it down by keyword. This means you may come to the wrong conclusions. We recently compared two ads as you can see in the screen shot below (I changed many of the details to protect the privacy of our client).

Google Adwords AB tests must be analyzed by keyword to avoid reaching the wrong conclusions

The ad on the left had a conversion rate of 1.1% as oppose to the ad on the right which had a conversion rate of 1.8%. However when we break it down by keywords we see that Keyword 2 had:

• A much higher click through rate for ad B
  • In order to improve reliability of the test we run two A type ads and make sure they have the same CTR before we stop testing—this accounts for the higher number of visitors for the ad on the left. If the ads were equal the ad on the left should have twice as many visitors.
• A much higher conversion rate than the other keywords
• A much higher conversion rate than the same keyword has for ad A.

This can skew the results. If we take out keyword 2 we may find that the ads are similar, or even that ad A is better

Different Google Ads for Different Keywords

Look at Keyword 4 and you see that the opposite is true—Ad A is best. Therefore Keyword 4 needs ad A. If  you didn’t do this analysis by keyword you would come to the wrong conclusions and think Ad B is best for the ad group.

The results that you need to remember are:

• For one keyword one ad is best and for another keyword the other ad is best.
• The keywords 2 and 4 need to be in different ad groups so the ads can be optimized for them
• You have to eliminate keywords which have results opposite to the overall trend before analyzing AB ad tests. In a perfect world you want have one exact match keyword per ad group—unfortunately this is not practible
• You have to eliminate keywords yielding vastly different results from the overall trend
• You have to look at each significant keyword separately

You may also ask how these two different keywords ended up in the same ad group to begin with. In a world with time restraints it is not practical to give each keyword its own group. Even if each keyword had its own group you would still have to do the above analysis as people will use different modifiers for each keyword if you use phrase or broad match. That is why the above analysis is based on the actual keywords used by the searchers and not the phrase matched keywords in the ad group.

This is what makes it important to compare ads by keyword.

There are lots of additional insights you can get from the above analysis but I will leave that for another post

Mike Moran thinks that Six Sigma isn’t relevant to internet marketing.

“Six Sigma is an excellent way to deliver high quality with repeatable processes, such as manufacturing your product, but it is exactly the wrong goal in Internet marketing.”

Bryan Eisenberg, on the other hand, thinks that it is important enough to build into the web marketing process.

“It may sound like pure theory, but Six Sigma is practical and yields enormous return on investment (ROI). We apply its principles to Web marketing…”

More…

I found a web analytics book worth reading. Although I knew almost everything Avinash Kaushik wrote, I learned more from this book than any other one source.

Finding out what your web site users think by talking to them comes up many times in the book. This is very important, however smaller companies may not have the resources to do this offline research in a scientific manner. In those cases you could make contact with people who converted. Although this is a very special segment, it is better than not contacting anyone. As Avinash emphasizes, with web analytics most of the data won’t be perfect and we have to base our actions on what we have.

Comparing traffic trends to competitor’s on Alexa is another good idea, but the figures will not be accurate for small companies (and not completely accurate for big ones either as he notes).

MSN drives less traffic to most of my clients, so I spend less time on MSN. Avinash points out one tip I really liked — the MSN search funnel. It shows you what people search for before and after your keyword. This can give you many insights and it is a lot of fun. I tried it with keywords for one of my B2B clients and nothing came up, as I expected. However it is worth checking out for more mainstream keywords.

Although I didn’t count all the graphs shown in the book from different web analytics packages, I have the impression that graphs from ClickTracks outnumber all the others combined. This is probably due to its ease of use and ability to show important information if you know what kind of labels you need to segment visitors. Ease of configuration should not be overlooked when choosing a web analytics package. As you dig deeper into your statistics you will find more and more things to check and configure and if you have to rely on your IT people it will slow you down.

Sites which are designed to generate sales leads are harder to analyze than ecommerce sites as there is no clear way to judge success. One important metric is–how many people contact the company. We call this a hard conversion. However, especially when sales cycles are long, it is also important to measure return visitors, time on site, etc. We call these soft conversions.

What happens when these two metrics conflict? That is exactly what happened to me when analyzing a client’s log files recently. For ad no. 1, the hard conversions were higher, for ad no. 2 the soft conversions were higher. After further analysis the answer to the riddle became clear.

Ad no. one was attracting people who were quicker decision makers as it was geared to this type of persona. Ad no. 2 was attracting people who had a longer and more complicated decision making process so they spent more time on the site and were less quick to contact the company.

Next step is to formulate a strategy to capitalize on this important information.