In a recent presentation to a group of University of San Francisco MBA's I showed the slide below to explain the web analytics process.


Now it strikes me that the key point I made (see the center of the graph) misses the mark somewhat, since it does not describe the methodology we use to optimize for ROI. Thus the title title of this post. Whatever we do in Web Analytics it comes down to testing, testing, testing. We try out multiple creative ideas, formats, channels, media, landing pages & offers, then keep what works and kill the rest.
In the early days of this blog I theorized about the scientific method in web analytics,  and "experiments" in the scientific method can be equated with "testing". Now however it strikes me, that the "survival of the fittest" of Darwinian evolution is a method that is more efficient, if not more effective. Advanced ad servers, web analytics tools and multivariate testing environments, large amount of online interactions + raw computational power today provide tools, which enable us to quickly experiment with a large number of variables and choose the variations that drive visitors to take action most effectively
Regrettably this brave new world of online marketing has not fully arrived in China yet (although I hear is going to arrive before the Olympics ). The main challenges I see are:

• Unless the client is an e-commerce client (which is very rare in China) or a cost per lead client (less rare but still not mainstream), it is often difficult to define and agree on a success event (action). But as long as you don't have a target to optimize for, it is impossible to effectively test campaigns. Often the first step of an effective web analytics effort will focus on, is working with the client to identify how they define success for an online marketing campaign
  
• Clients are averse to testing, since
  
• they trust their judgment (or their creative / media agencies judgment) to tell them what will work and what won't, more efficiently than their target audience will through testing. The truth is, all their research, focus groups and (more often) gut feeling is not as accurate as a well planned and executed test (actually their are often quite far off). Experience has shown that the most educated approach is following Einstein: "The more I learn, the more I realize I don't know". Therefore I need to test to be sure.
  
• they believe testing wastes money, since creative / web development effort and media money is spent on media, creative and landing pages that will not work as well. This is based partly on the first point, since you have to be comfortable with the current plan to believe that testing wastes money. Experience in the US has shown, and SEM practice globally and in China has proven that well executed test (e.g. following an 80/20 rule, spending 20% of the budget on testing and 80% on running the optimized campaign) will outperform untested campaigns every time by a significant margin.
• The media serving infrastructure and media buying practice does not allow for rapid testing or changes in budget allocation (by media, by position, by creative). What can be tested more or less effectively right now is creative execution, landing page design and call to action. While the media buying process adjusts to the international standard this is a good place to start. Often times landing page testing & optimization alone yield a performance boost of several 100% (if some of the alternatives follow best practices, as most current landing pages in China do not).
  
• Timelines are often not planned to allow for testing, including creating multiple creatives, calls to action or landing pages

If you have experiences about testing, especially if you are in China, please let me know, ideally through a comment on this blog or by email to florian-dot-pihs-at-ap-dot-mccann-dot-com.

And happy Chinese New Year
恭喜发财