Chinalytics Longmarch Blog

You have a great web analytics tool? Check.(That was easy!) You have smart people that not only report data but provide insights? Check. (Congratulations, these are hard to find) Now the main challenge for your team is driving change into the organization (our your clients), make recommendations and make sure these recommendations are implemented. This is often more about communication, politics and strategy than the common egghead image of web analytics suggests.
For an interesting take at how to get started, take a look at this post over at Occam’s Razor

Google is doing a great job in making Analytics accessible for everyone with Google Analytics. Now the key challenge  is how to use this new wealth of data effectively to drive change and increase the ROI. An interesting resource I recently came across is the "Google Conversion University" playlist at Youtube. Take a look at all these videos, but if you short on time, get the quick fix and watch Avinash talking about "Bounce Rate" my most favorite metric of them all. (note to self: find a topic to disagree with Avinash). For Chinese users, make sure you fire up your proxy before enjoying the wealth of learning available there.
And if you really want to do me a favor, share your experience with bounce rates for campaign landing pages with me in the comments or by mail florian-dot-pihs-at-ap-dot-mccann-dot-com.

During the Ad:tech event here in Beijing I had a chance to catch up with a couple of good friend in the industry. It was good to see yo guys! During our semi-sober discussion about whats good whats not, we all agreed that the development of effective web marketing in China is limited by a lack of a strong measurement and analytics foundation. Without a solid underpinning of ROI, web marketing will not be able to flourish.
There was not real breakthrough on the Analytics front in the China World Hotel, and Analytics panel was almost too sad. So things need to change. The first step we believe is distributing analytics knowledge more effective and to evangelize better (see my post on "Why Google and Baidu need a web analytics evangelist in China". To get things starter, Nguyen Le of WanMo, T. R. Harrington of Darwin Marketing informally started the Chinese Web Analytics Association over dinner at Purple Haze. While the irony that non of us is Chinese, is not lost to me, its a good start. And this sorry state will hopefully change soon (I mean with new members, not with changing passports). I will keep you updated in this blog on the progress and hope to hear from you if you have any recommendations in the comments section.

In one of my earliest post I mentioned that we are currently testing GA for some our sites. I am still impressed by the tool and what it allows web analyst to do free of charge. Some of they key benefits I see is so far

• Easy & quick to implement (especially when you track many sites, or third party coop partners this is critical)





• User friendly interface in English but also in Chinese (e.g. localization)





• Easy access to segmentation and comparisons, which allows quick on the spot analysis





• Great geo visualization. Each country and city within the country is displayed on clear map. This allows great segmentation based in city tier and provide. ( Other tool do that as well, but I have never seen it visualized as well)

So far my only gripe is with the geo segmentation. We have run 2 campaign with large partners, and implemented GA in their site. In both campaigns (Among the several million visitors), the most popular location is WuZhong city. "a prefecture-level city in the Ningxia autonomous region of the People’s Republic of China." according to Wikipedia. This just plain makes no sense. Beijing I can understand, Shanghai, Guangzhou, Shenzhen? Maybe. But Wuzhong? It seems some 3rd party data vendor has messed up, or its just all the unknowns that are put into the Wuzhong bucket. Ningxia jokes anyone?
Anyway, GA is a great product, and wider acceptance (with smart analysis) in China will certainly drive the ecosystem to become more effective and rational.

After a couple of esoteric post, this is somewhat more practical. At least for those of you in China, who are usering Omniture’s SiteCatalyst tool (What, that leaves…. only myself and our team, anyways).
In a recent campaign we discovered that 5 out of the top 10 next pages of our campaign landing page were actually other campaign landing pages of campaigns we were running during the analysis period. There was no direct link on our landing page to any of the other landing pages, so it took some time to puzzle this out.
Out best thesis is the following:

Omniture’s understates a landing page bounce rate significantly if,

1. Your campaigns has a high bounce to begin with AND





2.  You are running several ad campaigns at the same time,
   especially when the campaigns run on the same media AND





3. You are in China. None of our other geos had this problem, which is troubling…

Since Omniture uses a 30 min session timeout, users that
come to a landing page, bounce from there, and click on another campaign banner that lead to the same site within 30 min, are not calculated as bounces. In this situation the 2^nd
landing page will be treated as a “Next Page” in SiteCatalyst’s page flow. See graph (click to enlarge):

Taking these “Fake Next pages” into account, we found that our "Non bounced visits" decreased by more that 50%, which basically turns our conclusion on its head. Since we could not directly calculate the bounce rate for the fake next pages (it is not clear how many these fake next pages were coming for entries, as compared to in site traffic) we use a metric that should resemble the bounce rate as closely as possible: Bounce Rate* = (Exits + Fake Next page) / Visits. Its not the WAA definition, but a man has to do, what a man has to do. So here we are, with an even higher higher bounce rate, that almost seems irrational. I will keep you posted on our progress in eating this elephant.

I am currently reading Robert Pirsig’s book Zen and the Art of Motorcycle Maintenance. And it’s description of Scientific Method made me think that practitioners of Web Analytics really are scientist is a way that the good ones follow the scientific method closely.
The book describes 6 steps that are followed to find "truth" (some use 4 or 8 steps, but the underlying process remains the same)

1. Statement of the problem (scientific question)





2. Hypothesis of the cause of problem





3. Experiments designed to test the hypothesis





4. Predicted results of experiments





5. Observed results of experiments





6. Conclusion from the results of experiments

The key insight from using this method is that each step is critical and build on the concise execution of the previous steps. Any mistake or oversight along the way will result in a wrong conclusion. What is interesting is that I often find the first step most difficult. In order to state a problem effectively we need to be able to answer to questions I have posted before in a previous attempt to create a more formal methodology:

1. What is the current performance?





2. Is it good or bad?

Only if we answered question 2 well, we will be able to formulate an effective problems statement. That might sound easy at first, the devil is in the detail.

It seems Avinash stirred up a lot of controversy with his post on engagement (see Sunday’s post). In a series of thoughtful comment I want to highlight Gary Angels response, which is leading the pack in terms of clarity among analytics experts that defend "Engagement" as a metric. I especially like what Gary responds to Avinash’s last point. He says:

• Engagement isn’t a proxy for measuring an outcome on a site at all. It’s more often a means of aggregating a set of outcomes into a single visitor score or segment. Since a set of outcomes can’t reasonably be described by just appending all of the events into one long name, the analyst is always going to have to pick a name that reasonably represents the overall concept. In many situations, and for many sites, “Engagement” does a pretty good job of carrying that baggage

My conclusion is that measuring Engagement can be an effective tool in the web analytics toolbox. But like every tool, it can used for good and bad purposes (much like nuclear energy, I guess). In order to use Engagement effectively we need to be aware of certain preconditions and caveats.

• Engagement is not a standard metrics that is the same for every company, or even every campaign of one company, which often makes it difficult to compare and improve on it over time.





• Engagement should never be an excuse not to define, track and analyze specific outcomes of a page. Without a clearly defined objective and target action a landing page has no value.





• What Engagement can do however, is summarize a set of outcomes (e.g. Video played, PDF downloaded, link clicked) and the compare this set of outcomes across different visitor segments (e.g. search vs. banner ads, media A vs. Media B, Landing Page A vs. Landing Page B). This can lead to valuable conclusions and actions.

Now the main challenge for non e-commerce sites is to prove beyond doubt, that a higher engagement leads to a better, faster or less expensive achievement of the objective. That again is very unique across companies and will often involve additional surveys.

Due to the Chinese National Day holiday I am little late on this one, but Avinash Kaushik just earned another feather in his guru cap with his post on engagement as a web analytics metric. His main points are:

1. Engagement is not a metric that anyone understands and even when used it rarely drives the action / improvement on the website.
2. Because it is not really a metric, it is an excuse. An excuse for an unwillingness to sit down and identify why a site exists. An excuse for a unwillingness to identify real metrics thatbmeasure if your web presence is productive.
3. It is nearly impossible to define engagement in a standard way that can be applied across the board. Definitions that exist are either too broad (to cover every nuance) or too narrow (hence very unique)
4. At the heart of it engagement tries to measure something deeply qualitative. Yet most efforts to measure it in our world tend to be hard core quantitative

Its great to see someone spelling it out so clearly, so please go ahead and read the whole post.

My own experience trying to implement Avinash’s suggestions for our own clients, suggests a deeper meaning between the lines of this post. An answer to the question “Why do Web Analytics practitioners still use Engagement as a metric, when most of us should share Avinash’s skepticism?”
Its because in most companies Web Analytics is in a silo, where it is not involved in strategic planning and decision making. Following Avinash’s methodology we end up asking strategic questions over an over again, but in many companies web analysts are not expected and / or not allowed to ask questions like:

• What is the objective of your site?
• What is the objective of this campaign?
• Can that objective be measured by analyzing clickstream data?
• What action do you want the user to take on the landing page?

These questions are owned by other silos, so we end up defining metrics that are under our control. We look at the data from our web analytics tools and hope to gain additional insights from “Engagement” however we define it. This may not solve the problem, but it is within our circle of influence.

So Avinash, what I hear your really saying is this: “Web Analysts of the world unite, and increase your circle of influence. Make Web Analytics a strategic function”. Oh, and one more thing: Don’t call your approach Web Analytics 2.0. Call it by its real name. Don’t hide behind a pretty moniker. Call it Strategic Web Analytics