Rise and Fall of the Yahoo! Web Analytics Forum

The Web Analytics Forum on Yahoo! Groups was started back in 2004 by one of the giants of the web analytics industry, Eric Peterson. Since then, it has been a major hub for web analytics community news, events, and discussion – so much so that Avinash Kaushik was included “You have not only heard of the Yahoo! Web Analytics group, but 20 minutes of each day is spent reading all the posts”  as item #9 in his Top Ten: Signs You are a Great Analyst.

After being out of the loop of the Web Analytics Forum for a while due to some email changes, I was recently reminded of the relevance of this forum and subscribed again for daily updates. While doing so, I noticed some interesting stats on message history that are shown at the bottom of the home page. Being the type of person who processes data much better when it is visually represented, rather than being shown as numbers in a table (as I believe most of us are), I dropped the data into Excel and created the quick chart shown above.

I was disheartened to see that although message volume on the forum grew fairly steadily from inception in 2004 up until April, 2008, it has trended down since then. I have to admit that I have never posted a message to the forum, so I am as much a part of the problem as anybody, but from a community member perspective it is disappointing to see this resource waning. No doubt it is at least partly a result of the rise of social media and the resulting fragmentation of communications, but the Web Analytics Forum has a valuable role to play, in my opinion, as a central repository of current – and historical – information related to the practice of web analytics.

So I will continue following the Web Analytics Forum, looking for opportunities to jump in when I have something useful to add, and I hope others will do the same.

And to the forum founder Eric Peterson, as well as all those who have followed in his footsteps and contributed valuable information and insights to the Web Analytics forum in the past, present, and future…THANK YOU!

Choosing Indicators: Separating KPIs from KRIs

Distinguishing between Key Performance Indicators (KPIs) and Key Result Indicators (KRIs) can improve clarity and help focus attention where it matters most.

One of the most important components of effective web analytics is identification of key performance indicators that can provide a focal point for site improvement amidst the multitude of data points that are available.  Important as it is, though, there are still lots of organizations who don’t have their KPIs fully worked out. This despite lots of attention being drawn to this area, through articles like Avinash’s “KPIs to Die For” and, of course, Eric Peterson’s “Big Book of Key Performance Indicators” (now available free!).

In his book ‘Key Performance Indicators: Developing, Implementing and Using Winning KPIs‘, David Parmeter makes the point right on page 1 that there is a difference between key result indicators and key performance indicators.  These two types of indicators are often confused, but it is important to understand this difference and to delineate it in reporting, because while results are what we ultimately want, performance is where the action is.

Key Result Indicators

These are the measures that tell you how well you’ve done in terms of your larger objectives and provide “a clear picture of whether you are traveling in the right direction”, as Mr Parmeter says.  You can view these as being at the strategic level, measuring how well the chosen strategy is working. For web analytics, they are connected to company objectives for your website, and so would be things like:

  • online revenue (ecommerce)
  • leads generated (lead gen)
  • visits (publishing).

These are obviously critical to track and report on, especially to those higher up the corporate ladder. They may be viewed relative to targets that have been set. But they do not tell you what to do to improve. This is where your key performance indicators come in.

Key Performance Indicators

The reason these are called ‘performance‘ indicators is that they are directly pointing at actions being performed.  The reason they are designated as ‘key‘ performance indicators is that they are critical to achieving our desired results. These are metrics at the tactical, operational level that may be far removed from key results, but nonetheless impact them. Because of their connection to operational activities, when one of our key performance indicators goes awry, we know what to do and it drives us into immediate action.

Parmeter gives the example of British Airways and plane delays.  In the 1980’s, BA apparently determined that of all the things that go into running a successful airline, delayed departure of planes created a cascading effect that undermined a ton of other activities and ultimately inhibited the company from achieving financial goals. Such was the focus on this KPI that if a plane was delayed beyond a certain threshold, a local manager would receive a call from a senior executive. Action taken (by the senior executive…and presumably by the local manager if he wanted to keep his job).

Identifying Key Performance Indicators

Identifying key performance indicators starts with clarifying your goals and the results that indicate your progress toward them, and then digging underneath those result indicators to uncover the performance aspects that have the biggest difference on the result indicators. The KPIs are not necessarily obvious, and the process of identifying and agreeing on them may require some extensive consultation with stakeholders.  Here is a basic framework with simplified example that hopefully gives a sense of the minimum requirements for developing effective KPIs:

  1. Start by clarifying your goals. (increase monthly revenue from search engine visitors by 20% within 6 months.)
  2. Define key result indicators that reflect your goals. (revenue from search engine visitors)
  3. Identify key aspects of performance that are critical to achieving the desired results. (number of search engine visitors, percentage of visitors that make a purchase, value of orders placed by visitors who make a purchase)
  4. Derive key performance indicators that reflect effectiveness or efficiency of these aspects of performance. (% increase in search engine visitors, conversion rate, average order value)

Once the KRIs and KPIs have been identified, it is then a matter of reporting on them, incorporating analysis of what is happening and what is being done to improve the situation. After all, there’s no point in identifying KPIs if you don’t have information-rich, easily distributable reports for monitoring trends and correlations and spurring decision-making.

Is that all there is?

None of this is to say that other data collected but not deemed to be key indicators should be ignored.  The point is to create some focal points for further analysis. Is conversion rate going down? Then let’s look at what search engines or keywords are responsible. Let’s check landing pages. Are products out of stock? Because we have key performance indicators to set the context, we can hunt down root causes and make recommendations that will flow through to improved results.

In addition, though, we still want to keep an eye on other metrics to spot changing trends that may either offer opportunities or portend troubles ahead.  There are lots of indicators that are not necessarily ‘key’ but could be relevant depending on the situation. But, in the absence of unlimited time for analysis, these take a backseat to thoes ‘key’ indicators that we should be monitoring religiously.

Final Word on KPIs

One more thing to consider is that in the constantly-changing world of online marketing, today’s KPI may be tomorrow’s irrelevant data point. And even the KPIs you choose today may not be as meaningful as originally thought: there may be others that get closer to the heart of what drives critical success factors.  So it’s a good idea to periodically review your KPIs for relevance, and even keep a couple alternatives in your back pocket in case they may prove to be more useful.

Moral of the story…there is no final word on KPIs.

WAA Certification: More than Just an Exam

WAA-certification-logoRecently the Web Analytics Association launched a ‘Certified Web Analyst’ program. Although at first glance it may seem that passing a test is all that’s required, there are in fact some significant education and experience requirements as well.

It’s All About Credibility

In the works for two years, the program is designed as a ‘mechanism to elevate professional standards and recognize professionals who have demonstrated their knowledge of the web analytics industry.’  Cool…independent, third party validation that someone who is promoting him/herself as a web analytics professional actually knows his/her stuff.  Whether you’re in the trenches trying to be taken a little more seriously, a consultant trying to landing new clients, or just generally trying to advance your career, who can’t use an extra shot of credibility?

Writing the Exam

The center-piece of this program is an exam that applicants must pass in order to receive the designation.  Currently available only at proctored exam locations (eMetrics, similar conferences), pen-and-paper style, they’ll be moving to more widespread computer-based testing centers in the US and beyond throughout 2010.

Education + Experience Requirements

I initially had the impression from the promo emails, etc that writing the exam was all there was to it, but that’s only the tip of the iceberg.  Eligibility requirements include one of two combinations of education + experience:

1. High-school + 5 yrs online business experience, incl 3 yrs of web analytics.


2. 4-yr degree + 3 yrs web analytics experience.

Interestingly, there is no accommodation in the education requirements for the UBC Award of Achievement in Web Analytics , which has been offered for several years now in conjunction with the Web Analytics Association (which provides course content and tutors). In fact, 3 of the 4 courses in the UBC program (all but the intro course) comprise the body of knowledge that is tested in the exam…

Body of Knowledge

So, if you have (or are well on your way) to satisfying the education and experience requirements, you want to know what is going to be on the exam before you shell out your $600+ exam fee, right?  If you have taken the UBC web analytics courses mentioned above, you are in luck, as this material forms the basis for the exam questions.  If you haven’t taken UBC courses, or if you need to refresh your memory, the WAA has provided a very helpful document called ‘Knowledge Required for Certification‘ that provides an overview of the subject areas to be covered on the exam.

In fact, from quickly flipping through the 37 page document, I’d say it provides a nice overview of the kinds of things you should know if you are/aspire-to-be a web analytics professional – whether you plan to take the exam or not. It doesn’t cover everything in the web analytics universe and it is maybe a little heavily oriented toward the clickstream in a web 2.0 world, but the essential foundations of web analysis are there.


With the combination of education + experience + exam requirements, the weight of the WAA, and the top-shelf web analytics minds that have gone into establishing this program, it seems to me that this is more than just another certification.  I think it will prove to be a valuable program that represents a nice milestone in our industry.

I look forward to writing the exam and (hopefully) getting certified as soon as the opportunity arises, and hope the experience – and rewards – are positive for everybody else who goes for it.

Meantime, I’m interested to hear any thoughts from those who have taken the exam, plan to, or plan NOT to.

3 Charts for a Potent PPC Dashboard

Whatever tool(s) you use to manage your PPC campaigns – search engine interfaces, desktop clients, campaign management software, or some combination – you get a lot of analysis and reporting power.  However, virtually all of these tools still come up short in terms of providing concise yet insightful overviews of PPC activity that can be presented to stakeholders on a periodic basis and quickly comprehended.

So you inevitably end up exporting data to Excel for manipulation. But then what?

Here’s an approach relies on 3 easy to interpret charts that cover all the essentials: overall volume metrics, cost efficiency metrics, and operational efficiency metrics.  While these charts may not answer all the questions we need answered, they provide a comprehensive summary of campaign activity that can be used to quickly:  a) get a sense of what is working (or not) and b) figure out where the next level of analysis needs to go. (The example below is for a lead generation site, but you could easily add revenue/ROI metrics for an ecommerce site.)

1. Cost & Volume

First thing everybody wants to know is: how much money are we spending? Second thing is: what are we getting for the money?  This chart answers those questions in a straightforward fashion, using 2 vertical axes as necessary.

ppc dashboard chart - cost and volume

Pretty obvious what is going on here (the point is to make it obvious): spend is up dramatically in Dec and clicks are increasing even faster.  In this case, a result of broader use of the content network. Cost has been close to budget – may be within tolerance or may require further explanation. Conversions also increased strongly, although we have to use some caution in comparing the rate of change, as conversions are on a different axis.  Still, positive movement all around (hence the green number disc), so let’s move on to the next chart.

2. Cost Efficiency

ppc dashboard chart - cost efficiency

Here we move from the ‘how much bang?’ to ‘how much bang for our buck?’  Cost per click has come down due to our careful bid management and, more importantly, cost per conversion is trending downward as well.  However, there is still work to be done, as we can see the cost per conversion is above target (even though the target increased slightly for the Christmas season).  Let’s keep that in mind as we move on to the next chart. (The number disc is yellow – reminding us that we need to watch this one.)

3. Operational Efficiency

ppc dashboard chart - operational efficiency

Now we take a look under the hood to see how the engine is running (to switch metaphors in mid-stream). Our click rate was already disturbingly low (not good for quality score) but we are somewhat okay with that as we are aggressively qualifying visitors for this campaign.  Not surprisingly, with more content network we are seeing even lower click through rate in December than November.  More distressing: slide in conversion rate.  Previously hovering around our target, it has now dipped way below.  So our engine is in need of a tune-up – and the number disc is marked in red to focus our attention.  All things being equal, if we can get the conversion rate up, our cost per conversion will decrease, fixing the problem highlighted on chart 2.

Next Steps

With this clear overview of the status of our PPC campaigns, we can identify the next steps to take in order to figure out how to improve performance.  These are likely to include:

  • Looking at the traffic sources to see if we need to pull back advertising spend and/or lower bids on search engines that are not performing.  Maybe the content network is not giving us the quality we need at a cost we are willing to pay? Maybe we can improve this by weeding out high traffic/low conversion placements.
  • Drilling down into the campaigns to zero in on the campaigns/ad groups/keywords that are contributing most to the lower conversion rate.  Does messaging need to be adjusted?  Do we need to refine keyword match types?
  • Following through to landing pages and looking at bounce rates to see if there the messaging is appropriately aligned and conversion path unobstructed.

The main thing is, we have a solid foundation on which to proceed and a consistent framework that we can use for assessing next month’s results.

Numbers are Not Enough

The numbers are critical, but they rarely tell the story by themselves.  It’s important to include analysis along with the charts.  One of the reasons for number the charts is so that comments can be associated directly with a chart.  It’s best when these comments are maintained on the dashboard over time for continuity.

Gettin’ Certified: Google Analytics & AdWords

Observations on Google Analytics IQ and Google Advertising Professional Qualified Individual Exams

In the search marketing business we all work hard to deliver the goods and continuously improve our skills – we need to in order to  stay relevant in a rapidly changing environment and to demonstrate our value to our employers/clients.  We can let the quality of our work speak for itself, but  it can also be useful to have some more official validation of our general knowledge of our domain.  Which is where certification fits in.  Sometimes required by employers,  sometimes essential in establishing credibility with clients, and sometimes just good to have in the back pocket.

Recently I completed two certification exams in order to ‘prove my proficiency’ in the wonderful world of Google: Google Analytics Individual Qualification (IQ) and Google Advertising Professional Exam.  So I thought I’d share my experiences and observations with others who may be pursuing either or both of these designations.

Google Analytics Individual Qualification (IQ)

The Google Analytics Individual Qualification was announced on the Google Analytics Blog in March, 2009.  It is earned simply by passing the Google Analytics IQ test.  This is meant to prove that you have proficiency in using Google Analytics.  The idea is to separate beginners/casual users – who can, after all, make quite effective use of Google Analytics due its user friendliness – from those that leverage GA more fully to enhance organizational performance and teach others to do so.

In order to prepare for the test, Google offers ‘Conversion University‘, delivered via a series of free, short, on-demand online videos.  This is a pretty comprehensive look at most of the key features of GA and provides a good grounding in how to use the program.  It is useful to go through the videos even if you have no intention of the writing the test, especially because the broken into highly digestible chunks.

Although recommended, you don’t need to go through Conversion University to get to the test – you can write the IQ Test anytime.  It is delivered online at your convenience, costs $50 and you have 90 minutes to complete about 100 multiple choice questions.  Once you pass, they send you a colorful certificate that you can print out and hang in your cubicle. Ta da…


Google Advertising Professional Exam

The Google Advertising Professional Exam is one of the requirements for becoming a Qualified Individual in the Google Advertising Professionals program.  (The other requirements relate to mainly to maintaining a minimum level of AdWords account activity.)

Again, Google provides some thorough resources for exam preparation through the AdWords Learning Center.  The Learning Center has a series of online lessons in 9 sections that are available both as video or text lessons.  Personally, I focused on the text lessons, which I expedited by clicking on the ‘Print Section’ link to see the whole section on one page, rather than broken up into a bunch of short pages.  A nice feature of the Learning Center is that there are short multiple choice quiz questions at the end of each lesson, which helps to a) reinforce the learning and b) prepare for the exam, which is very similar in style and content.

As with the Conversion University, the AdWords Learning Center contains a lot of good information on getting the most from AdWords and is useful even if you are not planning to write the exam.  It does, however, also have a lot of information that may not be relevant to you – such as billing policies in foreign countries or tactics for ‘selling’ AdWords to potential clients.

If it is recommended to take review Conversion University prior to taking the Google Analytics test, I would rate is as HIGHLY recommended to go through the Learning Center – at least the quizzes – before taking on the AdWords exam.  When you are ready, you can jump into the exam via the ‘Pro Center’ tab in your My Client Center.

The AdWords exam has a bit of a bad reputation for being sprinkled with questions that are of limited relevance and/or trickily worded to create ambiguous meaning.  This was an assessment conveyed to me by both colleagues and others in the industry, such as the bloggers at SEO Speedwagon (which is the best seo-related blog name I’ve come across in a while).  While there were definitely some of these questions cropping up, I seemed to me that Google may have responded to some of the criticisms and tightened up the test to make it a more accurate reflection of professional competence.  Overall, I found it a pretty accurate reflection of the Learning Center – even to the extent of some Learning Center quiz questions showing up in the exam.

Bottom Line

Based on my experience, and discussions with colleagues, there is about 9-12 hrs of study involved in each of these exams, but may be less depending on your level of knowledge/experience with these programs.

Both Google Analytics IQ and Google Advertising Pro qualifications have an expiry date: AdWords offers a better deal because it is good for 2 yrs, while you will have to retake the Google Analytics IQ exam in 18 mths to retain your status.

Hard to put a value on these certifications but they do represent tangible evidence that you probably know what you are talking about and can enhance the credibility of yourself and your organization.  Hey, if you can get Google to vouch for you, why not go for it?