Choosing Indicators: Separating KPIs from KRIs

Distinguishing between Key Performance Indicators (KPIs) and Key Result Indicators (KRIs) can improve clarity and help focus attention where it matters most.

One of the most important components of effective web analytics is identification of key performance indicators that can provide a focal point for site improvement amidst the multitude of data points that are available.  Important as it is, though, there are still lots of organizations who don’t have their KPIs fully worked out. This despite lots of attention being drawn to this area, through articles like Avinash’s “KPIs to Die For” and, of course, Eric Peterson’s “Big Book of Key Performance Indicators” (now available free!).

In his book ‘Key Performance Indicators: Developing, Implementing and Using Winning KPIs‘, David Parmeter makes the point right on page 1 that there is a difference between key result indicators and key performance indicators.  These two types of indicators are often confused, but it is important to understand this difference and to delineate it in reporting, because while results are what we ultimately want, performance is where the action is.

Key Result Indicators

These are the measures that tell you how well you’ve done in terms of your larger objectives and provide “a clear picture of whether you are traveling in the right direction”, as Mr Parmeter says.  You can view these as being at the strategic level, measuring how well the chosen strategy is working. For web analytics, they are connected to company objectives for your website, and so would be things like:

  • online revenue (ecommerce)
  • leads generated (lead gen)
  • visits (publishing).

These are obviously critical to track and report on, especially to those higher up the corporate ladder. They may be viewed relative to targets that have been set. But they do not tell you what to do to improve. This is where your key performance indicators come in.

Key Performance Indicators

The reason these are called ‘performance‘ indicators is that they are directly pointing at actions being performed.  The reason they are designated as ‘key‘ performance indicators is that they are critical to achieving our desired results. These are metrics at the tactical, operational level that may be far removed from key results, but nonetheless impact them. Because of their connection to operational activities, when one of our key performance indicators goes awry, we know what to do and it drives us into immediate action.

Parmeter gives the example of British Airways and plane delays.  In the 1980’s, BA apparently determined that of all the things that go into running a successful airline, delayed departure of planes created a cascading effect that undermined a ton of other activities and ultimately inhibited the company from achieving financial goals. Such was the focus on this KPI that if a plane was delayed beyond a certain threshold, a local manager would receive a call from a senior executive. Action taken (by the senior executive…and presumably by the local manager if he wanted to keep his job).

Identifying Key Performance Indicators

Identifying key performance indicators starts with clarifying your goals and the results that indicate your progress toward them, and then digging underneath those result indicators to uncover the performance aspects that have the biggest difference on the result indicators. The KPIs are not necessarily obvious, and the process of identifying and agreeing on them may require some extensive consultation with stakeholders.  Here is a basic framework with simplified example that hopefully gives a sense of the minimum requirements for developing effective KPIs:

  1. Start by clarifying your goals. (increase monthly revenue from search engine visitors by 20% within 6 months.)
  2. Define key result indicators that reflect your goals. (revenue from search engine visitors)
  3. Identify key aspects of performance that are critical to achieving the desired results. (number of search engine visitors, percentage of visitors that make a purchase, value of orders placed by visitors who make a purchase)
  4. Derive key performance indicators that reflect effectiveness or efficiency of these aspects of performance. (% increase in search engine visitors, conversion rate, average order value)

Once the KRIs and KPIs have been identified, it is then a matter of reporting on them, incorporating analysis of what is happening and what is being done to improve the situation. After all, there’s no point in identifying KPIs if you don’t have information-rich, easily distributable reports for monitoring trends and correlations and spurring decision-making.

Is that all there is?

None of this is to say that other data collected but not deemed to be key indicators should be ignored.  The point is to create some focal points for further analysis. Is conversion rate going down? Then let’s look at what search engines or keywords are responsible. Let’s check landing pages. Are products out of stock? Because we have key performance indicators to set the context, we can hunt down root causes and make recommendations that will flow through to improved results.

In addition, though, we still want to keep an eye on other metrics to spot changing trends that may either offer opportunities or portend troubles ahead.  There are lots of indicators that are not necessarily ‘key’ but could be relevant depending on the situation. But, in the absence of unlimited time for analysis, these take a backseat to thoes ‘key’ indicators that we should be monitoring religiously.

Final Word on KPIs

One more thing to consider is that in the constantly-changing world of online marketing, today’s KPI may be tomorrow’s irrelevant data point. And even the KPIs you choose today may not be as meaningful as originally thought: there may be others that get closer to the heart of what drives critical success factors.  So it’s a good idea to periodically review your KPIs for relevance, and even keep a couple alternatives in your back pocket in case they may prove to be more useful.

Moral of the story…there is no final word on KPIs.

Advertisements

Careful with that Bounce Rate, Eugene

Bounce rate is a useful metric but can be misleading if not used in context

We all know that bounce rate conveys vital information about site performance and can provide very useful clues about where to look for improvements. Especially since Avinash nominated it as ‘sexiest web metric ever‘ back in ’07. As is often the case with web metrics, though, the numbers shouldn’t just be taken at face value. It can be important to consider metrics definitions and their use by web analytics tools in order to make well-informed decisions.

Here’s a case in point that I encountered recently in a site’s Google Analytics ‘Top Content‘ report:

Bounce Rate in Content Report

Clearly there is a problem here that requires urgent attention: high traffic pages with huge bounce rates! It’s important to keep in mind, though, that bounce rate is calculated only for visits that start with the page in question.  Being the ‘Top Content’ report, the pageviews account for all views of that page, whether the first page in a visit or the 50th.

So the report is NOT saying that of the 2,215 views of the first page in the list,  2,076 (93.75%) of them resulted in bounces.  It IS saying that of all the visits that started with this page, 97.5% of them went no further.  This, then, begs the question: “well, how many visits started with this page?

For the answer to this question, we turn to the ‘Top Landing Pages‘ report in Google Analytics (or equivalent in your tool of choice). And here we see a story that puts things in perspective:

Bounce rate from Landing Pages report

Yeah, that first page in the list has a high bounce rate…but it is based on only 16 entrances compared to over 2,200 pageviews in total! (I know we’re mixing metrics here, but the story is valid.)

Depending on the purpose and value of that page, this still may be cause for concern. But chances are there are bigger fish to fry.

There are other ways that bounce rate – and indeed most web metrics –  can be twisted by context, so it is prudent to keep in mind the nuances behind the data being presented by our trusty tools.

(For those interested, there is further discussion of the ins-and-outs of bounce rates in Google Analytics here on the Google Analytics Help Forum – particularly relevant if you are wondering how to reconcile content and navigation reports.)