Choosing Indicators: Separating KPIs from KRIs

Distinguishing between Key Performance Indicators (KPIs) and Key Result Indicators (KRIs) can improve clarity and help focus attention where it matters most.

One of the most important components of effective web analytics is identification of key performance indicators that can provide a focal point for site improvement amidst the multitude of data points that are available.  Important as it is, though, there are still lots of organizations who don’t have their KPIs fully worked out. This despite lots of attention being drawn to this area, through articles like Avinash’s “KPIs to Die For” and, of course, Eric Peterson’s “Big Book of Key Performance Indicators” (now available free!).

In his book ‘Key Performance Indicators: Developing, Implementing and Using Winning KPIs‘, David Parmeter makes the point right on page 1 that there is a difference between key result indicators and key performance indicators.  These two types of indicators are often confused, but it is important to understand this difference and to delineate it in reporting, because while results are what we ultimately want, performance is where the action is.

Key Result Indicators

These are the measures that tell you how well you’ve done in terms of your larger objectives and provide “a clear picture of whether you are traveling in the right direction”, as Mr Parmeter says.  You can view these as being at the strategic level, measuring how well the chosen strategy is working. For web analytics, they are connected to company objectives for your website, and so would be things like:

  • online revenue (ecommerce)
  • leads generated (lead gen)
  • visits (publishing).

These are obviously critical to track and report on, especially to those higher up the corporate ladder. They may be viewed relative to targets that have been set. But they do not tell you what to do to improve. This is where your key performance indicators come in.

Key Performance Indicators

The reason these are called ‘performance‘ indicators is that they are directly pointing at actions being performed.  The reason they are designated as ‘key‘ performance indicators is that they are critical to achieving our desired results. These are metrics at the tactical, operational level that may be far removed from key results, but nonetheless impact them. Because of their connection to operational activities, when one of our key performance indicators goes awry, we know what to do and it drives us into immediate action.

Parmeter gives the example of British Airways and plane delays.  In the 1980’s, BA apparently determined that of all the things that go into running a successful airline, delayed departure of planes created a cascading effect that undermined a ton of other activities and ultimately inhibited the company from achieving financial goals. Such was the focus on this KPI that if a plane was delayed beyond a certain threshold, a local manager would receive a call from a senior executive. Action taken (by the senior executive…and presumably by the local manager if he wanted to keep his job).

Identifying Key Performance Indicators

Identifying key performance indicators starts with clarifying your goals and the results that indicate your progress toward them, and then digging underneath those result indicators to uncover the performance aspects that have the biggest difference on the result indicators. The KPIs are not necessarily obvious, and the process of identifying and agreeing on them may require some extensive consultation with stakeholders.  Here is a basic framework with simplified example that hopefully gives a sense of the minimum requirements for developing effective KPIs:

  1. Start by clarifying your goals. (increase monthly revenue from search engine visitors by 20% within 6 months.)
  2. Define key result indicators that reflect your goals. (revenue from search engine visitors)
  3. Identify key aspects of performance that are critical to achieving the desired results. (number of search engine visitors, percentage of visitors that make a purchase, value of orders placed by visitors who make a purchase)
  4. Derive key performance indicators that reflect effectiveness or efficiency of these aspects of performance. (% increase in search engine visitors, conversion rate, average order value)

Once the KRIs and KPIs have been identified, it is then a matter of reporting on them, incorporating analysis of what is happening and what is being done to improve the situation. After all, there’s no point in identifying KPIs if you don’t have information-rich, easily distributable reports for monitoring trends and correlations and spurring decision-making.

Is that all there is?

None of this is to say that other data collected but not deemed to be key indicators should be ignored.  The point is to create some focal points for further analysis. Is conversion rate going down? Then let’s look at what search engines or keywords are responsible. Let’s check landing pages. Are products out of stock? Because we have key performance indicators to set the context, we can hunt down root causes and make recommendations that will flow through to improved results.

In addition, though, we still want to keep an eye on other metrics to spot changing trends that may either offer opportunities or portend troubles ahead.  There are lots of indicators that are not necessarily ‘key’ but could be relevant depending on the situation. But, in the absence of unlimited time for analysis, these take a backseat to thoes ‘key’ indicators that we should be monitoring religiously.

Final Word on KPIs

One more thing to consider is that in the constantly-changing world of online marketing, today’s KPI may be tomorrow’s irrelevant data point. And even the KPIs you choose today may not be as meaningful as originally thought: there may be others that get closer to the heart of what drives critical success factors.  So it’s a good idea to periodically review your KPIs for relevance, and even keep a couple alternatives in your back pocket in case they may prove to be more useful.

Moral of the story…there is no final word on KPIs.

Choosing KPIs: Visitors or Visits?

Recently had a client situation where we were providing the client with monthly organic visitor numbers to their ecommerce site. One of my colleagues showed me a report received from the client, which had the same data, but marked as ‘visits‘.  So that got me thinking…what do they really want to measure here: visitors or visits? And do they know? And have they thought about what difference it makes? And, of course, what recommendation can we provide?

Both numbers are important (although they may not be critical – depending on the outcomes you need to measure) and they provide similar information, but there are some important differences. Leaving aside the argument over whether either of these satisfies the criteria to become a real KPI, let’s consider the uses of each metric in the context of this client.

Visitors

I’m pretty sure that since the reporting was done on a one month period, that the tool the client is using reports ‘unique visitors’. (i.e. People – or at least browser cookies – that are only counted once during the period.)

[Side Note on ‘visitors’ in Google Analytics:

For Google Analytics, apparently the term ‘visitor’ is not enough and they even go beyond ‘unique visitor’ to insist they are reporting  on ‘absolute unique visitors’.  Of course, this over-states the case, given the limitation of cookies. But, ok, we get it, this is your best count of individuals visiting the site during a given time period.  More confusing terminology is used in the ‘New vs Returning’ report.  This is reporting visits, rather than visitors (as explained on the Google Analytics Blog) but the term ‘visitor’ is also used.  So maybe it would clear things up to refer to ‘New Visits vs Return Visits’. ]

It is good to know how many people have come to your site, just as it is good to know how many people walk into your store in the mall. It gives you an idea of the total number of customers/potential customers that you are drawing in, and allows you to compare trends over time to spot opportunities or problems.

But there’s a big difference between a person poking their head into your store on their way to the food court, then never to returning again, and a person who repeatedly makes the trip to your store, even if they don’t purchase something every time. And this is where I think visits may provide more relevant, actionable information than visitors for this client.

Visits

As always, metrics that warrant attention vary depending on the nature and goals of a site. The client I’m talking about has a B2B ecommerce site that sells a broad mix of commercial products, including many that represent ‘repeat‘ or ‘modified repeat’ purchases in Buyersphere terms.  So, yeah, it is interesting to know how many people visit the site and to hopefully see this grow over time, but more critical in this case is the number of visits.

We are looking at organic traffic, and we are trying to use search engines to drive as many visits on as many relevant search terms as possible to the site. New visitors, certainly, but if we can capture visits from searchers who already know the site, so much the better, giving us the opportunity to further build on a relationship already established.

Further, we are already using this logic in measuring paid traffic, by counting ‘clicks’.  Not necessarily the same as visits, but likely to be closer to visits than it is to unique visitors.  So comparing organic visits to paid clicks may not quite be apples-to-apples, but it is at least apples-to-pears and pears are more like apples than oranges. (Visitors being oranges…you get the idea.)

Where the Rubber Meets the Road: Conversions

We all know – because Avinash has drilled it into us with his trinity approach 🙂 – that it is essential to move from clickstream data to outcomes. (And, to be fair, virtually all leading web analytics advocates promote a similar philosophy.) So the number of visitors is interesting, the number of visits may be more so, but we need to get to the real reason our site exists: conversions. In this case, purchases.  And to make decisions about optimization and resource allocation, we need to understand the efficiency of various channels bringing visits to our site and this means: conversion rate.  And to get a conversion rate that makes sense, we need to have the most appropriate denominator.

Which brings us back to visitors vs visits.  Yes, it can be useful to know what percentage of unique visitors in a month made a purchase, but wouldn’t it be more useful – in the case of this B2B ecommerce site selling repeat purchase products – to know the percentage of visits that resulted in a purchase?  For a lot of B2B sites, the purchase pattern may resemble that of a car dealership: long consideration phase involving multiple visits, probably multiple decision-markers, (hopefully) culminating in a purchase that will serve the buyer’s needs for a lengthy period. This particular client has a site that is more like an industrial grocery store.

So in terms of organic traffic, it is quite possible that the same visitor may return to this site several times during a month searching for different products (in fact, there could be several different searches during the same visit, so visits are not the same as searches, but probably close).  If we really want to understand how efficient our site is in converting organic traffic, we should be calculating conversion rate = orders / visits.

This also helps us compare organic search engine traffic with paid search traffic, where conversion rate = orders / clicks.

Conclusion

Focus on visitors or visits, as appropriate to site type and objectives, but do so consciously. Recommendation for this client: switch from a focus on organic visitors to organic visits.

It might even be worthwhile to consider tracking and analyzing conversions against visits for some keywords (‘repeat’ purchase) and against visitors for other keywords (larger, less frequent, or ‘blank-slate’ purchases).

For other sites, visitors – or a visitor segment – may be more relevant.  With reference particularly to non-ecommerce sites, Anil Batra has a great blog post on how to dive in and select the appropriate denominator for your conversion rate.