The most-heard sentence from Managers: we must measure everything. However, this is far from being true. Not everything must be measured.
Lets imagine you own a business. An e-business. Maybe successful, maybe not (yet). You are in the moment in which you consider implementing a tracking tool (Google Analytics, Omniture, MixPanel, etc.). Then the fateful sentence comes: we must measure everything. Please track every single click, every single action, every field entered in a form, etc. EVERYTHING! Usually when a Manager is asked why everything should be measured he answers: because everything can be optimized with data.
At this point two sentences crosses the Manager’s mind:
“In you can’t measure it, you can’t improve it”.
“If you didn’t measure it, it didn’t happen”.
The first sentence is absolutely true in almost all situations. The second one would need some elaboration. Indeed, is true: if you didn’t measure it, it didn’t happen. My question, as an analyst, is: “Yes, we did not measure it. Yes, indeed, it did not happen. So?“. I will state it very clearly here and now: it’s not mandatory to track everything. Why? Simply because you can’t optimize everything at the same time, or simply because the benefit of optimizing some features is insignificant.
When we measure everything we overcomplicate the implementation. It becomes a pain, it becomes never-ending. The analyst is always thinking what to measure instead of actionating the data that arises from the current implementation. The tool’s interface turns into a nightmare: sampling, unclean data, impractical volumes of data to process, etc. The answer to this is: keep it simple.
How do we keep simple a tracking tool implementation? The key is to design Measurement Plans. Avinash Kaushik expresses this in a majestic way. A Measurement Plan consists of five steps:
– Goals: identify the business objectives (sell more, get more leads, increase CLV, decrease returns, improve margins, etc.). According to Kaushik’s framework, the Goals must be Doable, Understandable, Manageable, and Beneficial).
– Strategies: for each objective identify crisp goals. They must be specific in the sense that they will be used to accomplish the goals (increase repurchase ratio, increase new users, decrease budget for some marketing campaigns, etc.)
– KPIs: no need to say what a Key Performance Indicator is. Of course, there are metrics that will tell you how are we doing with respect the established strategies.
– Targets: not mandatory but very useful. They are used to establish an end-point for our KPIs.
– Segments: The most important part of the plan. We take segments of users or behaviors that we’ll analyze in order to understand where the fail or the success is (new users, paid campaign users, mobile users, users that land in the homepage, etc.). This is the hardest point within the Measurement Plan, and this is really where actionability arises.
The measurement plan is devoted to actionability. Whatever it brings no action is not a valid Goal or Strategy. A KPI or a segment that is not devoted to fulfil a strategy is not useful. At the end is very simple. If your Measurement Plans does not consider that a given button should be tracked, don’t track it! If your Measurement Plan does not consider that the fields of a form should be tracked, don’t track them! If your Measurement Plan does not consider that scrolling should be tracked, don’t track it! In this way you will have a clean tracking tool interface full of data that can be transformed into actionable information.
Ideally the Measurement Plans are build by the team of analysts. They (should) know the business and they (should) know the technology. They are able to talk to the Business stakeholders and talk to the developers. They can gather business requirements across the company, think about business opportunities, and transform them into technical specifications. They are also able to gather all the data, build the KPIs, compare them to the targets, and recommend actions. In other words, they must take ownership of the Measurement Plans, from its conceptualization to its implementation.
At this point, I want to recall Kaushik’s “Three Layers of So What” Test. It’s a very simple test that will help you to decide whether a KPI (or a metric) is useful or not. Against every metric you want to report ask the question “So What?” three times. Each question provides an answer that will raise another question. If at the third time you don’t get a clear action that must be taken, then you just face a nice-to-have metric and not an actionable one. Non-actionable metrics keep the focus off what is really important. Non-actionable metrics and, hence, non-actionable tracking, is like having Diogenes syndrome for data: you collect, collect, and collect data without extracting any useful information from it.
Last but not least, don’t fell guilty for leaving features of your site without tracking. Feel proud for the recommendations to actions taken from the tracked items. And, again, keep it simple.
