Measuring Impact: Part One – Why Activity is Not Enough

You can sign up to our LinkedIn newsletter here.

Impact, outcomes, activity, SMART goals, KPIs, and OKRs—this smorgasbord of management jargon too often belies the reality that activity is not enough.

How do you ensure that activity delivers the impact you aspire to?

There is nothing wrong with measuring activity. However, unless you have thought through how the activity will contribute to outcomes and how those outcomes support the desired impact, you risk being a busy fool.

The Wrong Goal can make for Failure

As I write this in December 2024, the governing Labour Party in the UK is seeking to clarify how their ‘Five Missions’ can be experienced as improved outcomes by the UK electorate. They believe that improved outcomes will have the impact of ensuring they are re-elected for a second term. Commentators explain that, for example, being the fastest growing economy as measured by GDP in the G7 group of nations by the end of their five-year term may be a SMART goal. However, the recent Biden/Harris experience has shown that an increase in GDP in the USA failed to connect to the lived experience of their electorate as an improvement in discretionary household income. They didn’t feel richer, and the ‘cost of living crisis’ continued to be experienced unabated. The Democrats have lost power.

Measuring the Wrong Activities in the Wrong Way

The second issue is that the way the activity is measured is often insufficient to understand the implications.

For example, imagine you’re running a team where you want to improve engagement among team members. You might decide to measure the number of times team members talk to one another, believing that more interaction equals more engagement. However, this alone does not provide evidence that engagement is increasing. You also need to measure how engagement changes due to those interactions. If there is a clear linkage, the activity contributes to team engagement.

But is team engagement the ultimate impact you’re aiming for? Or is there something bigger going on? Such as reducing staff absence or improving employee retention? If the goal is to increase retention, it’s essential to measure and understand that directly. Increased engagement might reduce turnover, but it may not. What is the hierarchy of causality: does A cause B, resulting in C? If, in fact, C causes B and A, you have misunderstood the hierarchy of causality. For example, if a few key staff members leave, in many contexts, it impacts morale and gives others ‘permission’ to re-evaluate whether they want to stay. In this case, an increase in staff turnover can result in a further increase in staff turnover. Many of the charity CEOs I worked with post-COVID found a catastrophic loss of experienced team members in a follow-the-leader doom loop. The increased pressure on the remaining staff threatened to result in further departures. Significant intervention by the executive team was necessary to recover the situation.

Are KPIs always Key Performance Indicators?

In the past, many organisations relied on key performance indicators (KPIs), as if measuring a relevant activity would indicate performance. However, these indicators were often unrelated to outcomes or impact and merely measured activity. The further issue was that many people would ‘game’ the indicators. It was possible to ensure you got the necessary score without changing the behaviours to achieve the desired corporate outcome.

It is essential to be strategic about what you measure and why. Whilst activity is easy to measure, other methods, such as self-assessments, interviews, storytelling, or case studies, provide richer insights into impact. An example of this is the idea of a 360-degree staff feedback process. Many organisations have adopted this in a measurement-focused way; they typically will survey all the relevant people for the person’s 360-degree. They will have a set of questions asking you to rate on a scale from very true to very untrue. The report has numerical plausibility, although each individual respondent may interpret the scale differently. More importantly, the data is often insufficient for the individual to understand how to act. When I did 360-degree feedback exercises for more senior people, where there was the opportunity to spend time and money, I conducted short interviews with the people the subject reported to, the people who reported to them, and their peers. I captured the conversations so that I could quote back verbatim comments to the subject. I could show them examples of the impact of specific behaviours on others , which was much more actionable.

The Quantitative Trap

Don’t dismiss things that are not numerically measurable. Qualitative insights from conversations and lived experiences can be as valuable as quantitative measures in understanding impact. Personal truths and anecdotal evidence provide unique perspectives on the actual effects of activities and interventions.

Conclusions

If you truly care about impact, then recognise that activity metrics are not enough. It’s essential to link activities to outcomes and seek deeper understanding through both quantitative and qualitative methods.