There is a big push, correctly, for arts organizations to employ data driven decision-making processes. However, not all data is of equal value. Colleen Dilenschneider recently made a good post separating data into diagnostic metrics, key performance indicators and vanity metrics.
The most valuable data for an organization is key performance indicators (KPIs). Most data points are diagnostic metrics which, while valuable, support KPIs rather than measure progress toward organizational goals. Vanity metrics make you look impressive, especially in front of your governing board, but are not really useful in measuring the success of the organization.
Dilenschneider defines key performance indicators as:
…measurements that show progress toward business- or mission-based goals. They provide critical checkpoints on the route to reaching desired objectives.
Examples of perhaps the most obvious KPIs for cultural organizations include attendance counts, annual giving totals, and how an organization is performing compared to market potential…
Other examples of KPIs include reputation, educational value, satisfaction, value-for-price perceptions, admiration quotient, and “best in class” perceptions…These metrics have the highest direct correlation with success in achieving desired behaviors such as a visit, membership renewal, or mission-based action.
On first glance, diagnostic metrics may seem indistinguishable from KPIs, but upon closer examination, it becomes clearer that they are subsidiary measurements that inform KPIs. Dilenschneider uses the example of having good blood pressure doesn’t mean you are healthy, nor does having elevated blood pressure mean you will have a heart attack tomorrow. Blood pressure is merely one measure in the entire system of physical health.
“diagnostic metrics are data points that contribute to reaching KPIs and aid organizations in pinpointing opportunities. However, these metrics cannot stand in for KPIs because they are not as closely related to desired outcomes.
These include metrics like safety perceptions of the surrounding area, length of stay, social media likes and shares, and the number of people who interact with an exhibit, for instance.”
Vanity metrics are data that look good, but really have no relationship to the performance of the organization, nor can they help inform decision making. Dilenschneider points out that the digital age provides us with endless opportunities to generate vanity metrics.
I have seen funders ask for some of these data points which only contributes to the sense they are meaningful. For example, visits to your website or social media accounts may be a better measure of your social media staff’s ability to repost memes than legitimate engagement with your organization.
These can include social media follower counts, page views to your website, and the number of email subscribers, for instance. This doesn’t mean that having big numbers in these areas is bad! But having big numbers on these also doesn’t mean an organization is necessarily reaching its goals.
These new followers are not necessarily a “captive” audience, as some leaders who are less familiar with social media might think. If your organization cannot keep its new followers engaged, they’re unlikely to see future posts. Instead, a goal of social media is to cultivate a group of people who will actively engage with your organization and its mission, and carry out desired behaviors.
Dilenschneider warns that the problem with taking vanity metrics seriously is that it can result in directing resources away from taking productive action in relation to the true key performance indicators of the organization.