Tag Archives | metrics

Adding Qualitative to Your Social Media Measurement Mix

I should probably offer Mark Schaefer some sort of kickback since his blog never fails to stimulate new ideas. A couple weeks ago, he wrote a post on measurement in social media. Now you’re probably thinking, “Yet another blogpost talking about measurement? Why in the world is that so special?” It wasn’t just the content that was special…the post sparked some interesting comments around the necessity of measurement and types of measurement to consider for social media.

When most folks talk about metrics and ROI and all the various forms of measurement, they’re usually referring to a quantitative methodology. You know…like measuring number of Twitter retweets, Facebook fans, online WOM mentions, blogpost traffic, generated sales, etc. These are things that can be counted and evaluated fairly easily so long as you know why you’re doing it in the first place. Just measuring for the sake of appearances really isn’t going to help you or your organization get where you want to go. Which leads us to…

Why measure at all?
I’m not going to go into this too deeply because there are so many super-smart folks who’ve already made a compelling case for measuring online activity. I will merely add that measurement is a form of feedback, which is critical to learning what works and what needs to be improved. How will you know if your latest online customer engagement program is succeeding in meeting its objectives (you did establish objectives, right?) if you can’t measure the results.

Why add qualitative?
Because sometimes your quantitative data lies to you. Not deliberately, of course, but all those quantitative metrics you’re racking up may not be telling you the full story. This is particularly true in the area of social media where we’re trying to gauge not only action but more emotionally-charged and nebulous qualities like sentiment and beliefs. For instance, when a fan says they “love” their iPhone, what does that mean? Or when someone else tweets that your company’s sales efforts are old and they suck, what’s happening here? A strictly quantitative measurement approach likely will not dive deep enough here to give you tangible results you can use to connect with your customers and make necessary adjustments.

What kind of qualitative measurement methods can you use? The major knock against qualitative is the perception that it’s time-intensive, which can be true. But you have to weigh that through a cost/benefit analysis: is what I’m learning here worth the investment of resources? Still unsure? Then take a page from the work of social scientists and build a sample. Dont’ try to eat the elephant all at once. Your purpose here is to build bite-sized understanding. The key is to construct a random, representative sample that’s going to give you intelligible feedback on the sentiment of your customers (the whole topic of how to build good, measurable samples for social media probably should get a blogpost of its own).

Interviews: These don’t have to be long. Your objective here is to go deeper than a standard quantitative survey by uncovering the more subtle meanings of what “love” and “suck” mean for your customers.
Observations: The simple truth about us human beings is that we often say one thing only to turn around and do something rather different. There are plenty of reasons for this, but figuring out ways to observe our participants is a good way to get closer to actual action that drives behavior.

Do you still need quantitative?
YES! There’s no either/or proposition here…the best measurements will combine both quantitative and qualitative methods. Once we have a working hypothesis (we have to know why we’re doing this in the first place), it’s a recursive process where we use qualitative research to figure out what questions we need to ask, construct quantitative research to gather data, then another qualitative round to complement our data by delivering further depth of insight.

Okay, so it’s a rather high certainty you don’t have time to do recursive research, but the point here is that it’s important to not overuse quantitative measures. How can you best incorporate qualitative methods into your own plans? Or if you’ve used particular qualitative tactics, how well did they work for you?

photo credit: hutchscout (via Flickr)


At Connection Cafe: Is Your Data Collection Unbalanced?

For the Connection Cafe blog this month, I wrote about the need to use a balanced qualitative and quantitative approach to learning about constituents. Here’s a teaser of my latest post…the full post is at the Connection Cafe…

Mixed in with the work that I do at Convio, I’m also pursuing a Master’s degree in business anthropology. If you’re like most folks, you may be wondering what that is exactly. This field is somewhat new even though anthropology as a social science has been around for long time. Basically, business anthropologists work with organizations to help them understand things like staff culture, customer relationships, and product design. That’s fairly broad but at it’s core, we study people and their patterns of behavior. What I most love about it is that we are trained to help non-profits and businesses understand the deeper meaning of what seemingly appears ordinary and everyday…then take what works and amplify it.

For an example, let’s apply a business anthropology approach to a common issue among non-profits: how to better engage constituents. Hopefully you have plenty of metrics showing your email open-rates, donor conversion rates, website flowthrough rates, etc. You may also have survey results and graphical analysis. (And if you haven’t recently done this type of quantitative data collection, no worries…hopefully this post will reinvigorate you.)

Now take it one step further. Most businesses and non-profits commit to collecting quantitative data but usually neglect the qualitative data. The reason for this often rests with some common misperceptions that collecting and analyzing qualitative data is difficult, unmeasurable, and overly time-consuming. But, the fact is that every organization that is committed to developing better relationships with its constituents needs to employ a balanced data collection plan. Strict number crunching usually fails to get at the heart of the things that matter most to non-profit organizations which are people and their emotional connection to your cause. It all comes back to understanding the deeper meaning of things which numbers can only hint at.

In addition to your quantitative measurements, what types of qualitative data collection techniques should you consider? It depends largely on what you’re trying to learn. Start with the big question you want to try to answer. Here are two familiar scenarios:

1. If you host events like walks, pet adoptions, or volunteer pledge drives and want to know why individuals are giving their time (always a highly prized commodity) to your organization, consider a participant-observation program. You’ll be actively participating alongside your constituents, learning about their passions and why they believe your cause matters. Your aim is to see your organization’s relationship through the eyes of others and find the commonalities that they share.

2. If you want to know what exactly will help convert individuals from one-time donors to recurring donors (an even more prized commodity in these economic times!), consider an interview program. This is not just a survey in a different form…think of it as a semi-structured conversation guided by your big question. You’re trying to dive deeper into understanding the major themes of the relationship between your constituents and your organization.

One significant caveat to note here…these qualitative approaches are only effective when performed with a curious objectivity. If you think you already know the answers to your questions, you might want to consider employing another impartial staff member to do them or hire a consultant (a business anthropologist, perhaps?).

This is just a thin, surface-level slice of what a balanced quantitative and qualitative approach can deliver to your organization. My hope is that it sparks some dialogue inside your organization about how to best discover significant patterns and meanings within your constituency; then use this knowledge to improve the effectiveness of your actions. If you’re interested in learning more about the field of business anthropology shoot me an email at, leave a comment below, or follow the business anthropology tag on my own blog.