I should probably offer Mark Schaefer some sort of kickback since his blog never fails to stimulate new ideas. A couple weeks ago, he wrote a post on measurement in social media. Now you’re probably thinking, “Yet another blogpost talking about measurement? Why in the world is that so special?” It wasn’t just the content that was special…the post sparked some interesting comments around the necessity of measurement and types of measurement to consider for social media.
When most folks talk about metrics and ROI and all the various forms of measurement, they’re usually referring to a quantitative methodology. You know…like measuring number of Twitter retweets, Facebook fans, online WOM mentions, blogpost traffic, generated sales, etc. These are things that can be counted and evaluated fairly easily so long as you know why you’re doing it in the first place. Just measuring for the sake of appearances really isn’t going to help you or your organization get where you want to go. Which leads us to…
Why measure at all?
I’m not going to go into this too deeply because there are so many super-smart folks who’ve already made a compelling case for measuring online activity. I will merely add that measurement is a form of feedback, which is critical to learning what works and what needs to be improved. How will you know if your latest online customer engagement program is succeeding in meeting its objectives (you did establish objectives, right?) if you can’t measure the results.
Why add qualitative?
Because sometimes your quantitative data lies to you. Not deliberately, of course, but all those quantitative metrics you’re racking up may not be telling you the full story. This is particularly true in the area of social media where we’re trying to gauge not only action but more emotionally-charged and nebulous qualities like sentiment and beliefs. For instance, when a fan says they “love” their iPhone, what does that mean? Or when someone else tweets that your company’s sales efforts are old and they suck, what’s happening here? A strictly quantitative measurement approach likely will not dive deep enough here to give you tangible results you can use to connect with your customers and make necessary adjustments.
What kind of qualitative measurement methods can you use? The major knock against qualitative is the perception that it’s time-intensive, which can be true. But you have to weigh that through a cost/benefit analysis: is what I’m learning here worth the investment of resources? Still unsure? Then take a page from the work of social scientists and build a sample. Dont’ try to eat the elephant all at once. Your purpose here is to build bite-sized understanding. The key is to construct a random, representative sample that’s going to give you intelligible feedback on the sentiment of your customers (the whole topic of how to build good, measurable samples for social media probably should get a blogpost of its own).
Interviews: These don’t have to be long. Your objective here is to go deeper than a standard quantitative survey by uncovering the more subtle meanings of what “love” and “suck” mean for your customers.
Observations: The simple truth about us human beings is that we often say one thing only to turn around and do something rather different. There are plenty of reasons for this, but figuring out ways to observe our participants is a good way to get closer to actual action that drives behavior.
Do you still need quantitative?
YES! There’s no either/or proposition here…the best measurements will combine both quantitative and qualitative methods. Once we have a working hypothesis (we have to know why we’re doing this in the first place), it’s a recursive process where we use qualitative research to figure out what questions we need to ask, construct quantitative research to gather data, then another qualitative round to complement our data by delivering further depth of insight.
Okay, so it’s a rather high certainty you don’t have time to do recursive research, but the point here is that it’s important to not overuse quantitative measures. How can you best incorporate qualitative methods into your own plans? Or if you’ve used particular qualitative tactics, how well did they work for you?