Adding Qualitative to Your Social Media Measurement Mix
03.07.2010 | Chris Bailey
I should probably offer Mark Schaefer some sort of kickback since his blog never fails to stimulate new ideas. A couple weeks ago, he wrote a post on measurement in social media. Now you’re probably thinking, “Yet another blogpost talking about measurement? Why in the world is that so special?” It wasn’t just the content that was special…the post sparked some interesting comments around the necessity of measurement and types of measurement to consider for social media.
When most folks talk about metrics and ROI and all the various forms of measurement, they’re usually referring to a quantitative methodology. You know…like measuring number of Twitter retweets, Facebook fans, online WOM mentions, blogpost traffic, generated sales, etc. These are things that can be counted and evaluated fairly easily so long as you know why you’re doing it in the first place. Just measuring for the sake of appearances really isn’t going to help you or your organization get where you want to go. Which leads us to…
Why measure at all?
I’m not going to go into this too deeply because there are so many super-smart folks who’ve already made a compelling case for measuring online activity. I will merely add that measurement is a form of feedback, which is critical to learning what works and what needs to be improved. How will you know if your latest online customer engagement program is succeeding in meeting its objectives (you did establish objectives, right?) if you can’t measure the results.
Why add qualitative?
Because sometimes your quantitative data lies to you. Not deliberately, of course, but all those quantitative metrics you’re racking up may not be telling you the full story. This is particularly true in the area of social media where we’re trying to gauge not only action but more emotionally-charged and nebulous qualities like sentiment and beliefs. For instance, when a fan says they “love” their iPhone, what does that mean? Or when someone else tweets that your company’s sales efforts are old and they suck, what’s happening here? A strictly quantitative measurement approach likely will not dive deep enough here to give you tangible results you can use to connect with your customers and make necessary adjustments.
What kind of qualitative measurement methods can you use? The major knock against qualitative is the perception that it’s time-intensive, which can be true. But you have to weigh that through a cost/benefit analysis: is what I’m learning here worth the investment of resources? Still unsure? Then take a page from the work of social scientists and build a sample. Dont’ try to eat the elephant all at once. Your purpose here is to build bite-sized understanding. The key is to construct a random, representative sample that’s going to give you intelligible feedback on the sentiment of your customers (the whole topic of how to build good, measurable samples for social media probably should get a blogpost of its own).
Interviews: These don’t have to be long. Your objective here is to go deeper than a standard quantitative survey by uncovering the more subtle meanings of what “love” and “suck” mean for your customers.
Observations: The simple truth about us human beings is that we often say one thing only to turn around and do something rather different. There are plenty of reasons for this, but figuring out ways to observe our participants is a good way to get closer to actual action that drives behavior.
Do you still need quantitative?
YES! There’s no either/or proposition here…the best measurements will combine both quantitative and qualitative methods. Once we have a working hypothesis (we have to know why we’re doing this in the first place), it’s a recursive process where we use qualitative research to figure out what questions we need to ask, construct quantitative research to gather data, then another qualitative round to complement our data by delivering further depth of insight.
Okay, so it’s a rather high certainty you don’t have time to do recursive research, but the point here is that it’s important to not overuse quantitative measures. How can you best incorporate qualitative methods into your own plans? Or if you’ve used particular qualitative tactics, how well did they work for you?

Once upon a time I was a hiring manager, and perhaps if fate has its way again, I’ll be in a position with this type of responsibility again soon. But for now, I’m on the other side of the desk. After reviewing some recent hires by prominent organizations, a rather interesting pattern emerged: how similar the hires are to each other…and to the hiring manager.
I’m always fascinated with the day-after fallout of the Super Bowl adfest. There are plenty of people doing their Monday morning armchair quarterbacking thing, lamenting how terrible the commercials were and how much they continue to suck year after year. It’s at this point I try to take my branding hat off and recognize something I think is rather important. The commercials were not made for us. They were made for the 95% of everyone else who wants to be entertained. They were made for people like my dad who could give two craps if there was an overabundance of slapstick violence and dudes trying to pick up chicks (Love you, Dad!). The only metric here is whether the ads were amusing and some of them were very amusing and entertaining, indeed.
On Friday, I had the honor of speaking on Chris Bailey’s Entrepreneur Panel at the sold-out
I’m not the best one to talk about to-do lists, Getting Things Done, or the other various thinkings about task management. All too often, I still find myself just keeping those various things that need to be completed inside my head. I do have a Pro account with Remember the Milk so I can sync my tasks with my BlackBerry but it doesn’t take long for me to neglect the lists. Perhaps its a lack of discipline, but that’s not to say that I’m not productive…okay, maybe I’m not as productive as I could be.
A few weeks ago, Mack Collier asked the question of 








