8 Steps to Begin Your Social Media Measurement Program

Social media measurement programs are custom and unique for each company depending on their business objectives. Clients often ask me when they begin a measurement project “Here are the brands we want to monitor, what KPI’s should we include in our reports?” There are an unlimited amount of KPI's that you can use when trying to measure your brand in social media. You could have a crack team of 20 marketing pros with access to 10 different vendors, have an unlimited budget, and you still could not report on everything in a digestible fashion. You need to go into your program with a plan and a purpose in order to produce valuable and repeatable results.

1. Objectives: Have a clear business objective before you step into any measurement project. Are you looking at the competitive landscape? Monitor opinions of a product launch? Increase Brand Awareness? Customer Engagement? or Tracking the success of a specific marketing campaign?
2. Identify Channels: Understand that all channels are not equal and should not be treated that way. Total volume for every mention of your brand over the internet usually does not give you a clear picture on what is happening. Create publication/website groups to improve the relevance in your volume based KPI's.
3. Value: Only use KPI's that add value. Sure your graph looked really impressive in the slide deck, but what happens when your boss asks where that data came from, or why is that figure important? Have these answers ready for any KPI you plan to use in a social media measurement project.
4. Repeatable Metrics: Make sure that you are able to repeat the production of your KPI's. If it takes you the better part of your week to produce one metric,  you should find a metric or collection of metrics that are easier to produce.
5. Change: Be open to change. The social media landscape changes every day, you should be ready to as well. I think you can stop writing to your customers on Friendster now.
6. Weighted Metrics: Apply weighted values to each metric. Every metric has a different meaning to each company. Don't get stuck using a methodology that was built for an organization in a completely separate marketplace. Use the thought process behind the "Forrester Wave" as an example. Apply a specific weight to each KPI to reach one single score that can be easily shared with your executive team.
7. Relevancy: When creating your goals and identifying channels you want to track, it is just as important to identify content you don't want included in your data. When using a social media measurement vendor, make sure they are filtering out spam, press wires, or any other unwanted content that could skew your data.
8. Manual Effort: No social media measurement vendor will be able to automatically produce every piece of data that you need to reach your business objectives. There will always be a manual aspect to a quality social media measurement program and you should designate specific resources for this.

Posted by Nathan Ray on June 23, 2010 at 11:17 AM | Permalink | Comments (1) | TrackBack

Hello (Social) World.

I could probably talk for weeks at a time about printing, publishing, and enterprise content management (ECM) as a former technology research analyst. In fact, that was largely my job until 3 weeks ago, when I first joined the Cymfony Insights team. Researching content management, social media, analytics, and automation technologies; I focused on improving companies’ internal and external communication processes - from print to digital to social. Turning my attention to Cymfony, I intend to take advantage of this technology and research background to deliver unique insights about the social media channel, where it’s headed, and how to extract as much market information and leverage from it as possible.

While we are only on the cusp of the social revolution (no Marxist commentary intended), social media monitoring has already catalyzed a vast methodology shift in market research. No longer limited to standard measurement approaches such as surveys and focus groups, burgeoning online content – both social and traditional media – enables a new kind of monitoring. Focused on mashing disparate data sources, dynamically analyzing them, and responding in real-time, perhaps “listening” is a more appropriate term for this methodology.

As we get comfortable in the third millennium, it’s worth considering how far we’ve come from the first blog post and how much farther social media will take us. Penetrating every aspect of our culture, social media has the potential to re-define the way we work and the way we live. In the meantime, the Insights team (and this new member) is dedicated to providing visionary and credible studies of this exciting new world! (Twitter: @omriduek)

Posted by Omri Duek on January 25, 2010 at 11:23 AM | Permalink | Comments (0) | TrackBack

Cymfony's Super Bowl Analysis is a Double Award Winner!

  Pardon me while I toot our horn and go blatantly hard sell for one post!

I'm pleased to announce that the Association for the Measurement and Evaluation of Communications recognized our Super Bowl Advertising Audience Impact Report with 2 awards!

First, we took the Gold in the category "Best Use of Measurement for a Single Event". The award judges cited our combination of traditional and social media as a "benchmark for the future" and praised it as "an intellectual piece of work" that "had real value for advertisers".

In fact, they liked it so much the judges also awarded it Grand Prix -- Innovation Award. In explaining why we were selected for this award too, the judges said it was a "ground-breaking analysis" and we earned "Top marks for the idea and the execution"

Thanks, AMEC! We are honored!

PS. If you are with a brand that is advertising on the Super Bowl, or an agency for one of those brands, we are offering this service again this year! Come to the Super Bowl page of the Cymfony site to learn more.

Posted by Jim Nail on November 25, 2008 at 05:08 PM | Permalink | Comments (0) | TrackBack

2008 Measurement Summit

The Institute for Public Relation’s annual Measurement Summit is a great venue to discuss both challenges and successes in PR research. However, each year conversation tends to dwell on the same topics i.e. examples of measurement initiatives without discussion of how to overcome the following challenges we all struggle with:

• How can corporate communications folks validate PR research thus increasing their research budget?
• How can agencies help their clients demonstrate the necessity and value of research throughout the campaign process?

Panel discussions focusing on these issues could stimulate ideas that would help us all.

Another frustration voiced at the summit is the deficiency in PR measurement expertise among practitioners with individuals noting the wealth of knowledge at the pre-conference workshops taught at the Summit. How about holding these workshops annually in markets such as Chicago and New York where considerable numbers of practitioners work? This year saw a smaller number of practitioners probably due to the economy but if IPR brought the workshops to them I bet attendance would be significant and research proficiencies could be disseminated.

Lastly, I was surprised there were no presentations at the summit focusing on social media. Although Jim Macnamara held a pre-conference workshop on measuring the impact of blogs, none of the Summit talks focused on the evolving universe of social media and how communications professionals are faring in this medium.

This in light of the results of the 2008 Cone Business in Social Media study indicating that 85% of Americans believe a company should interact with its consumers via social media.

Our counterparts in advertising are certainly diving into this medium. The lack of social media discussion at the Summit is concerning…

Posted by Natasha Stevens on October 20, 2008 at 12:20 PM | Permalink | Comments (1) | TrackBack

Announcing "Verismo" for PR Measurement!

PR measurement is a topic of hot debate and little agreement. Cymfony's new approach, Verismo, aims to tame the wealth of data spit out by a sophisticated measurement platform and give PR professionals a simple, clear way to explain the results of their work. Read on or click here to find out more.

A sophisticated content aggregation, analysis, and tagging platform like Cymfony's Orchestra system is a bit of a blessing and a curse to PR measurement professionals.

The blessing is that it generates rich data that were impossible with physical clips. The curse is that there is so much data, it can be a little overwhelming.

So in trying to tame this beast, Cymfony went back to basics and asked: what are we really trying to prove? There is a pretty simple answer to this because there are two key questions a PR professional must answer:

1. Did we reach the audience we wanted to?
2. Did we communicate the messages we wanted to?

The difficulty in PR measurement is in picking the right data to answer these questions. There is no simple answer and no general agreement across the industry.

But in looking at all the work we do for our clients, there were some common themes that became the foundation for Verismo:

• Visibility: When an article mentions a company or brand, how man people does it reach? And is the company or brand prominently discussed in the article?
• Reputation: What is being said about the company or brand? Is it the message the PR person is trying to communicate?
• Influence: Does this particular publication or journalist carry more credibility than average so that what he or she writes have higher impact on the reader?
• Sentiment: Is it positive or negative?

VRIS became our key variables. Then we developed a model to weight and calculate these variables into a score that represents the degree to which a communication objective is achieved.

VRIS Model, which became Verismo.

To learn more, read our new white paper. And to understand how Verismo fits into the musical theme of our company (Cymfony, Orchestra, etc.) read this Wikipedia entry ;-)

Posted by Jim Nail on October 14, 2008 at 06:05 PM | Permalink | Comments (0) | TrackBack

McKinsey Study: Change Organizations/Processes and Be Happy!

McKinsey's second annual "Building the Web 2.0 Enterprise" has lots of interesting data but what caught my eye was that companies which have changed some aspect of their organization/processes or created new positions for social media are more satisfied with their social media efforts than others. My take: this is a new example of the old "paving the cowpaths" story...

If you just try to plunk a new technology on top of old patterns of doing business, you will be disappointed. It is also an old technology story, too: remember ERP and CRM?

This year is a transitional year in business: before 2007, social media was viewed as a fad, a curiousity, or something only unemployed kids with time on their hands did. In 2007, we saw companies at least dabble and experiment. In the "Social Media in Business" study Cymfony did earlier this year, we saw that many companies were continuing these experiments and some beginning to build out and expand them. This is a healthy trend and a necessary progression. We are definitely seeing more companies create social media positions and budgets as they start to look toward 2009.

Social media in business has been an obsession of mine all year. In addition to our study, we also sponsored an Aberdeen study to find out. Check out our Social Media In Business page to download the reports and listen to the podcasts. If you think your company is moving too slowly, all this data can help you make the business case for the budget and staff to accelerate!

Join our Social Media in Business Facebook group to discuss what changes organizations are making and learn from your peers!

Posted by Jim Nail on August 11, 2008 at 12:04 PM | Permalink | Comments (0) | TrackBack

VISA 42% Share of Olympic Sponsor Discussion? NOT!

Our respected competitor, Collective Intellect, blogged that in their tracking of social media discussion of Olympic sponsors, VISA had a commanding 42% share. That looked fishy to us, so we did our own analysis and found VISA has a 16% share. To us, it looks like CI made 3 significant errors. The moral of the story: in social media analysis you must carefully define and execute the project or you get bad results.

Error #1: we suspected that CI defined the VISA brand incorrectly. It is tricky to get right because a simplistic definition will pick up references to a visa -- a government-issued travel documentation -- not just VISA the credit card. When we dumbed down our definition of the VISA brand we came up with a 51% share of voice, more in line with their results.

When we use Cymfony's more sophisticated methodology to limit content to the brand VISA, it is still one of the most-discussed sponsors, but now is essentially tied with Coca-Cola.

Error #2: This error becomes even more apparent when you evaluate sentiment for the brand. Our friends at CI stated in a Mediapost article about their results that the majority of posts including visa were negative. Our results show a slight positive skew and point to the incorrect definition of the VISA brand as the problem.

To dissect this effect, we used 3 definitions of visa: our sophisicated brand definition, the keyword "visa", and a definition of the travel document. Our keyword search on "visa" shows similar results to CI's: a negative skew. Using our brand VISA definition, discussion is 28% positive vs 20% negative. Isolating documents discussing visa the travel document shows that the skew is caused by the negative discussions surrounding the Chinese government's denial of visas to some athletes, reporters, and protesters.

Error #3: One other problem with their analysis is that they only tracked 7 of the 12 worldwide partners of the Olympics and two of the brands they did not track -- Samsung and Lenovo -- are both in the top 5 of social media discussion. Add all 5 other sponsors Collective Intellect didn't track and VISA's share  drops more -- and we also see that Samsung becomes one of the top 3:

The Mediapost article cited some other findings that are erroneous, eg, that Pansonic has no negative posts. In our analysis, 20% of posts about Panasonic were negative.

But maybe I'll save detail on that for another post....

Posted by Jim Nail on August 8, 2008 at 04:02 PM | Permalink | Comments (7) | TrackBack

An Example of Confused Terminology

A recent article published in the PRSA's new PR Journal provides a example of how poorly the terms "output", "outtake" and "outcome" are understood in the industry, even in this peer-reviewed publication which features articles from the leading academics and industry practitioners.

My recent post, "Outputs, and Outtakes, and Outcomes Oh My!" lamented how frequently I see these terms misused. In addition to the comments I got on the blog, I had email exchanges with several people. One member of the IPR's Measurement Commission said "I'm extremely puzzled how anyone could possibly misinterpret outputs (which only relate to mentions) with outcomes (which must relate to attitude or behavior reinforcement or change)."

I didn't have to look far for an example.

In this paper titled "The Application of "Best Practices" in Public Relations Measurement and Evaluation Systems" published in the first issue of the PR Journal, EchoResearch's President David Michaelson and CEO Sandra Macleod, noted as "two of the world's leading experts in communications research and measurement", ouputs and outcomes are used incorrectly if you take, as I do, the IPR's Guidelines for Meauring the Effectiveness of PR Programs and Activities authored by Prof. Walter Lindenmann as the standard definition. (Note: David Michaelson is on the Institute for PR's Commission on Public Relations Measurement & Evaluation under whose aegis the guidelines were published.)

Before I address the misuse of the term, let me say the paper is useful and, for PR practioners unfamiliar with conducting market research it should be a "must read" to understand the basics of constructing a valid research methodology. However, IMHO, it falls short of delivering the "best practices" by skimming over the tough issues of how to isolate PR's impact from other marketing/communications/sales activities and how to link PR to sales. But this is a topic for another post, perhaps.

Back to the example. On page 3 of the paper, Michaelson and Macleod write:

"Companies specializing in public relations measurement and evaluation have traditionally focused on evaluating only the outcomes of public relations. These outcomes are most commonly the media or press coverage that is a direct result of media relations activities (outputs)."

These two sentences say that outcomes are the number of clips or impressions while outputs are the number of press releases issued, perhaps the number of reporters called or who attended a press conference, etc.

Here are Lindenmann's definitions on page 7 of his paper:

"...PR outputs, which are usually short-term and surface (e.g., the amount of press coverage received or exposure of a particular message)..."

"...PR outcomes (e.g., did the program or activity change opinion and attitude levels, and possibly behavior patterns?)."

If the experts and members of the IPR Measurement Commission don't use the terms accurately, the rest of the industry will be confused, as I believe they are today.

Why do I rant about this? We all know words are important -- that's why we do what we do. We also know that words come with certain associations and connotations from everyday usage and I believe the terms "outputs" and "outcomes" are commonly used in ways that are dramatically different than what the IPR intends. (Outtakes is just a strange word that only exists, perhaps, in the world of film editing.)

Michaelson's and Macleod's misuse of outputs above is an example where common usage gets substituted for the official definition. If someone asked me what my output was today, I'd probably answer "a blog post, 35 emails, and 3 meetings" or something. When a CEO thinks of outputs, he probably thinks about how many widgets come off the assembly lines of his factory, not the number of his widgets on store shelves. Thus when a CEO (or other non-communications exec) hears the term outputs in association with communications activities, she probably thinks in terms of the activities of the PR staff as Michaelson and Macleod describe them, and not how many articles the company was mentioned in. And, as I have said before, I believe the only "outcomes" C-level staff are interested in are those that contribute financially to the firm's performance.

It is natural for people to apply their common understanding of terms when they hear them in a new context. It is a lot of work to try to wrap new meanings around them and have those new meanings widely accepted. The IPR and PRSA need to either step up efforts to educate the industry on the correct use of these terms or consider changing them to terms that are understandable based on their common usage.

I believe change is the preferred course. My nominations for new, more readily understandable terms are:

• Media Influence: clips, impressions, message pick up that indicate that PR activities and programs influenced the media to distribute the intended information
• Audience Influence: perceptual or attidudinal changes that indicate the intended audience saw the information, paid attention to it, and that the desired change took place
• Business Influence: sales, stock price, or brand equity valuation increases that are linked to these activities

What's your vote: change or stick with outputs/outtakes/outcomes?

Posted by Jim Nail on November 5, 2007 at 07:38 AM | Permalink | Comments (6) | TrackBack

Outputs and Outtakes and Outcomes -- Oh My!

In my experience, the terms outputs, outtakes and outcomes are not well understood among PR professionals. And if they are not well understood, building a PR measurement program will get derailed by using incorrect approaches and flawed reporting. Therefore, I propose an alternate framework of Media Influence, Audience Influence, and Business Influence.

Since the Institue for PR published the paper "Guidelines for Measuring the Effectiveness of PR Programs and Activities" in 2002, these terms have gained currency in the industry. Almost every presentation I see includes the terms "outcomes" and "outputs". But I continue to see what I consider to be misclassifications of  what I consider outputs and PR activities as outcomes.

Perhaps because the terms are somewhat mysterious. And while I like the alliteration and parallelism in the "out-" triad, it may be too confusing. So I'd like to propose an alternative framework:

• Media Influence -- this is essentially the outputs: clips, impressions, quality of coverage. In other words: did our PR activities cause the media to respond to our messages and report on them?
• Audience Influence -- this is essentially outtakes, though I differ a bit in Prof. Lindenmann's definition which is "determining if those to whom the activity was directed received, paid attention to, comprehended, and retained particular messages." I would go the next step to measure if these messages change audience perception, attitudes, and intentions, which the paper categorizes as "outcomes". Thus, this category of measurement answers the question: did our PR activities cause the intended audience to respond to our messages? The main approaches here are likely to be based on market research surveys.
• Business Influence -- These outcomes I define  very strictly as a financial impact on the business which can be measured in one of three ways: sales, stock price, or brand equity valuation. This category of metrics answers the question: did our PR activities positively influence the business? I use this strict definition because 1) we know that at the end of the day, this is all the C-suite really cares about and 2) it is the most complicated, expensive, and difficult metrics to create and link to PR activities. The main approaches are likely to be sophisticated modeling techniques, like market mix modeling.

While I don't want to get the industry hung up in changing terms for change's sake, I think the distinction is important. It will make it much clearer what measurements and methdologies address which aspect of measurement and clear away some of the fog surrounding the purpose and usefulness of different approaches.

(I also think we need to rethink the Audience Influence approach in light of the idea of "engagement", but I'll save that for another post.)

Thoughts?

Posted by Jim Nail on October 31, 2007 at 10:22 AM | Permalink | Comments (5) | TrackBack

Social Media Forecast: Looks a lot like online advertising 1994 - 2000

Mediaweek, citing an eMarketer report, states, "User-generated content on the Web is set to rapidly shift from a budding consumer trend to a serious business over the next five years." For those of us who have watched the adoption of online advertising, we are seeing history repeat itself.

This eMarketer report states that UCG sites will earn $1 billion in ad revenues this year, and $4.3 billion by 2011. Mediaweek goes on to state, "Plus, users have shown no indication that creating their own Web content for others to consume is a passing fad, found eMarketer. By 2011, the researcher estimates there will be 95 million Web users creating content online, up from 64 million last year."

The analogy to Web 1.0 online advertising is this: 1994 is usually cited as the birth of online advertising, and in 1997, the medium's fourth year, the IAB reported $900 million in ad revenue. Arguably, 2005 was the first year of social media, so 2007, the third year of its existence, will see $1 billion. Very similar growth curve. Online advertising hit $4 billion in 1999, its sixth year; UGC will hit this landmark in its 7th year. Since UGC growth isn't being propelled by a bubble that was starting to inflate in 1999, the curve again looks similar.

At the same time eMarketer also published an article "UGC not Critical for Many Marketers" in which only 12% of survey respondents said UGC was "very important" to their marketing efforts. The top reasons according to eMarketer "lack of clear ROI was often named as an objection. Other reasons listed included "management doesn't embrace it yet," "we simply haven't given it enough priority to consider it at this point" and "we've had difficulty in getting the establishment to understand it."

Nonetheless, over 40% are either using it already or considering it for next year. emarketer explains, "respondents understood the benefits of using UGC. A third of respondents said UGC was cost-efficient compared with traditional marketing and advertising, and 31% said it was useful because the credibility of traditional advertising and marketing was declining."

This is really deja vu for me: I wrote the following in an October 2001 Forrester Report titled "Online Advertising Retrenches":

"Marketers wrestle with opposing forces that alternatively urge them forward and hold them back from adopting Web site display advertising. They sense the Web's potential but recoil from the risks of being a pioneer and so wait for proven metrics and ad formats to appear."

In a nutshell, consumer media consumption habits are changing faster than marketers can learn and adapt. But year by year, from dipping a toe in the water, to pilot tests, to full adoption, the shift is on an inexorable course. The only question is whether the shift will be faster or slower than the prognosicators think.

I have an obvious bias, but my money is on it growing faster. Being a former prognosicator, here is how I would think about it:

1. There is no VC-fueled dotcom spending here, so the growth is unlikely to be interrupted for 3 years as happend to online advertising.
2. Senior marketing executives and industry observers like Bob Garfield have woken up to the reality that the age of mass marketing is in decline and they no longer need to be convinced to move beyond the tried-and-true tactics of the past. Traditional advertising spending is slowing and "below-the line" (I've always hated that phrase) on CRM, interactive, direct mail, etc. is growing.
3. Social media represents a bigger change than online advertising was. Banners were another format to push messages through; blogs and social networks require a much more dynamic relationship with consumers.

IMHO, the biggest drag on the growth of online ads in the '90's was lack of urgency because marketers still had total confidence in TV and mass marketing. Without that comfort, marketers will be much more willing to invest in the necessary learning to add new social media tools to their marketing arsenal.

Posted by Jim Nail on July 5, 2007 at 02:39 PM | Permalink | Comments (2) | TrackBack

Sitemap | Privacy Policy | Terms & Conditions

© 2008 TNS Cymfony. All Rights Reserved.