« Our summer vacation is over.... | Main | McKinsey Study: Change Organizations/Processes and Be Happy! »

VISA 42% Share of Olympic Sponsor Discussion? NOT!

Our respected competitor, Collective Intellect, blogged that in their tracking of social media discussion of Olympic sponsors, VISA had a commanding 42% share. That looked fishy to us, so we did our own analysis and found VISA has a 16% share. To us, it looks like CI made 3 significant errors. The moral of the story: in social media analysis you must carefully define and execute the project or you get bad results.

Error #1: we suspected that CI defined the VISA brand incorrectly. It is tricky to get right because a simplistic definition will pick up references to a visa -- a government-issued travel documentation -- not just VISA the credit card. When we dumbed down our definition of the VISA brand we came up with a 51% share of voice, more in line with their results.

Olympic_visa_keyword_7

When we use Cymfony's more sophisticated methodology to limit content to the brand VISA, it is still one of the most-discussed sponsors, but now is essentially tied with Coca-Cola.

Olympic_visa_brand_6

Error #2: This error becomes even more apparent when you evaluate sentiment for the brand. Our friends at CI stated in a Mediapost article about their results that the majority of posts including visa were negative. Our results show a slight positive skew and point to the incorrect definition of the VISA brand as the problem.

To dissect this effect, we used 3 definitions of visa: our sophisicated brand definition, the keyword "visa", and a definition of the travel document. Our keyword search on "visa" shows similar results to CI's: a negative skew. Using our brand VISA definition, discussion is 28% positive vs 20% negative. Isolating documents discussing visa the travel document shows that the skew is caused by the negative discussions surrounding the Chinese government's denial of visas to some athletes, reporters, and protesters.

Olympic_visa_tonality

Error #3: One other problem with their analysis is that they only tracked 7 of the 12 worldwide partners of the Olympics and two of the brands they did not track -- Samsung and Lenovo -- are both in the top 5 of social media discussion. Add all 5 other sponsors Collective Intellect didn't track and VISA's share  drops more -- and we also see that Samsung becomes one of the top 3:

Olympic_sov

The Mediapost article cited some other findings that are erroneous, eg, that Pansonic has no negative posts. In our analysis, 20% of posts about Panasonic were negative.

But maybe I'll save detail on that for another post....

Posted by Jim Nail on August 8, 2008 at 04:02 PM | Email this post Permalink
Technorati Tags: , , ,

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8341cdcdc53ef00e553d6cd858833

Listed below are links to weblogs that reference VISA 42% Share of Olympic Sponsor Discussion? NOT!:

» mobile phones with free gifts from mobile phones with free gifts
Did you know you can get a valuable free gift with your next contract moble phone... such as Wii, HDTV,top spec Laptops, Playstation, Tom Tom and much more? And its all from a name you can trust... Tesco! YOu'd be nuts to go direct to network with this... [Read More]

Tracked on Sep 8, 2009 3:17:32 PM

Comments

Anna,

We use a technique called natural language processing which uses a much more sophisticated and complex definition of a brand than just a keyword. By looking at the context around the occurrence of the brand the technology determines whether it is really talking about the target brand (in this case the Visa credit card) vs a related but irrelevant concept (in this case a visa, meaning a government travel document).

Posted by: Jim Nail | Aug 11, 2008 4:33:11 PM

That's interesting anaysis. But, Jim, what do you mean by saying you are not using keyword search?
The algorithm you mentioned helps refining it, but the search still based on keyword, right?

Posted by: Anna Rokina | Aug 11, 2008 12:07:07 PM

So, Tim, what you are saying is that if Visa were your client, you would have sent then into a panic for a couple of days about this high level of negative posts. Then, when the "adaptive algorithm" figured out what was really going on, you would have to call them up and say, "Oops, never mind!"

Your updated numbers look better -- but you should post a correction to your original post. Add in the rest of the sponsors and then let's compare numbers again....

BTW, we are not using keyword search -- never have. Way too blunt an instrument and delivers results that look remarkably similar to your first post :-)

We spend time up front so that our algorithms start off with a tighter definition and our client don't have as much angst.

Posted by: Jim Nail | Aug 11, 2008 11:03:38 AM

While your keyword analysis is interesting, it's not adaptable enough to use in the rapidly changing environment of social media. We use a fairly rich combination of artificial intelligence and text analytics to pre-process our data set. The algorithm initially did include some of the government visa posts but it learned quickly, catching the semantic shift in the data. You're right though, we should have included more of the sponsors in the original post. There is a follow-up post with more detailed analysis of the overall landscape, as well as some thematic segmentation of the top negative sentiment. Look for a future blogpost from us on http://blog.collectiveintellect.com that goes into this in much more detail.

Tim Wolters
CTO
Collective Intellect

Posted by: Tim Wolters | Aug 9, 2008 6:19:48 PM

Vicious but nice point Jim. Maybe this Social media thing isn't so easy after all :)

Posted by: David Rabjohns, MotiveQuest | Aug 9, 2008 2:49:21 PM

Good point and you're right. I included UPS because CI mentioned them as well, and their conclusion about them was wrong too. I guess that is another good point for my next post.

Posted by: Jim Nail | Aug 8, 2008 6:03:10 PM

You seem to have 13 sponsors in your list of 12. UPS isn't a global sponsor of the Olympics; it's a local sponsor.

Posted by: Glenn Fannick | Aug 8, 2008 5:32:15 PM

The comments to this entry are closed.