by Eric Hill, Co-founder & President, MyMediaInfo
When did a failing grade of 60% become a reason for celebration? And better yet, why do companies that invest millions of dollars into Lean Efficiency Methodologies or Six Sigma even look at, let alone purchase, social media monitoring solutions that produce results with error rates that would be a failing grade for any student?
A few weeks ago, the nice folks at Influence People brought their Monitoring Social Media road show to Boston. The conference was jam-packed with industry leaders discussing how to measure social media, why to measure it and if it should be measured at all. I went with the intention of getting one question answered: Why would a company use automated sentiment analysis (that is, having a computer read a story and determine whether it is positive, negative or neutral) knowing that is inherently flawed?
In an article on www.destinationcrm.com, Forrester Principal Analyst Suresh Vittal states, “Fundamentally, the reality of the space is, you have a market where automated sentiment is a good first step but you need to go beyond the step in focusing on the accuracy.” The article also quotes Vittal saying, “In talking to clients who have deployed some form of [automated] sentiment analysis, accuracy rests at about 50 percent. “
I was pleasantly surprised to hear the majority of the conference speakers agree that technology is only a starting point when it comes to monitoring and measuring social and traditional media. They believe a blended approach is best; leveraging technology to help gather, sort and organize large amounts of information so that it can then be coded appropriately by a human analyst. They concluded that powerful social media monitoring tools like Radian6 are best used in tandem with a human reading, coding and organizing the information to fit your organization.
Having analyzed results scored by automated sentiment analysis, I’ll tell you the results can be shocking. I’ve seen tweets slamming a company rated as positive, and to the other extreme, blog posts praising the same company while slamming its competitors rated as negative for the company. Bottom Line: If you’re currently using or plan to use an automated tool, be careful of the results. Without the attention of a dedicated analyst, you may be reporting and basing your social media strategy on flawed results.
As for my question, I never did get an answer. But at least I know that the industry experts agree with me in that there are flaws in relying solely on automated sentiment analysis. Now that’s a reason to go to Chuck E. Cheese’s.
