Yammer & Klout: An intranet game too far
Earlier today, Yammer and Klout announced details of a new strategic partnership that will see Klout scores added to Yammer profiles. On the face of it, these are two complementary technologies — one providing prescient social networking functionality and the other giving an instant read-out of internal influence. But is it really a perfect marriage?
Have you got Klout?
If you’re a regular user of social media, you’re probably fully aware of Klout. Using a number of data points, Klout processes your social media output and re-use and gives users a score of 1-100, with the higher marks reflecting higher social media influence. The service started with Twitter and has since added Facebook, Instagram, LinkedIn, G+ and a few more into the calculation mix.
Whilst we have no details directly from Yammer, it would seem likely that the data points Klout uses for Twitter and Facebook would be similar to those it will employ on Yammer Klout. Key to the success of such a partnership then, is the value of these data points inside the enterprise.
Get scoring
Klout don’t say precisely how their scoring is actually calculated — no surprise there — but they do give plenty of hints on the key data points.
- Follower Count : A measure of the number of people who are following you. The inference is clear: the more people who follow you, the more people there are who find your content interesting and the more chance your content will be shared. But does this hold true internally on your enterprise social network? Would follower count be correlated with seniority — everyone following the CEO, no one following Susan from accounts? But then maybe your CEO is the most influential? The value of this data point is debatable.
- Retweets / Shares : A measure of how many times your original content is shared by others. Klout also state that this is not a linear calculation in that generating 100 shares from 10 posts would generate a higher score than 100 shares from a thousand original posts. So this again is a measure of interest, of content stickiness and therefore, possibly, of how influential you are in your communities. Internally, this could correlate with seniority, but may be a reflection of someone with an interesting (not influential) job or demonstrate someone who has the propensity to gossip!
- List Memberships : A measure of how many people have listed you. This, in our view, is a problematic measure as you could be listed for both positive and negative reasons. Being added to a list entitled “Time wasters” is rather different to the list on “Top Content”.
- Live v Dead account : A measure of how many of your followers are real. Hopefully, this is not an issue internally as most, (but seemingly not all), of your colleagues will be real, but it would be interesting to know how many are active. You may have lots of followers, but if they’re not ever likely to re-share your content, then they have little value in generating an influence score.
As a result, the Klout algorithm draws a lot of criticism, with some suggesting it isn’t a great representation of online influence.
Evaluating influence in the enterprise
A straight port of the technologies and algorithms from Klout within the enterprise presents other problems too, given that influence differs within the enterprise. Scoring is based heavily on levels of participation, and by definition this means the time-poor (such as your C-suite) will find themselves with lower scores. Klout arguably rates pontificators over doers.
Does your CEO really want to know that Ian – the snotty temp from the mail room – is more influential than him? Is this information going to further your career or the intranet’s budget next year? The prospect of enraging senior leaders by suggesting they’re not influential in their own areas of expertise will make many practitioners nervous about influence scoring.
Real influence in a work context would involve the individual’s ability to act on the information received, which in turn is linked to decision-making ability within the organisation. We’ve talked before about measuring outcomes, not outputs – and Klout scoring is simply a measure of output. More robust analysis of influence within the enterprise should look at what happened as a result of a conversation or collaboration, and the relevancy of the knowledge shared. Klout scores just measure how much noise there is on a topic, and how far your messaging reaches – not what business impact your communication had.
Game v. Job
One of the very fair criticisms of gamification relates to mis-alignment in goals. When the goals of the game are perfectly in sync with the goals of the task, job or company, then the motivation derived from a great gamified process is hugely valuable. The risk though is clear: when there is mis-alignment and employees are presented with the choice of a fun game or meeting a dull job objective, there’s a risk that they choose the wrong path. The game v. job balance is critical.
Our concern with adding Klout scores to an enterprise social network is that the score becomes a game. When presented so publicly on a profile, employees will check out each other’s scores, and will work on improving their own — it’s human nature to compete and play. So the question which every organisation should ask is whether the factors that contribute towards an influence score, are behaviours that you would wish your employees to demonstrate? If you score it, it’ll be ranked and the engagement loop will drive behaviour.
Are building a large follower base, creating content with a high propensity to be shared and being a member of other people’s lists part of any job in your organisation? Not yet is our hunch but networking, and doing great work that’s of value to large swathes of the organisation, probably are. So, there’s probably some alignment for some people and adding an influence score may encourage positive action.
But think of poor Susan in accounts. Spreadsheets. Financial trading period end dates. POs. Chasing you for invoices. There’s little in her daily output that has broad enterprise appeal and you’re unlikely to consider sharing her posts. You’re unlikely to follow Susan unless she has the pleasure of working with you on balancing your cost centre. All in, Susan’s influence, by Klout data points, is likely to be low and making that public will be hugely de-motivational to her. Who wants to be told so clearly that no-one cares?
Summary
This is an interesting step, but as details emerge on the mechanics, we’ll have a better view on impact and value. As ever, we welcome your views – what do you think?
Jonathan, great post. I’d be extremely wary of Klout scores in the enterprise. (By coincidence I posted earlier today about how gamification needs to be handled with care http://bit.ly/136jlTK). Generally measures and scores of online participation are a lousy way to reflect the overall contribution of individuals – as you say time-poor leaders, Susan in accounts and possibly non-deskbound employees. It’s not only deflating for them to be ranked as lower than Ian the snotty temp from the mail room, but I think also will just confuse most employees. I can also see some organisations where a high Klout score is regarded as a measure of time spent online – not necessarily regarded as a positive for example in professional services firms where non-chargeable time is still often seen as an overhead.
Yes, a really interesting post, and I agree with Steve Bynghall’s comment too (Steve, perhaps my mention of you will help your Klout score rocket – or possibly not). Gamification is likely to be considered by many intranet managers (myself included). I participated in several sessions in the recent (and excellent) Digistal Workplace 24. Accenture had written its own algorithms to make sure people weren’t gaming the gamification (oh the irony), and I think this is likely to be a key factor in gamified intranets. The Tata Capital example showed, in my view, poor implementation and planning. But that shouldn’t be a reason not to try to identify key influencers. I think algorithms could be built to identify outliers like Ian in the mail room being a raving loon and, equally, Jasmine the CEO being the top influencer just because she’s the CEO.
I think it could be the opposite. If Susan provides good content I would read it, and her rising klout would encourage others to do so too. And who says CEOs need to be the most influential?
The problem is not whether or not Klout accurately measures actual subject matter expertise (SME)-knowledge in the enterprise (it doesn’t), but that people automatically cling to any new stat as if it is “the” answer to the question (it isn’t). What makes Klout useful is its attempt to measure the depth of our social interactions (how far our message is amplified), not just the volume of our tweets (we call this spam). Like most stats, it’s not hard science of any specific point in time, but an indicator to help you identify trends over time. I’m interested to see how they integrate their algorithms with internal systems — that will be the key, I believe, to widespread adoption across the enterprise. Until then, people should only think of it as one piece of data in a field of metrics, not “the” piece of data that they should base all of their social measurements around.
I do agree Christian that the key question now is how Klout will apply their algorithms to Yammer’s enterprise data. However applied, the risk remains that employees will use this data score — rightly or wrongly — as an indicator of social amplification and then work to drive their score higher. This might be good for the company, but might not. Very risky.
But they should look at it that way, and use it as an amplification KPI — that’s what it is. And like every other KPI your organization measures, people will learn how to manipulate it and shape it to best fit their own needs. Welcome to the world of statistics. But to your point, the larger issue is not that employees will follow human nature and do everything possible to bend and shape the metrics that likely determine their bonus (or whether they even keep their jobs), but that employers will once again be lazy and not take this fantastic new data insight and use it — but also regularly optimize it (and all other KPIs and metrics) so make sure the measurements reflect what is actually happening, rather than have what is happening be manipulated to fit into the measurements.
Unless it’s tied to absolute work flows and processess which are highly variable and contextual across organizations than it opens the door for all the pitfalls of “gaming the system”.
Yammer is predicated on “activity streams and discussion threads” which requires mass adoption to have any meaningful impact as an org-wide platform (i.e. intranet).
People are lights out busy doing their work. Team work is done in a very focused, isolated, task driven, goal oriented way. Not by the fantasy of mass collaboration. People don’t have the time or attention to “drink from the fire hose” of information from activity streams or consume multiple discussions to find some relavent bits. No “numbered score” will significantly change this, at least not in any sustainable way.
Finally, as quickly exampled here in the comments, Klout (or the like) become very shallow in conveying any meaninful insights other than a gamified leaderboard. Again, unless its work/task/goal relevant “social measurements” are just that social and less important to actual work and business needs.
Great article and much needed discussion, @DigitalJonathan et al.
Not a great fan of Klout and I think it’s a mistake for Yammer to include it but I’ll let the master of analytics provide an insight here about it. Avinash calls them ‘compund metrics’. Well worth a read.
http://www.kaushik.net/avinash/analytics-tips-improve-search-social-compound-metrics/
[…] See on intranetizen.com […]
[…] See on intranetizen.com […]