Facebook "emotional contagion" Study: A Roundup of Reactions

In case you missed it, there was a dust-up this weekend around the web because of a social science study involving manipulation of Facebook news feeds of users (which might include you, if you are an English language user). Here are three points of contention (in order of intensity):

• Ethics - Was there informed consent?
• Statistical significance - The effect was small, but the data large, what does this mean?
• Linguistics - How did they define and track "emotion "?

First, the original study itself:

Experimental evidence of massive-scale emotional contagion through social networks. Kramer et al. PNAS. Synopsis (from PNAS)

We show, via a massive (N = 689,003) experiment on Facebook, that emotional states can be transferred to others via emotional contagion, leading people to experience the same emotions without their awareness. We provide experimental evidence that emotional contagion occurs without direct interaction between people (exposure to a friend expressing an emotion is sufficient), and in the complete absence of nonverbal cues.

My two cents: We'll never see the actual language data, so the many questions this study raises are destined to be left unanswered.

The Roundup

In Defense of Facebook: If you can only read one analysis, read Tal Yarkoni's deep dive response to the study and its critics. It's worth a full read (comments too). He makes a lot of important points, including the weakness of the effect, the rather tame facts of the actual experiments, and the normalcy of manipulation (that's how life works) but for me, this take-down of the core assumptions underlying the study is the Money Quote:

the fact that users in the experimental conditions produced content with very slightly more positive or negative emotional content doesn’t mean that those users actually felt any differently. It’s entirely possible–and I would argue, even probable–that much of the effect was driven by changes in the expression of ideas or feelings that were already on users’ minds. For example, suppose I log onto Facebook intending to write a status update to the effect that I had an “awesome day today at the beach with my besties!” Now imagine that, as soon as I log in, I see in my news feed that an acquaintance’s father just passed away. I might very well think twice about posting my own message–not necessarily because the news has made me feel sad myself, but because it surely seems a bit unseemly to celebrate one’s own good fortune around people who are currently grieving. I would argue that such subtle behavioral changes, while certainly responsive to others’ emotions, shouldn’t really be considered genuine cases of emotional contagion. 

the Empire strikes back: Humanities Professor Alan Jacobs counters Yarkoni, using language that at times seemed to verge on unhinged, but hyperbole aside, he takes issue with claims that the experiment was ethical simply because users signed a user agreement (that few of them ever actually read). Money Quote:

This seems to be missing the point of the complaints about Facebook’s behavior. The complaints are not “Facebook successfully manipulated users’ emotions” but rather “Facebook attempted to manipulate users’ emotions without informing them that they were being experimented on.” That’s where the ethical question lies, not with the degree of the manipulation’s success. “Who cares if that guy was shooting at you? He missed, didn’t he?” — that seems to be Yarkoni’s attitude. 

Facebook admits manipulating users' emotions by modifying news feeds: Across the pond, The Guardian got into the kerfuffle. Never one to miss a chance to go full metal Orwell on us, the Guardian gives us this ridiculous Money Quote with not a whiff of counter-argument:

In a series of Twitter posts, Clay Johnson, the co-founder of Blue State Digital, the firm that built and managed Barack Obama's online campaign for the presidency in 2008, said: "The Facebook 'transmission of anger' experiment is terrifying." He asked: "Could the CIA incite revolution in Sudan by pressuring Facebook to promote discontent? Should that be legal? Could Mark Zuckerberg swing an election by promoting Upworthy [a website aggregating viral content] posts two weeks beforehand? Should that be legal?"

This Clay Johnson guy is hilarious, in a dangerously stupid way. How does his bonkers ranting rate two paragraphs in a Guardian story?


Everything We Know About Facebook's Secret Mood Manipulation Experiment: The Atlantic provides a roundup of sorts and a review of the basic facts, and some much needed sanity about the limitations of LIWC (which is a limited, dictionary tool that, except for the evangelical zeal of its creator James Pennebaker, would be little more than a toy for undergrad English majors to play with). Article also provides important quotes from the study's editor, Princeton's Susan Fiske. This also links to a full interview with professor Fiske.

Emotional Contagion on Facebook? More Like Bad Research Methods: If you have time to read two and only two analyses of the Facebook study, first read Yarkoni above, then read John Grohol's excellent fisking of the (mis-)use of LIWC as tool for linguistic study. Money Quote:

much of human communication includes subtleties ... — without even delving into sarcasm, short-hand abbreviations that act as negation words, phrases that negate the previous sentence, emojis, etc. — you can’t even tell how accurate or inaccurate the resulting analysis by these researchers is. Since the LIWC 2007 ignores these subtle realities of informal human communication, so do the researchers.

Analyzing Facebook's PNAS paper on Emotional Contagion: Nitin Madnani provides an NLPers
detailed fisking of the experimental methods, with special attention paid to the flaws of LIWC (with bonus comment from Brendan O'Connor, recent CMU grad and new U Amherst professor). Money Quote:

Far and away, my biggest complaint is that the Facebook scientists simply used a word list to determine whether a post was positive or negative. As someone who works in natural language processing (including on the task of analyzing sentiment in documents), such a rudimentary system would be treated with extreme skepticism in our conferences and journals. There are just too many problems with the approach, e.g. negation ("I am not very happy today because ..."). From the paper, it doesn't look like the authors tried to address these problems. In short, I am skeptical the whether the experiment actually measures anything useful. One way to address comments such as mine is to actually release the data to the public along with some honest error analysis about how well such a naive approach actually worked.

Facebook’s Unethical Experiment: Tal Yarkoni's article above provides a pretty thorough fisking of this Slate screed. I'll just add that Slate is never the place I'd go to for well reasoned, scientific analysis. A blow-by-blow deep dive into the last episode of Orange Is The New Black? Oh yeah, Slate has that genre down cold.


Anger Builds Over Facebook's Emotion-Manipulation Study: The site that never met a listicle it didn't love, Mashable provides a short article that fails to live up to its title. They provide little evidence that anger is building beyond screen grabs of a whopping four Twitter feeds. Note, they completely ignore the range of people supporting the study (no quotes from the authors, for example). As far as I can tell, there is no hashtag for anti-Facebook study tweets.


Facebook Manipulated User News Feeds To Create Emotional Responses: Forbes wonders aloud about the mis-use of the study by marketers. Money Quote:

What harm might flow from manipulating user timelines to create emotions?  Well, consider the controversial study published last year (not by Facebook researchers) that said companies should tailor their marketing to women based on how they felt about their appearance.  That marketing study began by examining the days and times when women felt the worst about themselves, finding that women felt most vulnerable on Mondays and felt the best about themselves on Thursdays ... The Facebook study, combined with last year’s marketing study suggests that marketers may not need to wait until Mondays or Thursdays to have an emotional impact, instead  social media companies may be able to manipulate timelines and news feeds to create emotionally fueled marketing opportunities.

You don't have to work hard to convince me that marketing professionals have a habit of half-digesting science they barely understand to try to manipulate consumers. That's par for the course in that field, as far as I can tell. Just don't know what scientists producing the original studies can do about it. Monkey's gonna throw shit. Don't blame the banana they ate.


Creepy Study Shows Facebook Can Tweak Your Moods Through ‘Emotional Contagion’. The Blaze witer Zach Noble summed up the negative reaction this way: a victory for scientific understanding with some really creepy ramifications. But I think it only seems creepy if you mis-understand the actual methods.

Final Thought: It's the bad science that creeps me out more than the questionable ethics. Facebook is data, lets use it wisely.

Thinking Words (part 1)

(image from make-noise.com)

I’d like to present a brief lesson in contemporary linguistic research with the goal of showing that we live in a marvelous age of quick and ready research tools freely available to even the most humble of internet users. Hence, a little effort goes a long way. My point is that when we make claims about language usage (and by "we" I mostly mean those of us who present our claims about language to the public via the interwebz) we need not make such claims based on our intuitions and emotions; rather, we can perform a little due diligence in a way that linguistic pontificators of the past simply could not. And bully for us.

My subject for today’s Full Liberman is this classic example of language mavenry from Prospect magazine: Words that think for us by Edward Skidelsky, lecturer in philosophy at Exeter University (HT Arts and Letters Daily). In this article, Skidelsky laments the following “linguistic shift”:

No words are more typical of our moral culture than “inappropriate” and “unacceptable.” They seem bland, gentle even, yet they carry the full force of official power. When you hear them, you feel that you are being tied up with little pieces of soft string. Inappropriate and unacceptable began their modern careers in the 1980s as part of the jargon of political correctness. They have more or less replaced a number of older, more exact terms: coarse, tactless, vulgar, lewd. They encompass most of what would formerly have been called “improper” or “indecent.”…“Inappropriate” and “unacceptable” are the catchwords of a moralism that dare not speak its name. They hide all measure of righteous fury behind the mask of bureaucratic neutrality. For the sake of our own humanity, we should strike them from our vocabulary.


UPDATE: A very lively discussion of the meaning of the words in question (something I largely ignore) has broken out on Language Log here)

This article makes four testable linguistic claims:

1. The words inappropriate and unacceptable have increased in frequency over the last couple decades.
2. This frequency increase is due to replacing other words: coarse, tactless, vulgar, lewd, improper, and indecent.
3. These other words are “older”
4. These other words are “more exact”

With a little investigation using entirely freely available online linguistics tools, we can easily fact check each of these claims. In the interest of time, I'll answer the first two together.

First and Second -- Has the frequency of inappropriate and unacceptable increased since the 1980s? & have they replaced the other words?

In order to quickly get some data, I took this to mean the frequency of the first two words have increased while the frequency of the other words have decreased since the 1980s (is this is an unfair interpretation?. In any case, that’s how I operationalized my methodology.). Thanks to Mark Davies excellent resource, the TIME Corpus of American English (100 million words, 1923-2006, requires registration, but it's free) we can quickly get a snapshot of the frequency of each word’s usage for the last 9 decades (not bad, huh? Thanks Mark!!).

Caveat: raw frequency is a poor data point by itself. What we really need is a way to compare apples to apples and oranges to oranges, and the problem we have is different sized corpora for each decade. Fear not, Davies does this work for us. His handy dandy interface allows us to report frequency per million, thus giving us comparable frequencies across different decades.

Using the TIME corpus, I discovered the frequency per million of each word per decade. Then I entered that data into a spread sheet. I used Excel 2007 to create a line graph of these frequencies.

Here's the relevant data:


And here's the graph:

UPDATE (2hrs after original post): original graph was confusing (same graph, just confusing labels) so I fixed it.

What this shows us is that both inappropriate and unacceptable do in fact show a rise in frequency (consistent with Skidelsky's claim), but starting in the 1960s, not 1980s. However, unacceptable shows a more recent dramatic decline, which is inconsistent with his claim. Lewd actually made a bit of a comeback in the 1990s (thank you Mr. Clinton?), but has since dropped back (it's a bit of a jumpy word, isn't it?). The other words do seem to be falling off in usage, consistent with Skidelsky's claim. So the picture is not quite what Skidelsky thinks it is, though he does seem to be on to something.

UPDATE: See myl's plot of this same data (but grouping the words as Skidelsky does) here which suggests that "'coarse', 'tactless', 'vulgar' etc. declined until WWII and then stayed about the same, perhaps with an additional decline in past decade; while 'inappropriate' and 'unacceptable' rose gradually from the 1930s to 1970 or so, and then leveled off. " The plot does suggest that we could view the two groups as having roughly inverted frequency, somewhat conforming to Skidelsky's hunch.

Third -- Are these other four words “older”?

Unfortunately, as I am no longer affiliated with a university, therefore I have no access to the OED (I’ve decided not to pay the $295 for their individual subscription. Condemn me if you must). If anyone would care to look those up and post them in comments, I’d be happy to update. Most of these words have multiple senses and the question is, when did the most relevant sense enter usage? For that, the OED is most valuable. Again, you can do that work for me, or send me a check for $295.

However, a simple search of the Merriam Webster online dictionary gives us a quick answer:

unacceptable = 15th century
inappropriate = 1804
coarse = 14th century
tactless = circa 1847
vulgar = 14th century
lewd = 14th century
improper = 15th century
indecent = circa 1587

This data suggests these five words fall into roughly two groups:

A -- words that entered the language around the 19th century

• Set A = inappropriate, tactless

B -- words that entered the language around the 15-16 centuries

• Set B = unacceptable, coarse, vulgar, lewd, improper, indecent

This grouping does not conform to Skidelsky’s assumption that inappropriate & unacceptable fall together in a newer class and the others in an older class.


UPDATE: much thanks to commenter panoptical who provides the following OED dates which appear to largely confirm the Merriam Webster dates, with the notable except of lewd which dates back to Old English it seems...does have a certain Beowulf ring to it, doesn't it?

unacceptable: 1483
inappropriate: 1804
coarse: 1424
tactless: 1847
vulgar: 1391
lewd: c890
improper: 1531
indecent: 1563


Fourth -- Are the other words "more exact"?

Finding a way to empirically test this is a challenge I will take up in later post (you can see Wordnet coming, can't you?). It will require teasing apart senses and relationships between senses (oh my, I wish I had the OED right now...).