Facebook "emotional contagion" Study: A Roundup of Reactions

In case you missed it, there was a dust-up this weekend around the web because of a social science study involving manipulation of Facebook news feeds of users (which might include you, if you are an English language user). Here are three points of contention (in order of intensity):

• Ethics - Was there informed consent?
• Statistical significance - The effect was small, but the data large, what does this mean?
• Linguistics - How did they define and track "emotion "?

First, the original study itself:

Experimental evidence of massive-scale emotional contagion through social networks. Kramer et al. PNAS. Synopsis (from PNAS)

We show, via a massive (N = 689,003) experiment on Facebook, that emotional states can be transferred to others via emotional contagion, leading people to experience the same emotions without their awareness. We provide experimental evidence that emotional contagion occurs without direct interaction between people (exposure to a friend expressing an emotion is sufficient), and in the complete absence of nonverbal cues.

My two cents: We'll never see the actual language data, so the many questions this study raises are destined to be left unanswered.

The Roundup

In Defense of Facebook: If you can only read one analysis, read Tal Yarkoni's deep dive response to the study and its critics. It's worth a full read (comments too). He makes a lot of important points, including the weakness of the effect, the rather tame facts of the actual experiments, and the normalcy of manipulation (that's how life works) but for me, this take-down of the core assumptions underlying the study is the Money Quote:

the fact that users in the experimental conditions produced content with very slightly more positive or negative emotional content doesn’t mean that those users actually felt any differently. It’s entirely possible–and I would argue, even probable–that much of the effect was driven by changes in the expression of ideas or feelings that were already on users’ minds. For example, suppose I log onto Facebook intending to write a status update to the effect that I had an “awesome day today at the beach with my besties!” Now imagine that, as soon as I log in, I see in my news feed that an acquaintance’s father just passed away. I might very well think twice about posting my own message–not necessarily because the news has made me feel sad myself, but because it surely seems a bit unseemly to celebrate one’s own good fortune around people who are currently grieving. I would argue that such subtle behavioral changes, while certainly responsive to others’ emotions, shouldn’t really be considered genuine cases of emotional contagion. 

the Empire strikes back: Humanities Professor Alan Jacobs counters Yarkoni, using language that at times seemed to verge on unhinged, but hyperbole aside, he takes issue with claims that the experiment was ethical simply because users signed a user agreement (that few of them ever actually read). Money Quote:

This seems to be missing the point of the complaints about Facebook’s behavior. The complaints are not “Facebook successfully manipulated users’ emotions” but rather “Facebook attempted to manipulate users’ emotions without informing them that they were being experimented on.” That’s where the ethical question lies, not with the degree of the manipulation’s success. “Who cares if that guy was shooting at you? He missed, didn’t he?” — that seems to be Yarkoni’s attitude. 

Facebook admits manipulating users' emotions by modifying news feeds: Across the pond, The Guardian got into the kerfuffle. Never one to miss a chance to go full metal Orwell on us, the Guardian gives us this ridiculous Money Quote with not a whiff of counter-argument:

In a series of Twitter posts, Clay Johnson, the co-founder of Blue State Digital, the firm that built and managed Barack Obama's online campaign for the presidency in 2008, said: "The Facebook 'transmission of anger' experiment is terrifying." He asked: "Could the CIA incite revolution in Sudan by pressuring Facebook to promote discontent? Should that be legal? Could Mark Zuckerberg swing an election by promoting Upworthy [a website aggregating viral content] posts two weeks beforehand? Should that be legal?"

This Clay Johnson guy is hilarious, in a dangerously stupid way. How does his bonkers ranting rate two paragraphs in a Guardian story?


Everything We Know About Facebook's Secret Mood Manipulation Experiment: The Atlantic provides a roundup of sorts and a review of the basic facts, and some much needed sanity about the limitations of LIWC (which is a limited, dictionary tool that, except for the evangelical zeal of its creator James Pennebaker, would be little more than a toy for undergrad English majors to play with). Article also provides important quotes from the study's editor, Princeton's Susan Fiske. This also links to a full interview with professor Fiske.

Emotional Contagion on Facebook? More Like Bad Research Methods: If you have time to read two and only two analyses of the Facebook study, first read Yarkoni above, then read John Grohol's excellent fisking of the (mis-)use of LIWC as tool for linguistic study. Money Quote:

much of human communication includes subtleties ... — without even delving into sarcasm, short-hand abbreviations that act as negation words, phrases that negate the previous sentence, emojis, etc. — you can’t even tell how accurate or inaccurate the resulting analysis by these researchers is. Since the LIWC 2007 ignores these subtle realities of informal human communication, so do the researchers.

Analyzing Facebook's PNAS paper on Emotional Contagion: Nitin Madnani provides an NLPers
detailed fisking of the experimental methods, with special attention paid to the flaws of LIWC (with bonus comment from Brendan O'Connor, recent CMU grad and new U Amherst professor). Money Quote:

Far and away, my biggest complaint is that the Facebook scientists simply used a word list to determine whether a post was positive or negative. As someone who works in natural language processing (including on the task of analyzing sentiment in documents), such a rudimentary system would be treated with extreme skepticism in our conferences and journals. There are just too many problems with the approach, e.g. negation ("I am not very happy today because ..."). From the paper, it doesn't look like the authors tried to address these problems. In short, I am skeptical the whether the experiment actually measures anything useful. One way to address comments such as mine is to actually release the data to the public along with some honest error analysis about how well such a naive approach actually worked.

Facebook’s Unethical Experiment: Tal Yarkoni's article above provides a pretty thorough fisking of this Slate screed. I'll just add that Slate is never the place I'd go to for well reasoned, scientific analysis. A blow-by-blow deep dive into the last episode of Orange Is The New Black? Oh yeah, Slate has that genre down cold.


Anger Builds Over Facebook's Emotion-Manipulation Study: The site that never met a listicle it didn't love, Mashable provides a short article that fails to live up to its title. They provide little evidence that anger is building beyond screen grabs of a whopping four Twitter feeds. Note, they completely ignore the range of people supporting the study (no quotes from the authors, for example). As far as I can tell, there is no hashtag for anti-Facebook study tweets.


Facebook Manipulated User News Feeds To Create Emotional Responses: Forbes wonders aloud about the mis-use of the study by marketers. Money Quote:

What harm might flow from manipulating user timelines to create emotions?  Well, consider the controversial study published last year (not by Facebook researchers) that said companies should tailor their marketing to women based on how they felt about their appearance.  That marketing study began by examining the days and times when women felt the worst about themselves, finding that women felt most vulnerable on Mondays and felt the best about themselves on Thursdays ... The Facebook study, combined with last year’s marketing study suggests that marketers may not need to wait until Mondays or Thursdays to have an emotional impact, instead  social media companies may be able to manipulate timelines and news feeds to create emotionally fueled marketing opportunities.

You don't have to work hard to convince me that marketing professionals have a habit of half-digesting science they barely understand to try to manipulate consumers. That's par for the course in that field, as far as I can tell. Just don't know what scientists producing the original studies can do about it. Monkey's gonna throw shit. Don't blame the banana they ate.


Creepy Study Shows Facebook Can Tweak Your Moods Through ‘Emotional Contagion’. The Blaze witer Zach Noble summed up the negative reaction this way: a victory for scientific understanding with some really creepy ramifications. But I think it only seems creepy if you mis-understand the actual methods.

Final Thought: It's the bad science that creeps me out more than the questionable ethics. Facebook is data, lets use it wisely.

Again With The Bad Science Reporting....

A nice post over at Thoughtomics debunks an all too typical example of bad science reporting run amok involving chickens and eggs and proteins... you see where this is going, right? sigh...

Money Quote:

I didn’t exactly hold mainstream science journalism in high esteem, but I’m amazed that science journalists continue ‘covering’ science stories in this way, even when readers are calling them out. While the trouble may have started with a misleading introduction and a quirky quote, it is the journalist’s responsibility to check facts and put a story into a context. Coverage like this does more harm than good for the public image of science reporting and scientists themselves.

Boom Boom Syntax

Mr. Verb has a post up about yet another NYT's article on animal language that does a poor job of reporting the facts:

I've been wondering about what syntax really is and how we would show it exists since reading this in the NYT this morning. It reports work by Klaus Zuberbühler and others arguing that Campbell's monkeys (cute critters, see pic) in Ivory Coast not only have some sound-meaning correspondences (boom boom mean 'come here once', krak means 'leopard', etc.), but that they have what they're calling inflectional morphology, a suffix -oo, which sounds like an auditory evidential — indicating you've heard but not seen something.

As Mr. Verb points out, the original scholarly article is not yet available so we are unable to fact check this one...yet.

Abracadabra! I Win!

(image from Slate.com)

I tend to avoid Slate.com these days because, frankly, I typically find myself scoffing at some idiot article they've published that promotes such a ridiculous mis-reading of academic research that it's hardly worth finishing... like this one from today: A Better Way to Fight With Your Husband which linked to this article: The Healthiest Way To Fight With Your Husband. It's a classic piece of idiot journalism worthy of a Full Liberman* if only it weren't so trivial and obvious as to be beneath the man, so I'll take a crack at it.

The big point is that fabulous new research from real life scholars (psychologists nonetheless, and they're almost like scientists) proves that women should use particular words when yelling at their husbands (the experiment used heterosexual married couples). Pretty awesome, ain't it! Just use the right words, and like a magic key you can unlock the mysteries of the brain and make it do what you please (okay, I'm starting to exaggerate, but less than you might think).

First let's look at the way the academic article is summarized in the puff piece that Slate linked to:

A new study of married couples, however, has found physiological evidence for one technique to diffuse tension: choosing the right fighting words. Couples who used analytical language, such as “think,” “understand,” “because,” or “reason,” during heated arguments were able to keep important stress-related chemicals in check, according to research published in the latest issue of the journal Health Psychology. Cytokines are inflammatory chemicals that spike during periods of prolonged tension and can lower your immunity and lead to early frailty, Type 2 diabetes, arthritis, and some cancers. The authors noted a curious gender twist in their results. Husbands benefitted from their wives’ measured language, but a man’s carefully chosen words had little effect on a woman’s cytokine balance.

To be fair, here is a passage from the authors' abstract of the original article:

Effects of word use were not mediated by ruminative thoughts after conflict. Although both men and women benefited from their own cognitive engagement, only husbands' IL-6 patterns were affected by spouses' engagement. Conclusion: In accord with research demonstrating the value of cognitive processing in emotional disclosure, this research suggests that productive communication patterns may help mitigate the adverse effects of relationship conflict on inflammatory dysregulation.

And here is a passage from this interview with the first author, Jennifer Graham, Penn State assistant professor of biobehavioral health:

"We specifically looked at words that are linked with cognitive processing in other research and which have been predictive of health in studies where people express emotion about stressful events," explained Graham. "These are words like 'think,' 'because,' 'reason' (and) 'why' that suggest people are either making sense of the conflict or at least thinking about it in a deep way."

For the study, the 42 couples made two separate overnight visits over two weeks.

"We found that, controlling for depressed mood, individuals who showed more evidence of cognitive discussion during their fights showed smaller increases in both Il-6 and TNF-alpha cytokines over a 24-hour period," said Graham, whose findings appear in the current issue of Health Psychology.

During their first visit, couples had a neutral, fairly supportive discussion with their spouse. But during the second visit, couples focused on the topic of greatest contention between them.

"An interviewer figured out ahead of time what made the man and woman most upset in terms of their relationship, and we gave each person a turn to talk about that issue," said Graham.

Researchers measured the levels of cytokines before and after the two visits and used linguistic software to determine the percentage of certain types of words from a transcript of the conversation. (my italics)

The researchers' results suggest that people who used more cognitive words during the fight showed a smaller increase in the Il-6 and TNF-alpha. Cognitive words used during the neutral discussion had no effect on the cytokines.

When they averaged the couples' cognitive words during the fight, they found a low average translated into a steeper increase in the husbands' Il-6 over time. There were no effects on the TNF-alpha. However, neither couple's nor spouse's cognitive word use predicted changes in wives' Il-6, or TNF-alpha levels for either wives or husbands.

Graham speculates that women may be more adept at communication and perhaps their cognitive word use had a bigger impact on their husbands. Wives also were more likely than husbands to use cognitive words.

Well, thank gawd they used fancy computers to count cognitive words! After reading these three descriptions, it was clear to me that the original work is likely flawed. I don't have access to the original study, unfortunately, but taken together, the abstract and first author's interview suggests to me that it makes the same mistake most non-linguists make: they assume the linguistics part is easy and don't put enough effort into it. Dr. Graham's initial claim in the interview jumps out at me: "We specifically looked at words that are linked with cognitive processing in other research..."

Hmm? Words that are "linked with cognitive processing?" What does this mean? I would love to see the references page to follow-up on this "other research." Graham later refers to these as "cognitive words." They are alternately referred to as analytical language, measured language, conflict-resolution words, and cerebral words. From the puff piece and the interview we have five examples:

1. because
2. reason
3. why
4. think
5. understand

Huh? One conjunction, one interrogative, and three verbs of cognition. Hmmm. Is there any intuitive reason to believe that "because" is "linked with cognitive processing" in some special way that other words are not? Is it the fact that it grammatically links clauses? Many words do this. Are the verbs on the list simply because they are verbs of cognition? Are run and jump less "linked with cognition" because they are verbs of motion? I would have to speculate on what this "other research" discovered about the magical properties of the special words that make them the key to brain chemicals. Abracadabra! Poof! Also, it's not at all clear to me why they averaged the couples' frequency count. What is this average supposed to tell us?

However, the puff piece makes the leap into idiotsville all by itself:

"The study is significant because it’s one of the first to link language with biological markers and show what kinds of words help sparring couples rather than just recommending they “communicate more,” explains James Pennebaker, chair of the department of psychology at the University of Texas-Austin, who has studied the role of language on relationships." (my italics).

Nope. No link. Just a transcript. Given the study's methodology of counting words in a transcript, at no point could they possibly have been able to show any causal relationship between a particular word's utterance and the levels of a particular chemical in a person's brain.

The puff piece authors pull the classic journalist's trick of "being fair" by adding actual linguist Deborah Tannen's skepticism of the "link" between particular words and particular chemicals, but they abandon all skepticism just a few sentences later and end with a bang! "”Even when it seems like he is ignoring you, your words may be having an effect—at least on a chemical level,” says Graham"

Sigh.

*I'm going to start using the term "The Full Liberman" to refer to Mark Liberman's excellent manner of debunking bad journalism (see here and here for examples).

UPDATE (11/28/09): A nice summary of Full Liberman's at LL here.