The “friendly world syndrome” induced by simple filtering rules
I’ve written previously about how filtered activity streams [edit: i.e. news feeds] can lead to biased views of behaviors in our social neighborhoods. Recent conversations with two people writing popular-press books on related topics have helped me clarify these ideas. Here I reprise previous comments on filtered activity streams, aiming to highlight how they apply even in the case of simple and transparent personalization rules, such as those used by Twitter.
—
Birds of a feather flock together. Once flying together, a flock is also subject to the same causes (e.g., storms, pests, prey). Our friends, family, neighbors, and colleagues are more similar to us for similar reasons (and others). So we should have no illusions that the behaviors, attitudes, outcomes, and beliefs of our social neighborhood are good indicators of those of other populations — like U.S. adults, Internet users, or homo sapiens of the past, present, or future. The apocryphal Pauline Kael quote “How could Nixon win? No one I know voted for him” suggests both the ease and error of this kind of inference. I take it as a given that people’s estimates of larger populations’ behaviors and beliefs are often biased in the direction of the behaviors and beliefs in their social neighborhoods. This is the case with and without “social media” and filtered activity streams — and even mediated communication in general.
That is, even without media, our personal experiences are not “representative” of the American experience, human experience, etc., but we do (and must) rely on it anyway. One simple cognitive tool here is using “ease of retrieval” to estimate how common or likely some event is: we can estimate how common something is based on how easy it is to think of. So if something prompts someone to consider how common a type of event is, they will (on average) estimate the event as more common if it is more easy to think of an example of the event, imagine the event, etc. And our personal experiences provide these examples and determine how easy they are to bring to mind. Both prompts and immediately prior experience can thus affect these frequency judgments via ease of retrieval effects.
Now this is not to say that we should think as ease of retrieval heuristics as biases per se. Large classes and frequent occurrences are often more available to mind than those that are smaller or less frequent. It is just that this is also often not the case, especially when there is great diversity in frequency among physical and social neighborhoods. But certainly we can see some cases where these heuristics fail.
Media are powerful sources of experiences that can make availability and actual frequency diverge, whether by increasing the biases in the direction of projecting our social neighborhoods onto larger population or in other, perhaps unexpected directions. In a classic and controversial line of research in the 1970s and 80s, Gerbner and colleagues argued that increased television-watching produces a “mean world syndrome” such that watching more TV causes people to increasingly overestimate, e.g., the fraction of adult U.S. men employed in law enforcement and the probability of being a victim of violent crime. Their work did not focus on investigating heuristics producing these effects, but others have suggested the availability heuristic (and related ease of retrieval effects) as at work. So even if my social neighborhood has fewer cops or victims of violent crime than the national average, media consumption and the availability heuristic can lead me to overestimate both.
Personalized and filtered activity streams certainly also affect us through some of the same psychological processes, leading to biases in users’ estimates of population-wide frequencies. They can aIso bias inference about our own social neighborhoods. If I try to estimate how likely a Facebook status update by a friend is to receive a comment, this estimate will be affected by the status updates I have seen recently. And if content with comments is more likely to be shown to me in my personalized filtered activity stream (a simple rule for selecting more interesting content, when there is too much for me to consume it all), then it will be easier for me to think of cases in which status updates by my friends do receive comments.
In my previous posts on these ideas, I have mainly focused on effects on beliefs about my social neighborhood and specifically behaviors and outcomes specific to the service providing the activity stream (e.g., receiving comments). But similar effects apply for beliefs about other behaviors, opinions, and outcomes. In particular, filtered activity streams can increase the sense that my social neighborhood (and perhaps the world) agrees with me. Say that content produced by my Facebook friends with comments and interaction from mutual friends is more likely to be shown in my filtered activity streams. Also assume that people are more likely to express their agreement in such a way than substantial disagreement. As long as I am likely to agree with most of my friends, then this simple rule for filtering produces an activity stream with content I agree with more than an unfiltered stream would. Thus, even if I have a substantial minority of friends with whom I disagree on politics, this filtering rule would likely make me see less of their content, since it is less likely to receive (approving) comments from mutual friends.
I’ve been casually calling this larger family of effects this the “friendly world syndrome” induced by filtered activity streams. Like the mean world syndrome of the television cultivation research described above, this picks out a family of unintentional effects of media. Unlike the mean world syndrome, the friendly world syndrome includes such results as overestimating how many friends I have in common with my friends, how much positive and accomplishment-reporting content my friends produce, and (as described) how much I agree with my friends.1
Even though the filtering rules I’ve described so far are quite simple and appealing, they still are more consistent with versions of activity streams that are filtered by fancy relevance models, which are often quite opaque to users. Facebook News Feed — and “Top News” in particular — is the standard example here. On the other hand, one might think that these arguments do not apply to Twitter, which does not apply any kind of machine learning model estimating relevance to filtering users’ streams. But Twitter actually does implement a filtering rule with important similarities to the “comments from mutual friends” rule described above. Twitter only shows “@replies” to a user on their home page when that user is following both the poster of the reply and the person being replied to.2 This rule makes a lot of sense, as a reply is often quite difficult to understand without the original tweet. Thus, I am much more likely to see people I follow replying to people I follow than to others (since the latter replies are encountered only from browsing away from the home page. I think this illustrates how even a straightforward, transparent rule for filtering content can magnify false consensus effects.
One aim in writing this is to clarify that a move from filtering activity streams using opaque machine learning models of relevance to filtering them with simple, transparent, user-configurable rules will likely be insufficient to prevent the friendly world syndrome. This change might have many positive effects and even reduce some of these effects by making people mindful of the filtering.3 But I don’t think these effects are so easily avoided in any media environment that includes sensible personalization for increased relevance and engagement.
- This might suggest that some of the false consensus effects observed in recent work using data collected about Facebook friends could be endogenous to Facebook. See Goel, S., Mason, W., & Watts, D. J. (2010). Real and perceived attitude agreement in social networks. Journal of Personality and Social Psychology, 99(4), 611-621. doi:10.1037/a0020697 [↩]
- Twitter offers the option to see all @replies written by people one is following, but 98% of users use the default option. Some users were unhappy with an earlier temporary removal of this feature. My sense is that the biggest complaint was that removing this feature removed a valuable means for discovering new people to follow. [↩]
- We are investigating this in ongoing experimental research. Also see Schwarz, N., Bless, H., Strack, F., Klumpp, G., Rittenauer-Schatka, H., & Simons, A. (1991). Ease of retrieval as information: Another look at the availability heuristic. Journal of Personality and Social Psychology, 61(2), 195-202. doi:10.1037/0022-3514.61.2.195 [↩]
Persuasion profiling and genres: Fogg in 2006
Maurits Kaptein and I have recently been thinking a lot about persuasion profiling — estimating and adapting to individual differences in responses to influence strategies based on past behavior and other information. With help from students, we’ve been running experiments and building statistical models that implement persuasion profiling.
My thinking on persuasion profiling is very much in BJ Fogg’s footsteps, since he has been talking about persuasion profiling in courses, lab meetings, and personal discussions since 2004 or earlier.
Just yesterday, I came across this transcript of BJ’s presentation for an FTC hearing in 2006. I was struck at how much it anticipates some of what Maurits and I have written recently (more on this later). I’m sure I watched the draft video of the presentation back then and it’s influenced me, even if I forgot some of the details.
Here is the relevant excerpt from BJ’s comments for the FTC:
Persuasion profiling means that each one of us has a different set of persuasion strategies that affect us. Just like we like different types of food or are vulnerable to giving in to different types of food on a diet, we are vulnerable to different types of persuasion strategies.
On the food example, I love old-fashioned popcorn, and if I go to a party and somebody has old-fashioned popcorn, I will probably break down and eat it. On the persuasion side of things, I know I’m vulnerable to trying new things, to challenges and to anything that gets measured. If that’s proposed to me, I’m going to be vulnerable and I’m going to give it a shot.
Whenever we go to a Web site and use an interactive system, it is likely they will be capturing what persuasion strategies work on us and will be using those when we use the service again. The mapping out of what makes me tick, what motivates me can also be bought or sold, just like a credit report.
So imagine I’m going in to buy a new car and the person selling me the car downloads my credit report but also buys my persuasion profile. I may or may not know about this. Imagine if persuasion profiles are available on political campaigns so that when I visit a Web site, the system knows it is B.J. Fogg, and it changes [its] approach based on my vulnerabilities when it comes to persuasion.
Persuasive technology will touch our lives anywhere that we access digital products or services, in the car, in our living room, on the Web, through our mobile phones and so on. Persuasive technology will be all around us, and unlike other media types, where you have 30-second commercial or a magazine ad, you have genres you can understand, when it comes to computer-based persuasion, it is so flexible that it won’t have genre boundaries. It will come to us in the ordinary course of our lives, as we are working on a Web site, as we are editing a document, as we are driving a car. There won’t be clear markers about when you are being persuaded and when you are not.
This last paragraph is about the “genrelessness” of many persuasive technologies. This isn’t directly on the topic of persuasion profiling, but I see it as critically relevant. Persuasion profiling is likely to be most effective when invisible and undisclosed to users. From this and the lack of genre-based flags for persuasive technology it follows that we will frequently be “persuasion profiled” without knowing it.
No entity without identity: individuating attitudes in social psychology
Social psychologists like to write about attitudes. In fact, following Allport (1935), many of them have happily commented that the attitude is the most central and indispensable construct in social psychology (e.g., Petty, Wegener, Fabrigar, 1997). Here is a standard definition of an attitude: an attitude is
a psychological tendency that is expressed by evaluating a particular entity with some degree of favor or disfavor. (Eagly & Chaiken, 2007, p. 598)
A somewhat more specific view has it that attitudes are
associations between a given object and a given summary evaluation of the object — associations that can vary in strength and, hence, in their accessibility from memory. (Fazio, 2007, p. 608)
Attitudes are also supposed to be important for predicting behavior, though the attitude–behavior link is the subject of a great deal of controversy, which I can’t fully treat here. An extreme, design-oriented view is expressed by a B.F. Skinner-channeling B.J. Fogg:
While Fogg isn’t representative of mainstream, contemporary social psychology, similarly skeptical thoughts are expressed by investigators like Schwartz (2007). On the other hand, one common view of the attitude–behavior link is that it is quite strong (Kraus, 1997), but that (a) many research methods fail to measure attitudes and behaviors with regard to the same entities (Ajzen & Fishbein, 1977) and (b) this link is an important empirical subject, not built into the attitude construct by definition (Fazio, 2007; Zanna & Rempel, 1988).
I’ll set aside for now just how useful attitudes are for predicting behavior. But what should we make of this construct? That is, should we keep it around? Do we expect something like social psychology’s attitudes to be part of a mature science of human behavior?
Maybe I’m a sucker for a good slogan, but when I read psychologists’ on attitudes, I think of Quine’s slogan: no entity without identity. That is, we shouldn’t posit objects that don’t have identity conditions — the conditions under which we say that X and Y are the same object.
This slogan, followed strictly in everyday life, can get tricky: a restaurant changes owners and name — is it the same restaurant? But it is pretty compelling when it comes to the entities we use in science. Of course, philosophers have debated this slogan — and many particular proposed cases of posited entities lacking identity conditions (e.g., entities in quantum physics) — so I’ll leave it that lacking identity conditions might vary in how much trouble it causes for a theory that uses such entities.
What I do want to comment on is how strikingly social psychology’s attitudes lack good identity conditions — and thus have no good way of being individuated. While we might think this doesn’t cause much trouble in this case (as I just noted), I actually think it creates a whole family of pseudo-problems that psychologists spend their time on and build theories around.
First, evidence that there is trouble in individuating attitudes: As is clear from the definition of an attitude provided above, attitudes are supposed to be individuated by their object:
This evaluative responding is directed to some entity or thing that is its object—that is, we may evaluate a person (George W. Bush), a city (Chicago), an ideology (conservatism), and a myriad of other entities. In the language of social psychology, an entity that is evaluated is known as an attitude object. Anything that is discriminable or held in mind, sometimes below the level of conscious awareness, can be evaluated and therefore can function as an attitude object. Attitude objects may be abstract (e.g., liberalism, religious fiindamentalism) or concrete (e.g., the White House, my green raincoat) as well as individual (e.g., Condoleezza Rice, my sister-in-law) or collective (e.g., undocumented workers, European nations). (Eagly & Chaiken, 2007, p. 584)
So, for example, I can have an attitude towards Obama. This attitude can then have internal structure, such that there are multiple evaluations involved (e.g., implicit and explicit). This seems pretty straightforward: it is at least somewhat clear when some cognitive structures share the Obama as object.1
But trouble is not far around the corner. Much discussion of attitudes involves attitudes objects that are abstract objects — like sets or classes of objects– embedded in a whole set of relationships. For example, I might have attitudes towards snakes, Blacks, or strawberry ice cream. And there isn’t any obvious way that the canonical class by which attitudes are to be individuated gets picked out. A person has evaluative responses to strawberry ice cream, Ben & Jerry’s brand ice cream, ice cream in general, the larger class of such foods (including frozen yogurt, gelato, “soft serve”), foods that cool one down when eaten, etc.
This doesn’t just work with ice cream. (Obama instantiates many properties and is a member of many relevant classes.)
At this point, you might be thinking, how does all this matter? Nothing hinges on whether X and Y are one attitude or two…2
The particular trouble on my mind is that social psychologists have actually introduced distinctions that make this individuation important. For example, Eagly & Chaiken (2007) make much of their distinction between intra-attitudinal and inter-attitudinal structure. They list different kinds of features each can have and use this distinction to tell different stories about attitude formation and maintenance. I’m not ready to give a full review of these kinds of cases in the literature, but I think this is a pretty compelling example of where it seems critical to have a good way of individuating attitudes if this theory is to work.
Maybe the deck was stacked against attitudes by my prior beliefs, but I’m not sure I see why they are a useful level of analysis distinct from associations embedded in networks or other, more general, knowledge structures.
What should we use in our science of human behavior instead?
I’m surprised to find myself recommending this, but what philosophers call propositional attitudes — attitudes towards propositions, which are something like what sentences/utterances express — seem pretty appealing. Of course, there has been a great deal of trouble individuating them (in fact, they are one of the kinds of entities Quine was so concerned about). But their individuation troubles aren’t quite so terrible as social psychology’s attitudes: a propositional attitude can involve multiple objects without trouble, and it is the propositional attitudes themselves that can then specify the relationships of these entities to other entities.
I’m far from sure that current theories of propositional attitudes are ready to be dropped in, unmodified, to work in empirical social psychology — Daniel Dennett has even warned philosophers to be wary of promoting propositional attitudes for use in cognitive science, since theory about them is in such a mess. But I do think we have reason to worry about the state of the attitude construct in theorizing by social psychologists.
References
Ajzen, I., & Fishbein, M. (1977). Attitude-Behavior Relations: A Theoretical Analysis and Review of Empirical Research. Psychological Bulletin, 84(5), 8–918.
Allport, G. W. (1935). Attitudes. In C. Murchison (Ed.), Handbook of Social Psychology (Vol. 2, pp. 798–844). Worcester, MA: Clark University Press.
Eagly, A. H., & Chaiken, S. (2007). The Advantages of an Inclusive Definition of Attitude. Social Cognition, 25(5), 582-602.
Fazio, R. H. (2007). Attitudes as object-evaluation associations of varying strength. Social Cognition, 25(5), 603-637.
Fodor, J. A. (1980). Methodological solipsism considered as a research strategy in cognitive psychology. Behavioral and Brain Sciences, 3(1), 63–73.
Kraus, S. J. (1995). Attitudes and the Prediction of Behavior: A Meta-Analysis of the Empirical Literature. Pers Soc Psychol Bull, 21(1), 58-75. doi: 10.1177/0146167295211007.
Petty, R. E., Wegener, D. T., & Fabrigar, L. R. (1997). Attitudes and Attitude Change. Annual Review of Psychology, 48(1), 609-647.
Quine, W.V.O. (1969). Speaking of Objects. Ontological Relativity and Other Essays. New York: Columbia University Press.
Schwarz, N. (2007). Attitude Construction: Evaluation in Context. Social Cognition, 25(5), 638-656.
Zanna, M. P., & Rempel, J. K. (1988). Attitudes: A new look at an old concept. The Social Psychology of Knowledge, 315–334.
- There is still plenty of room for trouble, but this will be common to many representational constructs. For example, there are the familiar problems of what attitudes Louis has towards Superman. Superman is Clark Kent, but it would be odd if this external fact (which Louis doesn’t know) should determine the structure of Louis’ mind. See Fodor (1980). [↩]
- You would likely be in good company, I’m guessing this is a thought that was running through the heads of many of the smart folks in the seminar, “Attitudes and Persuasion”, in which I rambled on about this issue two weeks ago. [↩]
Using social networks for persuasion profiling
BusinessWeek has an exhuberant review of current industry research and product development related to understanding social networks using data from social network sites and other online communication such as email. It includes snippets from people doing very interesting social science research, like Duncan Watts, Cameron Marlow, and danah boyd. So it is worth checking out, even if you’re already familiar with the Facebook Data Team’s recent public reports (“Maintained Relationships”, “Gesundheit!”).
But I actually want to comment not on their comments, but on this section:
In an industry where the majority of ads go unclicked, even a small boost can make a big difference. One San Francisco advertising company, Rapleaf, carried out a friend-based campaign for a credit-card company that wanted to sell bank products to existing customers. Tailoring offers based on friends’ responses helped lift the average click rate from 0.9% to 2.7%. Although 97.3% of the people surfed past the ads, the click rate still tripled.
Rapleaf, which has harvested data from blogs, online forums, and social networks, says it follows the network behavior of 480 million people. It furnishes friendship data to help customers fine-tune their promotions. Its studies indicate borrowers are a better bet if their friends have higher credit ratings. This might mean a home buyer with a middling credit risk score of 550 should be treated as closer to 600 if most of his or her friends are in that range, says Rapleaf CEO Auren Hoffman.
The idea is that since you are more likely to behave like your friends, their behavior can be used to profile you and tailor some marketing to be more likely to result in compliance.
In the Persuasive Technology Lab at Stanford University, BJ Fogg has long emphasized how powerful and worrying personalization based on this kind of “persuasion profile” can be. Imagine that rather than just personalizing screens based on the books you are expected to like (a familiar idea), Amazon selects the kinds of influence strategies used based on a representation of what strategies work best against you: “Dean is a sucker for limited-time offers”, “Foot-in-the-door works really well against Domenico, especially when he is buying a gift.”
In 2006 two of our students, Fred Leach and Schuyler Kaye, created this goofy video illustrating approximately this concept:
My sense is that this kind of personalization is in wide use at places like Amazon, except that their “units of analysis/personalization” are individual tactics (e.g., Gold Box offers), rather than the social influence strategies that can be implemented in many ways and in combination with each other.
What’s interesting about the Rapleaf work described by BusinessWeek is that this enables persuasion profiling even before a service provider or marketer knows anything about you — except that you were referred by or are otherwise connected to a person. This gives them the ability to estimate your persuasion profile by using your social neighborhood, even if you haven’t disclosed this information about your social network.
While there has been some research on individual differences in responses to influence strategies (including when used by computers), as far as I know there isn’t much work on just how much the responses of friends covary. As a tool for influencers online, it doesn’t matter as much whether this variation explained by friends’ responses is also explained by other variables, as long as those variables aren’t available for the influencers to collect. But for us social scientists, it would be interesting to understand the mechanism by which there is this relationship: is it just that friends are likely to be similar in a bunch of ways and these predict our “persuasion profiles”, or are the processes of relationship creation that directly involve these similarities.
This is an exciting and scary direction, and I want to learn more about it.
Being a lobster and using a hammer: “homuncular flexibility” and distal attribution
Jaron Lanier (2006) calls the ability of humans to learn to control virtual bodies that are quite different than our own “homuncular flexibility”. This is, for him, a dangerous idea. The idea is that the familiar mapping of the body represented in the cortical homunculus is only one option – we can flexibly act (and perceive) using quite other mappings, e.g., to virtual bodies. Your body can be tracked, and these movements can be used to control a lobster in virtual reality – just as one experiences (via head-mounted display, haptic feedback, etc.) the virtual space from the perspective of the lobster under your control.
This name and description makes this sound quite like science fiction. In this post, I assimilate homuncular flexibility to the much more general phenomenon of distal attribution (Loomis, 1992; White, 1970). When I have a perceptual experience, I can just as well attribute that experience – and take it as being directed at or about – more proximal or distal phenomena. For example, I can attribute it to my sensory surface, or I can attribute it to a flower in the distance. White (1970) proposed that more distal attribution occurs when the afference (perception) is lawfully related to efference (action) on the proximal side of that distal entity. That is, if my action and perception are lawfully related on “my side” of that entity in the causal tree, then I will make attributions to that entity. Loomis (1992) adds the requirement that this lawful relationship be successfully modeled. This is close, but not quite right, for if I can make distal attributions even in the absence of an actual lawful relationship that I successfully model, my (perhaps inaccurate) modeling of a (perhaps non-existent) lawful relationship will do just fine.
Just as I attribute a sensory experience to a flower and not the air between me and the flower, so the blind man or the skilled hammer-user can attribute a sensory experience to the ground or the nail, rather than the handle of the cane or hammer. On consideration, I think we can see that these phenomena are very much what Lanier is talking about. When I learn to operate (and, not treated by Lanier, 2006, sense) my lobster-body, it is because I have modeled an efference–afference relationship, yielding a kind of transparency. This is a quite familiar sort of experience. It might still be a quite dangerous or exciting idea, but its examples are ubiquitous, not restricted to virtual reality labs.
Lanier paraphrases biologist Jim Boyer as counting this capability as a kind of evolutionary artifact – a spandrel in the jargon of evolutionary theory. But I think a much better just-so evolutionary story can be given: it is this capability – to make distal attributions to the limits of the efference–afference relationships we successfully model – that makes us able to use tools so effectively. At an even more basic and general level, it is this capability that makes it possible for us to communicate meaningfully: our utterances have their meaning in the context of triangulating with other people such that the content of what we are saying is related to the common cause of both of our perceptual experiences (Davidson, 1984).
References
Davidson, D. (1984). Inquiries into Truth and Interpretation. Oxford: Clarendon Press.
Lanier, J. (2006). Homuncular flexibility. Edge.
Loomis, J. M. (1992). Distal attribution and presence. Presence: Teleoperators and Virtual Environments, 1(1), 113-119.
White, B. W. (1970). Perceptual findings with the vision-substitution system. IEEE Transactions on Man-Machine Systems, 11(1), 54-58.