Traits, adaptive systems & dimensionality reduction

Psychologists have posited numerous psychological traits and described causal roles they ought to play in determining human behavior. Most often, the canonical measure of a trait is a questionnaire. Investigators obtain this measure for some people and analyze how their scores predict some outcomes of interest. For example, many people have been interested in how psychological traits affect persuasion processes. Traits like need for cognition (NFC) have been posited and questionnaire items developed to measure them. Among other things, NFC affects how people respond to messages with arguments for varying quality.

How useful are these traits for explanation, prediction, and adaptive interaction? I can’t address all of this here, but I want to sketch an argument for their irrelevance to adaptive interaction — and then offer a tentative rejoinder.

Interactive technologies can tailor their messages to the tastes and susceptibilities of the people interacting with and through them. It might seem that these traits should figure in the statistical models used to make these adaptive selections. After all, some of the possible messages fit for, e.g., coaching a person to meet their exercise goals are more likely to be effective for low NFC people than high NFC people, and vice versa. However, the standard questionnaire measures of NFC cannot often be obtained for most users — certainly not in commerce settings, and even people signing up for a mobile coaching service likely don’t want to answer pages of questions. On the other hand, some Internet and mobile services have other abundant data available about their users, which could perhaps be used to construct an alternative measure of these traits. The trait-based-adaptation recipe is:

  1. obtain the questionnaire measure of the trait for a sample,
  2. predict this measure with data available for many individuals (e.g., log data),
  3. use this model to construct a measure for out-of-sample individuals.

This new measure could then be used to personalize the interactive experience based on this trait, such that if a version performs well (or poorly) for people with a particular score on the trait, then use (or don’t use) that version for people with similar scores.

But why involve the trait at all? Why not just personalize the interactive experience based on the responses of similar others? Since the new measure of the trait is just based on the available behavioral, demographic, and other logged data, one could simply predict responses based on those measure. Put in geometric terms, if the goal is to project the effects of different message onto available log data, why should one project the questionnaire measure of the trait onto the available log data and then project the effects onto this projection? This seems especially unappealing if one doesn’t fully trust the questionnaire measure to be accurate or one can’t be sure about which the set of all the traits that make a (substantial) difference.

I find this argument quite intuitively appealing, and it seems to resonate with others.1 But I think there are some reasons the recipe above could still be appealing.

One way to think about this recipe is as dimensionality reduction guided by theory about psychological traits. Available log data can often be used to construct countless predictors (or “features”, as the machine learning people call them). So one can very quickly get into a situation where the effective number of parameters for a full model predicting the effects of different messages is very large and will make for poor predictions. Nothing — no, not penalized regression, not even a support vector machine — makes this problem go away. Instead, one has to rely on the domain knowledge of the person constructing the predictors (i.e., doing the “feature engineering”) to pick some good ones.

So the tentative rejoinder is this: established psychological traits might often make good dimensions to predict effects of different version of a message, intervention, or experience with. And they may “come with” suggestions about what kinds of log data might serve as measures of them. They would be expected to be reusable across settings. Thus, I think this recipe is nonetheless deserves serious attention.

  1. I owe some clarity on this to some conversations with Mike Nowak and Maurits Kaptein. []

Will the desire for other perspectives trump the “friendly world syndrome”?

Some recent journalism at NPR and The New York Times has addressed some aspects of the “friendly world syndrome” created by personalized media. A theme common to both pieces is that people want to encounter different perspectives and will use available resources to do so. I’m a bit more skeptical.

Here’s Natasha Singer at The New York Times on cascades of memes, idioms, and links through online social networks (e.g., Twitter):

If we keep seeing the same links and catchphrases ricocheting around our social networks, it might mean we are being exposed only to what we want to hear, says Damon Centola, an assistant professor of economic sociology at the Massachusetts Institute of Technology.

“You might say to yourself: ‘I am in a group where I am not getting any views other than the ones I agree with. I’m curious to know what else is out there,’” Professor Centola says.

Consider a new hashtag: diversity.

This is how Singer ends this article in which the central example is “icantdateyou” leading Egypt-related idioms as a trending topic on Twitter. The suggestion here, by Centola and Singer, is that people will notice they are getting a biased perspective of how many people agree with them and what topics people care about — and then will take action to get other perspectives.

Why am I skeptical?

First, I doubt that we really realize the extent to which media — and personalized social media in particular — bias their perception of the frequency of beliefs and events. Even though people know that fiction TV programs (e.g., cop shows) don’t aim to represent reality, heavy TV watchers (on average) substantially overestimate the percent of adult men employed in law enforcement.1 That is, the processes that produce the “friendly world syndrome” function without conscious awareness and, perhaps, even despite it. So people can’t consciously choose to seek out diverse perspectives if they don’t know they are increasingly missing them.

Second, I doubt that people actually want diversity of perspectives all that much. Even if I realize divergent views are missing from my media experience, why would I seek them out? This might be desirable for some people (but not all), and even for those, the desire to encounter people who radically disagree has its limits.

Similar ideas pop up in a NPR All Things Considered segment by Laura Sydell. This short piece (audio, transcript) is part of NPR’s “Cultural Fragmentation” series.2 The segment begins with the worry that offline bubbles are replicated online and quotes me describing how attempts to filter for personal relevance also heighten the bias towards agreement in personalized media.

But much of the piece has actually focuses on how one person — Kyra Gaunt, a professor and musician — is using Twitter to connect and converse with new and different people. Gaunt describes her experience on Twitter as featuring debate, engagement, and “learning about black people even if you’ve never seen one before”. Sydell’s commentary identifies the public nature of Twitter as an important factor in facilitating experiencing diverse perspectives:

But, even though there is a lot of conversation going on among African Americans on Twitter, Professor Gaunt says it’s very different from the closed nature of Facebook because tweets are public.

I think this is true to some degree: much of the content produced by Facebook users is indeed public, but Facebook does not make it as easily searchable or discoverable (e.g., through trending topics). But more importantly, Facebook and Twitter differ in their affordances for conversation. Facebook ties responses to the original post, which means both that the original poster controls who can reply and that everyone who replies is part of the same conversation. Twitter supports replies through the @reply mechanism, so that anyone can reply but the conversation is fragmented, as repliers and consumers often do not see all replies. So, as I’ve described, even if you follow a few people you disagree with on Twitter, you’ll most likely see replies from the other people you follow, who — more often than not — you agree with.

Gaunt’s experience with Twitter is certainly not typical. She has over 3,300 followers and follows over 2,400, so many of her posts will generate replies from people she doesn’t know well but whose replies will appear in her main feed. And — if she looks beyond her main feed to the @Mentions page — she will see the replies from even those she does not follow herself. On the other hand, her followers will likely only see her posts and replies from others they follow.3

Nonetheless, Gaunt’s case is worth considering further, as does Sydell:

SYDELL: Gaunt says she’s made new friends through Twitter.

GAUNT: I’m meeting strangers. I met with two people I had engaged with through Twitter in the past 10 days who I’d never met in real time, in what we say in IRL, in real life. And I met them, and I felt like this is my tribe.

SYDELL: And Gaunt says they weren’t black. But the key word for some observers is tribe. Although there are people like Gaunt who are using social media to reach out, some observers are concerned that she is the exception to the rule, that most of us will be content to stay within our race, class, ethnicity, family or political party.

So Professor Gaunt is likely making connections with people she would not have otherwise. But — it is at least tempting to conclude from “this is my tribe” — they are not people with radically different beliefs and values, even if they have arrived at those beliefs and values from a membership in a different race or class.

  1. Gerbner, G., Gross, L., Morgan, M., & Signorielli, N. (1980). The “Mainstreaming” of America: Violence Profile No. 11. Journal of Communication, 30(3), 10-29. []
  2. I was also interviewed for the NPR segment. []
  3. One nice feature in “new Twitter” — the recently refresh of the Twitter user interface — is that clicking on a tweet will show some of the replies to it in the right column. This may offer an easier way for followers to discover diverse replies to the people they follow. But it is also not particularly usable, as it is often difficult to even trace what a reply is a reply to. []

Homophily and peer influence are messy business

Some social scientists have recently been getting themselves into trouble (and limelight) claiming that they have evidence of direct and indirect “contagion” (peer influence effects) in obesity, happiness, loneliness, etc. Statisticians and methodologists — and even science journalists — have pointed out their troubles. In observational data, peer influence effects are confounded with those of homophily and common external causes. That is, people are similar to other people in their social neighborhood because ties are more likely to form between similar people, and many external events that could cause the outcome are localized in networks (e.g., fast food restaurant opens down the street).

Econometricians1 have worked out the conditions necessary for peer influence effects to be identifiable.2 Very few studies have plausibly satisfied these requirements. But even if an investigator meets these requirements, it is worth remembering that homophily and peer influence are still tricky to think about — let along produce credible quantitative estimates of.

As Andrew Gelman notes, homophily can depend on network structure and information cascades (a kind of peer influence effect) to enable the homophilous relationships to form. Likewise, the success or failure of influence in a relationship can affect that relationship. For example, once I convert you to my way of thinking — let’s say, about climate change, we’ll be better friends. To me, it seems like some of the downstream consequences of our similarity should be attributed to peer influence. If I get fat and so you do, it could be peer influence in many ways: maybe that’s because I convinced you that owning a propane grill is more environmentally friendly (and then we both ended up grilling a lot more red meat). Sounds like peer influence to me. But it’s not that me getting fat caused you to.

Part of the problem here is looking only at peer influence effects in a single behavior or outcome at once. I look forward to the “clear thinking and adequate data” (Manski) that will allow us to better understand these processes in the future. Until then: scientists, please at least be modest in your claims and radical policy recommendations. This is messy business.

  1. They do statistics but speak a different language than big “S” statisticians — kind of like machine learning folks. []
  2. For example, see Manski, C. F. (2000). Economic analysis of social interactions. Journal of Economic Perspectives, 14(3):115–136. Economists call peer influence effects endogenous interactions and contextual interactions. []

Persuasion profiling and genres: Fogg in 2006

Maurits Kaptein and I have recently been thinking a lot about persuasion profiling — estimating and adapting to individual differences in responses to influence strategies based on past behavior and other information. With help from students, we’ve been running experiments and building statistical models that implement persuasion profiling.

My thinking on persuasion profiling is very much in BJ Fogg’s footsteps, since he has been talking about persuasion profiling in courses, lab meetings, and personal discussions since 2004 or earlier.

Just yesterday, I came across this transcript of BJ’s presentation for an FTC hearing in 2006. I was struck at how much it anticipates some of what Maurits and I have written recently (more on this later). I’m sure I watched the draft video of the presentation back then and it’s influenced me, even if I forgot some of the details.

Here is the relevant excerpt from BJ’s comments for the FTC:

Persuasion profiling means that each one of us has a different set of persuasion strategies that affect us. Just like we like different types of food or are vulnerable to giving in to different types of food on a diet, we are vulnerable to different types of persuasion strategies.

On the food example, I love old-fashioned popcorn, and if I go to a party and somebody has old-fashioned popcorn, I will probably break down and eat it. On the persuasion side of things, I know I’m vulnerable to trying new things, to challenges and to anything that gets measured. If that’s proposed to me, I’m going to be vulnerable and I’m going to give it a shot.

Whenever we go to a Web site and use an interactive system, it is likely they will be capturing what persuasion strategies work on us and will be using those when we use the service again. The mapping out of what makes me tick, what motivates me can also be bought or sold, just like a credit report.

So imagine I’m going in to buy a new car and the person selling me the car downloads my credit report but also buys my persuasion profile. I may or may not know about this. Imagine if persuasion profiles are available on political campaigns so that when I visit a Web site, the system knows it is B.J. Fogg, and it changes [its] approach based on my vulnerabilities when it comes to persuasion.

Persuasive technology will touch our lives anywhere that we access digital products or services, in the car, in our living room, on the Web, through our mobile phones and so on. Persuasive technology will be all around us, and unlike other media types, where you have 30-second commercial or a magazine ad, you have genres you can understand, when it comes to computer-based persuasion, it is so flexible that it won’t have genre boundaries. It will come to us in the ordinary course of our lives, as we are working on a Web site, as we are editing a document, as we are driving a car. There won’t be clear markers about when you are being persuaded and when you are not.

This last paragraph is about the “genrelessness” of many persuasive technologies. This isn’t directly on the topic of persuasion profiling, but I see it as critically relevant. Persuasion profiling is likely to be most effective when invisible and undisclosed to users. From this and the lack of genre-based flags for persuasive technology it follows that we will frequently be “persuasion profiled” without knowing it.

No entity without identity: individuating attitudes in social psychology

Social psychologists like to write about attitudes. In fact, following Allport (1935), many of them have happily commented that the attitude is the most central and indispensable construct in social psychology (e.g., Petty, Wegener, Fabrigar, 1997). Here is a standard definition of an attitude: an attitude is

a psychological tendency that is expressed by evaluating a particular entity with some degree of favor or disfavor. (Eagly & Chaiken, 2007, p. 598)

A somewhat more specific view has it that attitudes are

associations between a given object and a given summary evaluation of the object — associations that can vary in strength and, hence, in their accessibility from memory. (Fazio, 2007, p. 608)

Attitudes are also supposed to be important for predicting behavior, though the attitude–behavior link is the subject of a great deal of controversy, which I can’t fully treat here. An extreme, design-oriented view is expressed by a B.F. Skinner-channeling B.J. Fogg:

Don’t waste time mapping attitudes to behaviors. It’s a tough, useless problem. Blackbox attitudes. Focus on behavior change & metrics.

While Fogg isn’t representative of mainstream, contemporary social psychology, similarly skeptical thoughts are expressed by investigators like Schwartz (2007). On the other hand, one common view of the attitude–behavior link is that it is quite strong (Kraus, 1997), but that (a) many research methods fail to measure attitudes and behaviors with regard to the same entities (Ajzen & Fishbein, 1977) and (b) this link is an important empirical subject, not built into the attitude construct by definition (Fazio, 2007; Zanna & Rempel, 1988).

I’ll set aside for now just how useful attitudes are for predicting behavior. But what should we make of this construct? That is, should we keep it around? Do we expect something like social psychology’s attitudes to be part of a mature science of human behavior?

Maybe I’m a sucker for a good slogan, but when I read psychologists’ on attitudes, I think of Quine’s slogan: no entity without identity. That is, we shouldn’t posit objects that don’t have identity conditions — the conditions under which we say that X and Y are the same object.

This slogan, followed strictly in everyday life, can get tricky: a restaurant changes owners and name — is it the same restaurant? But it is pretty compelling when it comes to the entities we use in science. Of course, philosophers have debated this slogan — and many particular proposed cases of posited entities lacking identity conditions (e.g., entities in quantum physics) — so I’ll leave it that lacking identity conditions might vary in how much trouble it causes for a theory that uses such entities.

What I do want to comment on is how strikingly social psychology’s attitudes lack good identity conditions — and thus have no good way of being individuated. While we might think this doesn’t cause much trouble in this case (as I just noted), I actually think it creates a whole family of pseudo-problems that psychologists spend their time on and build theories around.

First, evidence that there is trouble in individuating attitudes: As is clear from the definition of an attitude provided above, attitudes are supposed to be individuated by their object:

This evaluative responding is directed to some entity or thing that is its object—that is, we may evaluate a person (George W. Bush), a city (Chicago), an ideology (conservatism), and a myriad of other entities. In the language of social psychology, an entity that is evaluated is known as an attitude object. Anything that is discriminable or held in mind, sometimes below the level of conscious awareness, can be evaluated and therefore can function as an attitude object. Attitude objects may be abstract (e.g., liberalism, religious fiindamentalism) or concrete (e.g., the White House, my green raincoat) as well as individual (e.g., Condoleezza Rice, my sister-in-law) or collective (e.g., undocumented workers, European nations). (Eagly & Chaiken, 2007, p. 584)

So, for example,  I can have an attitude towards Obama. This attitude can then have internal structure, such that there are multiple evaluations involved (e.g., implicit and explicit). This seems pretty straightforward: it is at least somewhat clear when some cognitive structures share the Obama as object.1

But trouble is not far around the corner. Much discussion of attitudes involves attitudes objects that are abstract objects — like sets or classes of objects– embedded in a whole set of relationships. For example, I might have attitudes towards snakes, Blacks, or strawberry ice cream. And there isn’t any obvious way that the canonical class by which attitudes are to be individuated gets picked out. A person has evaluative responses to strawberry ice cream, Ben & Jerry’s brand ice cream, ice cream in general, the larger class of such foods (including frozen yogurt, gelato, “soft serve”), foods that cool one down when eaten, etc.

This doesn’t just work with ice cream. (Obama instantiates many properties and is a member of many relevant classes.)

At this point, you might be thinking, how does all this matter? Nothing hinges on whether X and Y are one attitude or two…2

The particular trouble on my mind is that social psychologists have actually introduced distinctions that make this individuation important. For example, Eagly & Chaiken (2007) make much of their distinction between intra-attitudinal and inter-attitudinal structure. They list different kinds of features each can have and use this distinction to tell different stories about attitude formation and maintenance. I’m not ready to give a full review of these kinds of cases in the literature, but I think this is a pretty compelling example of where it seems critical to have a good way of individuating attitudes if this theory is to work.

Maybe the deck was stacked against attitudes by my prior beliefs, but I’m not sure I see why they are a useful level of analysis distinct from associations embedded in networks or other, more general, knowledge structures.

What should we use in our science of human behavior instead?

I’m surprised to find myself recommending this, but what philosophers call propositional attitudes — attitudes towards propositions, which are something like what sentences/utterances express — seem pretty appealing. Of course, there has been a great deal of trouble individuating them (in fact, they are one of the kinds of entities Quine was so concerned about). But their individuation troubles aren’t quite so terrible as social psychology’s attitudes: a propositional attitude can involve multiple objects without trouble, and it is the propositional attitudes themselves that can then specify the relationships of these entities to other entities.

I’m far from sure that current theories of propositional attitudes are ready to be dropped in, unmodified, to work in empirical social psychology — Daniel Dennett has even warned philosophers to be wary of promoting propositional attitudes for use in cognitive science, since theory about them is in such a mess. But I do think we have reason to worry about the state of the attitude construct in theorizing by social psychologists.

References

Ajzen, I., & Fishbein, M. (1977). Attitude-Behavior Relations: A Theoretical Analysis and Review of Empirical Research. Psychological Bulletin, 84(5), 8–918.

Allport, G. W. (1935). Attitudes. In C. Murchison (Ed.), Handbook of Social Psychology (Vol. 2, pp. 798–844). Worcester, MA: Clark University Press.

Eagly, A. H., & Chaiken, S. (2007). The Advantages of an Inclusive Definition of Attitude. Social Cognition, 25(5), 582-602.

Fazio, R. H. (2007). Attitudes as object-evaluation associations of varying strength. Social Cognition, 25(5), 603-637.

Fodor, J. A. (1980). Methodological solipsism considered as a research strategy in cognitive psychology. Behavioral and Brain Sciences, 3(1), 63–73.

Kraus, S. J. (1995). Attitudes and the Prediction of Behavior: A Meta-Analysis of the Empirical Literature. Pers Soc Psychol Bull, 21(1), 58-75. doi: 10.1177/0146167295211007.

Petty, R. E., Wegener, D. T., & Fabrigar, L. R. (1997). Attitudes and Attitude Change. Annual Review of Psychology, 48(1), 609-647.

Quine, W.V.O. (1969). Speaking of Objects. Ontological Relativity and Other Essays. New York: Columbia University Press.

Schwarz, N. (2007). Attitude Construction: Evaluation in Context. Social Cognition, 25(5), 638-656.

Zanna, M. P., & Rempel, J. K. (1988). Attitudes: A new look at an old concept. The Social Psychology of Knowledge, 315–334.

  1. There is still plenty of room for trouble, but this will be common to many representational constructs. For example, there are the familiar problems of what attitudes Louis has towards Superman. Superman is Clark Kent, but it would be odd if this external fact (which Louis doesn’t know) should determine the structure of Louis’ mind. See Fodor (1980). []
  2. You would likely be in good company, I’m guessing this is a thought that was running through the heads of many of the smart folks in the seminar, “Attitudes and Persuasion”, in which I rambled on about this issue two weeks ago. []