Ready-to-hand Dean Eckles blogs on people and technology

Right from the start of today’s Media Multitasking Workshop¹,  it’s clear that one big issue is just what people are talking about when they talk about multitasking. In this post, I want to highlight the relationship between defining different kinds of multitasking and people’s representations of the hierarchical structure of action.

It is helpful to start with a contrast between two kinds of cases.

Distributing attention towards a single goal

In the first, there is a single task or goal that involves dividing one’s attention, with the targets of attention somehow related, but of course somewhat independent. Patricia Greenfield used Pac-Man as an example: each of the ghosts must be attended to (in addition to Pac-Man himself), and each is moving independently, but each is related to the same larger goal.

Distributing attention among different goals

In the second kind of case, there are two completely unrelated tasks that divide attention, as in playing a game (e.g., solitaire) while also attending to a speech (e.g., in person, on TV). Anthony Wagner noted that in Greenfield’s listing of the benefits and costs of media multitasking, most of the listed benefits applied to the former case, while the costs she listed applied to the later. So keeping these different senses of multitasking straight is important.

Complications

But the conclusion should not be to think that this is a clear and stable distinction that slices multitasking phenomena in just the right way. Consider one ways of putting this distinction: the primary and secondary task can either be directed at the same goal or directed at different goals (or tasks). Let’s dig into this a bit more.²

Byron Reeves pointed out that sometimes “the IMing is about the game.” So we could distinguish whether the goal of the IMing is the same as the goal of the in-game task(s). But this making this kind of distinction requires identity conditions for goals or tasks that enable this distinction. As Ulrich Mayr commented, goals can be at many different levels, so in order to use goal identity as the criterion, one has to select a level in the hierarchy of goals.

Action identities and multitasking

We can think about this hierarchy of goals as the network of identities for an action that are connected with the “by” relation: one does one thing by doing (several) other things. If these goals are the goals of the person as they represent them, then this is the established approach taken by action identification theory (Vallacher & Wegner, 1987) — and this could be valuable lens for thinking about this. Action identification theory claims that people can report an action identity for what they are doing, and that this identity is the “prepotent identity”. This prepotent identity is generally the highest level identity under which the action is maintainable. This means that the prepotent identity is at least somewhat problematic if used to make this distinction between these two types of multitasking because then the distinction would be dependent on, e.g., how automatic or functionally transparent the behaviors involved are.

For example, if I am driving a car and everything is going well, I may represent the action as “seeing my friend Dave”. I may also represent my simultaneous, coordinating phone call with Dave under this same identity. But if driving becomes more difficult, then my prepotent identity will decrease in level in order to maintain the action. Then these two tasks would not share the prepotent action identity.

Prepotent action identities (i.e. the goal of the behavior as represented by the person in the moment) do not work to make this distinction for all uses. But I think that it actually does help makes some good distinctions about the experience of multitasking, especially if we examine change in action identities over time.

To return to case of media multitasking, consider the headline ticker on 24-hour news television. The headline ticker can be more or less related to what the talking heads are going on about. This could be evaluated as a semantic, topical relationship. But considered as a relationship of goals — and thus action identities — we can see that perhaps sometimes the goals coincide even when the content is quite different. For example, my goal may simply to be “get the latest news”, and I may be able to actually maintain this action — consuming both the headline ticker and the talking heads’ statements — under this high level identity. This is an importantly different case then if I don’t actually maintain the action at the level, but instead must descend to — and switch between — two (or more) lower level identities that are associated the two streams of content.

References

Vallacher, R. R., & Wegner, D. M. (1987). What do people think they’re doing? Action identification and human behavior. Psychological Review, 94(1), 3-15. 

1. The full name is the “Seminar on the impacts of media multitasking on children’s learning and development”. [↩]
2. As I was writing this, the topic re-emerged in the workshop discussion. I made some comments, but I think I may not have made myself clear to everyone. Hopefully this post is a bit of an improvement. [↩]

This post revisits some thoughts I’ve shared an earlier version of here. In articles over the past few years, John Bargh and his colleagues claim that cognitive psychology has operated with a narrow definition of unconscious processing that has led investigators to describe it as “dumb” and “limited”. Bargh prefers a definition of unconscious processing more popular in social psychology – a definition that allows him to claim a much broader, more pervasive, and “smarter” role for unconscious processing in our everyday lives. In particular, I summarize the two definitions used in Bargh’s argument (Bargh & Morsella 2008, p. 1) as the following:

Unconscious processing_cog is the processing of stimuli of which one is unaware.

Unconscious processing_soc is processing of which one is unaware, whether or not one is aware of the stimuli.

A helpful characterization of unconscious processing_soc is the question: “To what extent are people aware of and able to report on the true causes of their behavior?” (Nisbett & Wilson 1977). We can read this project as addressing first-person authority about causal trees that link external events to observable behavior.

What does it mean for the processing of a stimulus to be below conscious awareness? In particular, we can wonder, what is that one is aware of when one is aware of a mental process of one’s own? While determining whether unconscious processing_cog is going on requires specifying a stimulus to which the question is relative, unconscious processing_soc requires specifying a process to which the question is relative. There may well be troubles with specifying the stimulus, but there seem to be bigger questions about specifying the process.

There are many interesting and complex ways to identify a process for consideration or study. Perhaps the simplest kind of variation to consider is just differences of detail. First, consider the difference between knowing some general law about mental processing and knowing that one has in fact engaging in processing meeting the conditions of application for the law.

Second, consider the difference between knowing that one is processing some stimulus and that a various long list of things have a causal role (cf. the generic observation that causal chains are hard to come by, but causal trees are all around us) and knowing the specific causal role each has and the truth of various counterfactuals for situations in which those causes were absent.

Third, consider the difference between knowing that some kind of processing is going on that will accomplish an end (something like knowing the normative functional or teleological specification of the process, cf. Millikan 1990 on rule-following and biology) and the details of the implementation of that process in the brain (do you know the threshold for firing on that neuron?). We can observe that an extensionally identical process can always be considered under different descriptions; and any process that one is aware of can be decomposed into a description of extensionally identical sub-processes, of which one is unaware.

A bit trickier are variations in descriptions of processes that do not have law-like relationships between each other. For example, there are good arguments for why folk psychological descriptions of processes (e.g. I saw that A, so I believed that B, and, because I desired that C, I told him that D) are not reducible to descriptions of processes in physical or biological terms about the person.¹

We are still left with the question: What does it mean to be unaware of the imminent consequences of processing a stimulus?

References

1. There are likely more examples of this than commonly thought, but the one I am thinking of is the most famous: the weak supervenience of mental (intentional) states on physical states without there being psychophysical laws linking the two (Davidson 1963, Anscombe 1969, Putnam 1975). [↩]

At the Social Media Workshop, Katarina Segerståhl presented her on-going work on what she has termed extended information services or distributed user experiences — human-computer interactions that span multiple and heterogeneous devices (Segerståhl & Oinas-Kukkonen 2007). As a central example, she studies a persuasive technology service for planning, logging, reviewing, and motivating exercise: these parts of the experience are distributed across the user’s PC, mobile phone, and heart rate monitor.

In one interesting observation, Segerståhl notes that the specific user interfaces on one device can be helpful mental images even when a different device is in use: participants reported picturing their workout plan as it appeared on their laptop and using it to guide their actions during their workout, during which the obvious, physically present interface with the service was the heart rate monitor, not the earlier planning visualization. Her second focus is how to make these user experiences coherent, with clear practical applications in usability and user experience design (e.g., how can designers make the interfaces both appropriately consistent and differentiated?).

In this post, I want to connect this very interesting and relevant work with some other research at the historical and theoretical center of persuasive technology: source orientation in human-computer interaction. First, I’ll relate source orientation to the history and intellectual context of persuasive technology. Then I’ll consider how multi-device and multi-context interactions complicate source orientation.

Source orientation, social responses, and persuasive technology

As an incoming Ph.D. student at Stanford University, B.J. Fogg already had the goal of improving generalizable knowledge about how interactive technologies can change attitudes and behaviors by design. His previous graduate studies in rhetoric and literary criticism had given him understanding of one family of academic approaches to persuasion. And in running a newspaper and consulting on many document design (Schriver 1997) projects, the challenges and opportunities of designing for persuasion were to him clearly both practical and intellectually exciting.

The ongoing research of Profs. Clifford Nass and Byron Reeves attracted Fogg to Stanford to investigate just this. Nass and Reeves were studying people’s mindless social responses to information and communication technologies. Cliff Nass’s research program — called Computers as (or are) Social Actors (CASA) — was obviously relevant: if people treat computers socially, this “opens the door for computers to apply [...] social influence” to change attitudes and behaviors (Fogg 2002, p. 90). While clearly working within this program, Fogg focused on showing behavioral evidence of these responses (e.g., Fogg & Nass 1997): both because of the reliability of these measures and the standing of behavior change as a goal of practitioners.

Source orientation is central to the CASA research program — and the larger program Nass shared with Reeves. Underlying people’s mindless social responses to communication technologies is the fact that they often orient towards a proximal source rather than a distal one — even when under reflective consideration this does not make sense: people treat the box in front of them (a computer) as the source of information, rather than a (spatially and temporally) distant programmer or content creator. That is, their source orientation may not match the most relevant common cause of the the information. This means that features of the proximal source unduly influence e.g. the credibility of information presented or the effectiveness of attempts at behavior change.

For example, people will reciprocate with a particular computer if it is helpful, but not the same model running the same program right next to it (Fogg & Nass 1997, Moon 2000). Rather than orienting to the more distal program (or programmer), they orient to the box.¹

Multiple devices, Internet services, and unstable context

These source orientation effects have been repeatedly demonstrated by controlled laboratory experiments (for reviews, see Nass & Moon 2000, Sundar & Nass 2000), but this research has largely focused on interactions that do not involve multiple devices, Internet services, or use in changing contexts. How is source orientation different in human-computer interactions that have these features?

This question is of increasing practical importance because these interactions now make up a large part of our interactions with computers. If we want to describe, predict, and design for how people use computers everyday — checking their Facebook feed on their laptop and mobile phone, installing Google Desktop Search and dialing into Google 411, or taking photos with their Nokia phone and uploading them to Nokia’s Ovi Share — then we should test, extend, and/or modify our understanding of source orientation. So this topic matters for major corporations and their closely guarded brands.

So why should we expect that multiple devices, Internet services, and changing contexts of use will matter so much for source orientation? After having explained the theory and evidence above, this may already be somewhat clear, so I offer some suggestive questions.

1. If much of the experience (e.g. brand, visual style, on-screen agent) is consistent across these changes, how much will the effects of characteristics of the proximal source — the devices and contexts — be reduced?
2. What happens when the proximal device could be mindfully treated as a source (e.g., it makes its own contribution to the interaction), but so does a distance source (e.g., a server)? This could be especially interesting with different branding combination between the two (e.g., the device and service are both from Apple, or the device is from HTC and service is from Google).
3. What if the visual style or manifestation of the distal source varies substantially with the device used, perhaps taking on a style consistent with the device? This can already happen with SMS-based services, mobile Java applications, and voice agents that help you access distant media and services.

References

1. This actually is subject to a good deal of cross-cultural variation. Similar experiments with Japanese — rather than American — participants show reciprocity to groups of computers, rather than just individuals (Katagiri et al.) [↩]

Over at Captology Notebook, the blog of the Stanford Persuasive Technology Lab, Enrique Allen considers features of Facebook that influence users to update their status. Among other things, he highlights how Facebook lowers barriers to updating by giving users a clear sense of something they can right (”What are you doing right now?”).

I’d like to add another part of the interface for consideration: the box in the left box of the home page that shows your current status update with the most recent updates of your friends.

This visual association of my status and the most recent status updates of my friends seems to do at least a couple things:

Influencing the frequency of updates. In this example, my status was updated a few days ago. On the other hand, the status updates from my friends were each updated under an hour ago. This juxtaposes my stale status with the fresh updates of my peers. This can prompt comparison between their frequency of updates and mine, encouraging me to update.

The choice of the most recent updates by my Facebook friends amplifies this effect. Through automatic application of the availability heuristic, this can make me overestimate how recently my friends have updated their status (and thus the frequency of status updates). For example, the Facebook friend who updated their status three minutes ago might have not updated to three weeks prior. Or many of my Facebook friends may not frequently update their status messages, but I only see (and thus have most available to mind) the most recent. This is social influence through enabling and encouraging biased social comparison with — in a sense — an imagined group of peers modeled on those with the most recent performances of the target behavior (i.e., updating status).

Influencing the content of updates. In his original post, Enrique mentions how Facebook ensures that users have the ability to update their status by giving them a question that they can answer. Similarly, this box also gives users examples from their peers to draw on.

Of course, this can all run up against trouble. If I have few Facebook friends, none of them update their status much, or those who do update their status are not well liked by me, this comparison may fail to achieve increased updates.

Consider this interface in comparison to one that either

• showed recent status updates by your closest Facebook friends, or
• showed recent status updates and the associated average period for updates of your Facebook friends that most frequently update their status.

[Update: While the screenshot above is from the "new version" of Facebook, since I captured it they have apparently removed other people's updates from this box on the home page, as Sasha pointed out in the comments. I'm not sure why they would do this, but here are couple ideas:

• make lower items in this sidebar (right column) more visable on the home page -- including the ad there
• emphasize the filter buttons at the top of the news feed (left column) as the means to seeing status updates.

Given the analysis in the original post, we can consider whether this change is worth it: does this decrease status updates? I wonder if Facebook did a A-B test of this: my money would be on this significantly reducing status updates from the home page, especially from users with friends who do update their status.]

John Bargh, Professor of Psychology at Yale, and his ACME (Automaticity in Cognition, Motivation, and Emotion) Lab are doing very exciting work. I had read some articles by Bargh some time ago (e.g. Bargh & McKenna 2004) and encountered his work in the context of debates about how objects can automatically activate attitudes that apply to them. But it hasn’t been until recently (following a discussion with James Breckenridge) that I’ve begun to really engage with the larger body of research Bargh and his collaborators have produced — and the interesting reflections and arguments found in the reviews of this and related work that he and his collaborators have written.

I expect I’ll be writing more about this work, but in this and some follow-up posts I want to just say a little bit about the general character of the research and, more specifically, how this work engages with and employs definitions of ‘unconscious’ and ‘unconscious processing‘.

Bargh & Morsella (2008, in press, page numbers are to this version) highlights how cognitive psychology and social psychology have operated with different definitions and different emphasis in investigating what they call “unconscious”. For cognitive psychology, “subliminal information processing – [...] extracting meaning from stimuli of which one is not consciously aware” – has been paradigmatic of the unconscious (p. 1). That is, its study of unconscious processing is the study of the processing of stimuli of which one is unaware. On the other hand, for mainstream social psychology research, including work with priming, “the traditional focus has been on mental processes of which the individual is unaware, not on stimuli of which one is unaware” (Ibid.).

This is a striking difference that, as Bargh & Morsella illustrate, has consequences for how “dumb” or “smart” and “limited” or “pervasive” unconscious processing is. If unconscious processing is limited to processing of subliminal stimuli, then it doesn’t have much to go on. But the social psychology definition — the liberal, process-awareness definition — allows us to call a lot more things unconscious processing.

I recognize shortcomings with the cognitive psychology definition — the narrow, stimulus-awareness definition. And Bargh and Morsella’s statement of the process-awareness definition does enable them to say some striking things (e.g. about automatic activation of motivations).

But I also wonder whether this redefined term can bear much theoretical weight. Specifically, I have two concerns:

1. this definition makes what is unconscious depend on each person’s knowledge of the causes of their actions — and this can get tricky in unintuitive and highly individual ways
2. this definition seems to count on having good identity conditions for the kinds of objects to which ‘unconscious’ is supposed to apply (e.g. events, processes), but identity conditions (which are often hard to come by in general) are tricky for this domain in particular.

These are familiar problems in philosophy of mind, and they deserve consideration when designing theoretically useful definitions of unconscious processing. I aim to take up each of these issues in more detail in another post.

Bargh, J.A., & Morsella, E. (2008, in press). The unconscious mind. Perspectives on Psychological Science.

Bargh, J.A., & McKenna, K.Y.A. (2004). The Internet and social life. Annual review of psychology, 55, 573-590.