Producing, consuming, annotating (Social Mobile Media Workshop, Stanford University)
Today I’m attending the Social Mobile Media Workshop at Stanford University. It’s organized by researchers from Stanford’s HStar, Tampere University of Technology, and the Naval Postgraduate School. What follows is some still jagged thoughts that were prompted by the presentation this morning, rather than a straightforward account of the presentations.1
A big theme of the workshop this morning has been transitions among production and consumption — and the critical role of annotations and context-awareness in enabling many of the user experiences discussed. In many ways, this workshop took me back to thinking about mobile media sharing, which was at the center of a good deal of my previous work. At Yahoo! Research Berkeley we were informed by Marc Davis’s vision of enabling “the billions of daily media consumers to become daily media producers.” With ZoneTag we used context-awareness, sociality, and simplicity to influence people to create, annotate, and share photos from their mobile phones (Ahern et al. 2006, 2007).
Enabling and encouraging these behaviors (for all media types) remains a major goal for designers of participatory media; and this was explicit at several points throughout the workshop (e.g., in Teppo Raisanen’s broad presentation on persuasive technology). This morning there was discussion about the technical requirements for consuming, capturing, and sending media. Cases that traditionally seem to strictly structure and separate production and consumption may be (1) in need of revision and increased flexibility or (2) actually already involve production and consumption together through existing tools. Media production to be part of a two-way communication, it must be consumed, whether by peers or the traditional producers.
As an example of the first case, Sarah Lewis (Stanford) highlighted the importance of making distance learning experiences reciprocal, rather than enforcing an asymmetry in what media types can be shared by different participants. In a past distance learning situation focused on the African ecosystem, it was frustrating that video was only shared from the participants at Stanford to participants at African colleges — leaving the latter to respond only via text. A prototype system, Mobltz, she and her colleagues have built is designed to change this, supporting the creation of channels of media from multiple people (which also reminded me of Kyte.tv).
As an example of the second case, Timo Koskinenen (Nokia) presented a trial of mobile media capture tools for professional journalists. In this case the work flow of what is, in the end, a media production practice, involves also consumption in the form of review of one’s own materials and other journalists, as they edit, consider what new media to capture.
Throughout the sessions themselves and conversations with participants during breaks and lunch, having good annotations continued to come up as a requirement for many of the services discussed. While I think our ZoneTag work (and the free suggested tags Web service API it provides) made a good contribution in this area, as has a wide array of other work (e.g., von Ahn & Dabbish 2004, licensed in Google Image Labeler), there is still a lot of progress to make, especially in bringing this work to market and making it something that further services can build on.
References
Ahern, S., Davis, M., Eckles, D., King, S., Naaman, M., Nair, R., et al. (2006). ZoneTag: Designing Context-Aware Mobile Media Capture. In Adjunct Proc. Ubicomp (pp. 357-366).
Ahern, S., Eckles, D., Good, N. S., King, S., Naaman, M., & Nair, R. (2007). Over-exposed?: privacy patterns and considerations in online and mobile photo sharing. In Proc. CHI 2007 (pp. 357-366). ACM Press.
Ahn, L. V., & Dabbish, L. (2004). Labeling images with a computer game. In Proc. CHI 2004 (pp. 319-326).
- Blogging something at this level of roughness is still new for me… [↩]
Update your Facebook status: social comparison and the availability heuristic
[Update: This post uses an older Facebook UI as an example. Also see more recent posts on activity streams and the availability heuristic.]
Over at Captology Notebook, the blog of the Stanford Persuasive Technology Lab, Enrique Allen considers features of Facebook that influence users to update their status. Among other things, he highlights how Facebook lowers barriers to updating by giving users a clear sense of something they can right (“What are you doing right now?”).
I’d like to add another part of the interface for consideration: the box in the left box of the home page that shows your current status update with the most recent updates of your friends.
This visual association of my status and the most recent status updates of my friends seems to do at least a couple things:
Influencing the frequency of updates. In this example, my status was updated a few days ago. On the other hand, the status updates from my friends were each updated under an hour ago. This juxtaposes my stale status with the fresh updates of my peers. This can prompt comparison between their frequency of updates and mine, encouraging me to update.
The choice of the most recent updates by my Facebook friends amplifies this effect. Through automatic application of the availability heuristic, this can make me overestimate how recently my friends have updated their status (and thus the frequency of status updates). For example, the Facebook friend who updated their status three minutes ago might have not updated to three weeks prior. Or many of my Facebook friends may not frequently update their status messages, but I only see (and thus have most available to mind) the most recent. This is social influence through enabling and encouraging biased social comparison with — in a sense — an imagined group of peers modeled on those with the most recent performances of the target behavior (i.e., updating status).
Influencing the content of updates. In his original post, Enrique mentions how Facebook ensures that users have the ability to update their status by giving them a question that they can answer. Similarly, this box also gives users examples from their peers to draw on.
Of course, this can all run up against trouble. If I have few Facebook friends, none of them update their status much, or those who do update their status are not well liked by me, this comparison may fail to achieve increased updates.
Consider this interface in comparison to one that either
- showed recent status updates by your closest Facebook friends, or
- showed recent status updates and the associated average period for updates of your Facebook friends that most frequently update their status.
[Update: While the screenshot above is from the “new version” of Facebook, since I captured it they have apparently removed other people’s updates from this box on the home page, as Sasha pointed out in the comments. I’m not sure why they would do this, but here are couple ideas:
- make lower items in this sidebar (right column) more visable on the home page — including the ad there
- emphasize the filter buttons at the top of the news feed (left column) as the means to seeing status updates.
Given the analysis in the original post, we can consider whether this change is worth it: does this decrease status updates? I wonder if Facebook did a A-B test of this: my money would be on this significantly reducing status updates from the home page, especially from users with friends who do update their status.]
Reprioritizing human intelligence tasks for low latency and high throughput on Mechanical Turk
Amazon Mechanical Turk is a platform and market for human intelligence tasks (HITs) that are submitted by requesters and completed by workers (or “turkers”). Each HIT is associated with a payment, often a few cents. This post covers some basics of Mechanical Turk and shows its lack of designed-in support for dynamic reprioritization is problematic for some uses. I also mention some other factors that influence latency and throughput.
With mTurk one can create a HIT that asks someone to rate some search results for a query, evaluate the credibility of a Wikipedia article, draw a sheep facing left, enter names for a provided color, annotate a photo of a person with pose information, or create a storyboard illustrating a new product idea. So Mechanical Turk can be used in many ways for basic research, building a training set for machine learning, or actually enabling a (perhaps prototype) service in use through a kind of Wizard-of-Oz approach. Additionally, I’ve used mTurk to code images captured by participants in a lab experiment (more on this in another post or article).
When creating HITs, a requester can specify a QuestionForm (QF) (e.g., via command line tools or an SDK) that is then presented to the worker by Amazon. This can include images, free text answers, multiple choice, etc. Additionally one can embed Flash or Java objects in it. But the easiest way of creating HITs is to use a QF and not create a Java or Flash application of one’s own. This is especially true for HITs that are handled well by the basic question form. The other option is to create an ExternalQuestion (EQ), which is hosted on one’s own server and is displayed in an iFrame. This provides greater freedom but requires additional development and it is you that must host the page (though you can do so through Amazon’s S3). QF HITs (without embeds) also offer a familiar interface to workers (though it is possible to create a more efficient, custom interface by, e.g., making all the targets larger). So when possible, it is often preferable to use a QF rather than an EQ.
For some of the uses of mTurk for powering a service, it can be important to minimize latency for specific HITs1, including prioritizing particular new HITs over previously created HITs. For example, after some HIT has not been completed for a specific period after creation, it may still be important to complete it, but when it is completed may become less important. This can happen easily if the value of a HIT being completed has a sharp drop off after some time.
This should be done while maintaining high throughput; that is, you don’t want to reduce the rate at which your HITs are completed. When there are more HITs of the same type, workers can check a box to immediately start the next HIT of the same type when they submit the current one (see screenshot). Workers will often complete many HITs of the same type in a row. So throughput can drop substantially if any workers run out of HITs of the same type at any point: they may switch to another HIT type, or if they do your HITs once more appear, then there will be a delay. As we’ll see, these two requirements don’t seem to be well met by the platform — or at least certain uses of it.
Mechanical Turk does not provide a mechanism for prioritizing HITs of the same type, so without deleting all but particular high-priority HITs of that type, there is not a way to ensure that some particular HIT gets done before the rest. And deleting the other HITs would hurt throughput and increase latency for any new high-priority HITs added in the near future (since workers won’t simply start these once they finish their previous HITs).
EQ HITs allow one to avoid this problem. Unlike with QF HITs (without Flash and Java embeds), one does not have to specify the full content of the HIT in advance. When a worker accepts an EQ HIT, you can dynamically serve up the HIT you want to depending on changing priorities. But this means that you can’t take advantage of, e.g., the simplicity of creating and managing data from QF HITs. So though there are ways of coping, adding dynamic reprioritization to Mechanical Turk would be a boon for time-sensitive uses.
There are, of course, other factors that influence latency and throughput on mTurk when (EQ) HITs are reprioritized. Here are a few:
- HIT and sub-tasks duration. How long does it take for workers to complete a HIT, which may be composed of multiple sub-tasks? A worker cannot be assigned a new HIT until they complete (or reject) the previous one. This can be somewhat avoided by creating longer HITs that are subdivided into dynamically selected sub-tasks. This can be done with an EQ HIT or an embedded Flash or Java application in a QF HIT. But the sub-task duration is always a limiting factor, unless one is willing to force abortion of the current sub-task, replacing it will still in progress (with an EQ, Flash, or Java).
- Available workers. How many workers are logged into mTurk and completing task? How many are currently switching HIT types? This can vary with the time of day.
- Appeal of your HITs. How much do workers like your HITs — are they fun? How much do you pay for how much you ask? How many of their completed assignments do you approve?
- Reliability. How accurate or precise must your results be? How many workers do you need to complete a HIT before you have reliable results? Do other workers need to complete meta-HITs before the data can be used?
- I use the term HIT somewhat loosely in this article. There are at least three uses that each differ in their identity conditions. (1) There are HITs considered as human intelligence tasks, and thus divided as we divide tasks; this means that a HIT in another sense can be composed of multiple HITs in this sense (tasks or sub-tasks). (2) There are HITs in Amazon’s technical sense of the term: a HIT is something that has the same HIT ID and therefore has the same specification. In QF HITs without embeds, this means all instances (assignments) of a HIT are the same in content; but in EQ HIT this is not necessarily true, since the content can be determined when assigned. (3) Finally, there is what Amazon calls assignments, specific instances of a HITs that are only completed once. [↩]
Naming this blog “ready-to-hand”: Heidegger, Husserl, folk psychology, and HCI
The name of this blog, Ready-to-hand, is a translation of Heidegger’s term zuhanden, though interpreting Heidegger’s philosophy is not specifically a major interest of mine nor a focus here. Much has been made of the significance of phenomenology, most often Heidegger, for human-computer interaction (HCI) and interaction design (e.g., Winograd & Flores 1985, Dourish 2001). And I am generally pretty sympathetic to phenomenology as one inspiration for HCI research. I want to just note a bit about the term zuhanden and my choice of it in a larger context — of phenomenology, HCI, and a current research interest of mine: cues for assuming the intentional stance toward systems (more on this below).
The Lifeworld and ready-to-hand
Heidegger was a student of Edmund Husserl, and Heidegger’s Being and Time was to be dedicated to Husserl.1 There is really no question of the huge influence of Husserl on Heidegger.
My major introduction to both Husserl and Heidegger was from Prof. Dagfinn Føllesdal. Føllesdal (1979) details the relationship between their philosophies. He argues for the value of seeing much of Heidegger’s philosophy “as a translation of Husserl’s”:
The key to this puzzle, and also, I think, the key to understanding what goes on in Heidegger’s philosophy, is that Heidegger’s philosophy is basically isomorphic to that of Husserl. Where Husserl speaks of the ego, Heidegger speaks of Dasein, where Husserl speaks of the noema, Heidegger speaks of the structure of Dasein’s Being-in-the-world and so on. Husserl also observed this. Several places in his copy of Being and Time Husserl wrote in the margin that Heidegger was just translating Husserl’s phenomenology into another terminology. Thus, for example, on page 13 Husserl wrote: “Heidegger transposes or transforms the constitutive phenomenological clarification of all realms of entities and universals, the total region World into the anthropological. The problematic is translation, to the ego corresponds Dasein etc. Thereby everything becomes deep-soundingly unclear, and philosophically it loses its value.” Similarly, on page 62, Husserl remarks: “What is said here is my own theory, but without a deeper justification.” (p. 369, my emphasis)
Heidegger and his terms have certainly been more popular and in wider use since then.
Føllesdal also highlights where the two philosophers diverge.2 In particular, Heidegger gives a central role to the role of the body and action in constituting the world. While in his publications Husserl stuck to a focus on how perception constitutes the Lifeworld, Heidegger uses many examples from action.3 Our action in the world, including our skillfulness in action constitutes those objects we interact with for us.
Heidegger contrasts two modes of being (in addition to our own mode — being-in-the-world): present-at-hand and ready-to-hand (or alternatively, the occurant and the available (Dreyfus 1990)). The former is the mode of being consideration of an object as a physical thing present to us — or occurant, and Heidegger argues it constitutes the narrow focus of previous philosophical explorations of being. The latter is the stuff of every skilled action — available for action: the object becomes equipment, which can often be transparent in action, such that it becomes an extension of our body.
J.J. Gibson expresses this view in his proposal of an ecological psychology (in which perception and action are closely linked):
When in use, a tool is a sort of extension of the hand, almost an attachment to it or a part of the user’s own body, and thus is no longer a part of the environment of the user. […] This capacity to attach something to the body suggests that the boundary between the animal and the environment is not fixed at the surface of the skin but can shift. More generally it suggests that the absolute duality of “objective” and “subjective” is false. When we consider the affordances of things, we escape this philosophical dichotomy. (1979, p. 41)
While there may be troubles ahead for this view, I think the passage captures well something we all can understand: when we use scissors, we feel the paper cutting; and when a blind person uses a cane to feel in front of them, they can directly perceive the layout of the surface in front of them.
Transparency, abstraction, opacity, intentionality
Research and design in HCI has sought at times to achieve this transparency, sometimes by drawing on our rich knowledge of and skill with the ordinary physical and social world. Metaphor in HCI (e.g., the desktop metaphor) can be seen as one widespread attempt at this (cf. Blackwell 2006). This kind of transparency does not throw abstraction out of the picture. Rather the two go hand-in-hand: the specific physical properties of the present-at-hand are abstracted away, with quickly perceived affordances for action in their place.
But other kinds of abstraction are in play in HCI as well. Interactive technologies can function as social actors and agents– with particular cues eliciting social responses that are normally applied to other people (Nass and Moon 2000, Fogg 2002). One kind of social response, not yet as widely considered in the HCI literature, is assuming the intentional stance — explanation in terms of beliefs, desires, hopes, fears, etc. — towards the system. This is a powerful, flexible, and easy predictive and explanatory strategy often also called folk psychology (Dennett 1987), which may be a tacit theory or a means of simulating other minds. We can explain other people based on what they believe and desire.
But we can also do the same for other things. To use one of Dennett’s classic examples, we can do the same for a thermostat: why did it turn the heat on? It wanted to keep the house at some level of warmth, it believed that it was becoming colder than desired, and it believed that it could make it warmer by turning on the heat. While in the case of the thermostat, this strategy doesn’t hide much complexity (we could explain it with other strategies without much trouble), it can be hugely useful when the system in question is complex or otherwise opaque to other kinds of description (e.g., it is a black box).
We might think then that perceived complexity and opacity should both be cues for adopting the intentional stance. But if the previous research on social responses to computers (not to mention the broader literature on heuristics and mindlessness) has taught us anything, it is that made objects such as computers can evoke unexpected responses through other simplier cues. Some big remaining questions that I hope to take up in future posts and research:
- What are these cues, both features of the system and situational factors?
- How can designers influence people to interpret and explain systems using folk psychology?
- What are the advantages and disadvantages of evoking the intentional stance in users?
- How should we measure the use of the intentional stance?
- How is assuming the intentional stance towards a thing different (or the same) as it having being-in-the-world as its mode of being?
References
- But Husserl was Jewish, and Heidegger was himself a member of the Nazi party, so this did not happen in the first printing. [↩]
- Dreyfus (1990) is an alternative view that takes the divergence as quite radical; he sees Føllesdal as hugely underestimating the originality of Heidegger’s thought. Instead Dreyfus characterizes Husserl as formulating so clearly the Cartesian worldview that Heidegger recognized its failings and was thus able to radically and successfully critique it. [↩]
- It is worth noting that Husserl actually wrote about this as well, but in manuscripts, which Heidegger read years before writing Being and Time. [↩]
Expert users: agreement in focus from two threads of human-computer interaction research
Much of current human-computer interaction (HCI) research focuses on novice users in “walk-up and use” scenarios. I can think of three major causes for this:
- A general shift from examining non-discretionary use to discretionary use
- How much easier it is to find (and not train) study participants unfamiliar with a system than experts (especially with a system that is only a prototype)
- The push from practitioners in the direction, especially with the advent of the Web, where new users just show up at your site, often deep-linked
This focus sometimes comes in for criticism, especially when #2 is taken as a main cause of the choice.
On the other hand, some research threads in HCI continue to focus on expert use. As I’ve been reading a lot of research on both human performance modeling and situated & embodied approaches to HCI, it has been interesting to note that both instead have (comparatively) a much bigger focus on the performance and experience of expert and skilled use.
Grudin’s “Three Faces of Human-Computer Interaction” does a good job of explaining the human performance modeling (HPM) side of this. HPM owes a lot to human factors historically, and while The Psychology of Human-Computer Interaction successfully brought engineering-oriented cognitive psychology to the field, it was human factors, said Stuart Card, “that we were trying to improve” (Grudin 2005, p. 7). And the focus of human factors, which arose from maximizing productivity in industrial settings like factories, has been non-discretionary use. Fundamentally, it is hard for HPM to exist without a focus on expert use because many of the differences — and thus research contributions through new interaction techniques — can only be identified and are only important for use by experts or at least trained users. Grudin notes:
A leading modeler discouraged publication of a 1984 study of a repetitive task that showed people preferred a pleasant but slower interaction technique—a result significant for discretionary use, but not for modeling aimed at maximizing performance.
Situated action and embodied interaction approaches to HCI, which Harrison, Tatar, and Senger (2007) have called the “third paradigm of HCI”, are a bit different story. While HPM research, like a good amount in traditional cognitive science generally, contributes to science and design by assimilating people to information processors with actuators, situated and embodied interaction research borrows a fundamental concern of ethnomethodology, focusing on how people actively make behaviors intelligible by assimilating them to social and rational action.
There are at least three ways this motivates the study of skilled and expert users:
- Along with this research topic comes a methodological concern for studying behavior in context with the people who really do it. For example, to study publishing systems and technology, the existing practices of people working in such a setting of interest are of critical importance.
- These approaches emphasize the skills we all have and the value of drawing on them for design. For example, Dourish (2001) emphasizes the skills with which we all navigate the physical and social world as a resource for design. This is not unrelated to the first way.
- These approaches, like and through their relationships to the participatory design movement, have a political, social, and ethical interest in empowering those who will be impacted by technology, especially when otherwise its design — and the decision to adopt it — would be out of their control. Non-discretionary use in institutions is the paradigm prompting situation for this.
I don’t have a broad conclusion to make. Rather, I just find it of note and interesting that these two very different threads in HCI research stand out from much other work as similar in this regard. Some of my current research is connecting these two threads, so expect more on their relationship.
References
Dourish, P. (2001). Where the Action Is: The Foundations of Embodied Interaction. MIT Press.
Grudin, J. (2005). Three Faces of Human-Computer Interaction. IEEE Ann. Hist. Comput. 27, 4 (Oct. 2005), 46-62.
Harrison, S., Tatar, D., and Senger, P. (2007). The Three Paradigms of HCI. Extended Abstracts CHI 2007.