Researching Concepts with Comics According to Mark Wehner, Yahoo! Inc.
31 Oct 2006

Researching Concepts with Comics According to Mark Wehner, Yahoo! Inc.

31 Oct 2006

Two weeks ago, I attended a talk given by Mark Wehner of Yahoo! Inc. at a BayCHI event in Mountain View. At first glance many may balk at the idea of conducting research through mere drawings, but having heard and seen the impact this tool can make, I am now a huge enthusiast for this exploratory process. I am writing this article in the hopes that more companies and user experience designers investigate this technique to see how it can enhance their own product research.

This article summarizes the key concepts behind researching with comics as presented in the talk along with some other considerations around this technique.

Note: this talk was identical to the one presented at UPA’s Summit earlier this year. A few links to other discussions and examples inspired by that talk are given at the end of this article.

Participatory Design through Drawings, Words and Prototypes

Researching through comics is one of many participatory design techniques (approach to system development which actively incorporates prospective users into the design process). Comics can also be used as a product definitions document internally: between design teams, product development teams and stakeholders. For the time being, let us focus on comics as data gathering tools.

The visual representations of concepts provide an avenue for learning about how a product or feature may fit within a user’s life without actually building out or designing the system. These comics, then, approach the system from a very high level in order to abstract the idea from its implementation, and thus allow the participants to conceptualize the product for themselves. As such, reactions are based upon the ideas and not the particulars of the design or interaction.

User experience designers use tools such as surveys, traditional storyboards, and prototype testing to conduct research which can give insight into the compatibility of an idea with a particular user’s needs and perspectives. So, how then do comics extend these techniques? Mark pointed out the tradeoffs for surveys, storyboards and comics; let’s look at these in turn:

Surveys

Conducting surveys is a quantitative participatory design method of acquiring user feedback about a concept or feature. These are more frequently being conducted online due to simplicity and the added benefit of acquiring great amounts of data with minimal overhead costs.

Participatory design surveys heavily employ questions which explain/describe a concept and then ask the participant’s reaction based upon a range of choices. The Likert scale is most commonly used in this case. This scale was designed to measure user attitudes by quantifying subjective information using a set of responses such as “Strongly Disagree,” “Somewhat Disagree,” ‘Slightly Disagree,” “Neutral,” “Slightly Agree,” “Somewhat Agree” and “Strongly Agree.”

Benefits:

  • Surveys can provide a large set of comparable data which can be analyzed to provide a median reaction to a set of features or concepts.
  • No additional time is spent creating a prototype of the system for the explicit purpose of gathering data.
  • Gathered data can be used to focus design efforts in the more important areas.

Drawbacks:

  • The results of these tests do not directly translate to the actual relevance of a product in everyday use. Mark points us to the Harvard Business Review’s research in feature fatigue, which explains that the scenarios are not tangibly placed in context of their usage and thus are easily misinterpreted and features, bloated.

Traditional Storyboards

Traditional storyboards are a series of wireframes or screenshots which depict a system’s interface, arranged in a way which describes a particular set of use cases. By stepping a participant through these screens, the usability and practicality of the system within a specific scenario can be uncovered concurrently.

Benefits:

  • Storyboards provide qualitative design feedback of a system before spending the time to develop a prototype.
  • As an investigatory tool, storyboards can uncover more realistic reactions to the features viability in relation to the intended implementation.

Drawbacks:

  • Participants often have problems understanding wireframes as blueprints or conceptualizing the screenshots as a developed system.
  • Actually designing the interface is excessive and incompatible for cases in which we want to determine which features to implement; that is where we want feedback on the concept rather than the functioning product

Prototype Testing

Prototypes are generally considered to be very accurate in providing realistic reactions to a feature or product as it removes the ambiguity of representative figures, words or non-working models. The use of these as investigatory tools, however are most often avoided due to the extremely high cost to create these models, and the risk of limiting the user’s impact in shaping the design.

Benefits:

  • Provides feedback on a working implementation.

Drawbacks:

  • Not effective for researching concepts as the full development has to be executed, and the focus shifts to the tangible system and interface.

Pictures Are Worth a Thousand Words

Each of the methods described above provides valuable insight into a participant’s reaction to a set of features from different perspectives. The mechanisms used are varied from word descriptions to visual representation to interactive models. Although, each is effective in its own right, two major obstacles remain:

During the investigatory phase, we want to quickly and easily communicate a concept which the user can visualize and empathize with in the context of its use without actually building/designing the system

We need to ascertain what aspects are appealing, useful or confusing about the concept and which are not while still letting the user “experience” the system

The technique of articulating the user’s story with the product through comics tackles these obstacles extremely well. They become a central communication tool for both the design team and the end-user, answering the questions that the aforementioned techniques could not very well and reducing the time necessary for creating the test materials.

Comics as a Communication Tool

The greatest take-away from the talk was that comics are great conversation facilitators. Illustrations can be used very effectively to explore a feature or product with a potential user and illicit more meaningful conversation about its viability and appeal than using words or detailed interfaces. By using drawings, participants are given a set of concrete objects around which to focus their responses. Additionally, drawings are able to abstract the ideas so as to allow the user to focus on the concepts rather than the intricacies of how the system actually looks or functions as is the case with wireframes, designs or prototypes.

The participatory design process with comics consists of giving a participant a paper version of the storyboard on which to write and peruse, ensuring that the screens are large enough to be read and clearly perceived. After explaining the process to the participant, the moderator asks the user to read the comics (privately to process the story). The participant then walks through the comics again, thinking aloud and expressing his/her reactions. Beyond this a series of tasks to gain more insight are also facilitated:

  • participant walks through the scenarios again, annotating each screen with thoughts
  • participant walks through the scenarios again, annotating parts of the story that are appealing, useful, confusing or complex (these items should be tailored to the purpose of the test)
  • participant tells personal stories about how they could use the product/features

This process worked well for Mark’s team and was refined during the course of the testing. The data collection was also supplemented with affinity note-taking techniques employed by test observers.

Key Benefits of Comics

  • the ability to redefine use-cases based on prospective users’ own stories
  • acquiring qualitative data directly related to “experiencing” the concept
  • the opportunity to refine descriptions of the product’s use
  • discovering weak and strong points of the product, and the reasons for these distinctions
  • having a easy to read and understand representation of the essence of the product (with iterations)

Pointers for Creating Comics and Testing

Mark outlined a few things important to effective comics and testing

  • tell stories linearly: incorporate screens that help tell the story and remove decision point paths
  • iterate comics to improve comprehensibility and effectiveness
  • related to iterations and testing fatigue, allow enough time between sessions
  • explore use cases and personas to inform your comics
  • be aware of which cells are key to features that are important to stakeholders, etc.
  • expose only those elements of the interface that aid in telling the story, for e.g. using radio buttons to show decisions or other simple visual cues

Conclusion & Further Links

As described in this article, comics can serve as a great tool for refining product definitions and feature specifications. By using illustrations as a tool, participants are encouraged to imagine a certain product and the benefits it may offer based on depicted scenarios rather than expecting the user to draw these connections automatically from an interface. This mechanism expands an already rich set of tools that user experience designers use to conduct product research which ensures a usable and useful product for the target market.

This tool, as with others, uncovers many unexpected considerations and possibilities for a product that may not be thought of otherwise, for instance revealing previously unconsidered use cases. This is further enforcement of the necessity of thorough conceptual research prior to defining what a product will be and do. Furthermore, with tools such as these, user experience designers are a key resource for product definition which more companies should employ. Too often, executives, engineers and other professionals make key decisions about the features to add to a product without expert explorations of these ideas. Hopefully, as more tools and advocates arise these procedures will become commonplace, and the applicability of tools improved.

Summary of “Researching Concepts with Comics” from UPA Summit


Leave a comment
More Posts
Comments
  1. P.J. Onori November 3rd, 2006 10:45AM

    Wow, great article – such an interesting concept. I really like this idea as you do not have to spend precious time on something that may be potentially bound to fail. The amount of hours saved is tremendous – those hours can then be spent going through a thorough design process with something that has already had initial positive reactions. Can you imagine how much better a final product could be if design started with something that has been proven to work well in studies?

    Great writeup!

Comment

Leave a Reply