User Experience Evaluation Methods

User experience is widely used but in different understandings and ways. And sometime is very difficult to evaluate, below described three approaches for evaluating user experience.

1. Anticipated eXperience Evaluation (AXE)

This approach is used to address the following challenges like

  • experiences illustrations with words
  • imagine and explain an experience
  • endless interpretation ways of participants answers elicited as words

AXE is a qualitative method that has been designed to provide development teams insights on how future users might experience and value a product or service concept. AXE required one user at a time and doesn’t demand for trained researchers and special equipment. And it is suitable for web services, pc software, hardware and mobile software designs.

Design targets defined before evaluation by design team, on the basis of design targets results will be compared and give the shared understanding of the goals throughout the development stages and the ability to assess whether user’s perception of the concept matches the goals.

Concept Briefing

Concept conveyed through description and narratives. Also illustrations and lo-fi prototypes are used to clarify and for better understanding and handed these to participant for using throughout the session for clearing understandings about idea/concept.

This activity start at the beginning of the evaluation session the concept needs to be presented to participants each time in the same manner and order to guarantee comparable results.

Concept evaluation

This study has been done by three parts: instructions, warm-up, image pairs. The instructions are general tips or overview to introduce the participant with the evaluation approach and guide through the process and contains one warm-up exercise to practice the evaluation with the participant. And the image pairs approach to get the knowledge of participants experiences, attitudes, opinions and beliefs towards a given product concept.

Data analysis

Subsequent to the interview, the data is transcribed and analyzed. Following steps for coding the data:

  • Transcription, to get as much information as possible and has to be done word-to-word
  • Separate text that get from transcription process into manageable segments
  • Categorize by similar information after step 2
  • Sorted by positive, negative and not applicable evaluations

Strengths

  • No presuppositions
  • Users define freely their valuations and points of interest
  • deep insights into the participants real life context
  • No specific training or interviewing skills required
  • provide a method to study how people perceive a product/concept at a very early stage
  • help developers for further refining and steering

Weaknesses

  • analysis framework for AXE was formed through multiple iterations
  • comparability between similar concepts
  • requires visual literacy from participants
  • time consuming

2. Exploration Test

The exploration test approach is achieve via exploring already available products in market by doing competitive review for getting knowledge, understanding and gain perceptions of people about competitors strategies, work structure and future of your product.

We call this an ethnographic test for evaluating user’s perception of a design. We often listen and use term competitive analysis that mostly done by business development group’s, independent auditors for  assessing a company’s offerings from a financial or marketing perspective and gain understanding of how much competitors products cost, who buys them, where customers live, and what aspects advertising emphasizes.

the real goal of competitive user experience research is to figure out how to creatively differentiate your product from the competition—not just fix other people’s mistakes.

Strengths

  • to help executives in strategic decisions
  • focus groups reveals views of the brand and the identity of the product
  • reveal the fundamental strengths and weaknesses of the competition
  • easily get to know the more focused features by users
  • to get knowledge about main goals and basic needs of users
  • competitive research gives deeper understanding of what makes a good user experience
  • admiration for your competitors’ ability to solve problems and pride in your own product

Weaknesses

  • problem in to create a correct interview script and the subjective analysis
  • difficult to account all data and responses

Consider as a field study by using early prototypes, functional prototypes, products on market and one user at a time.

3. Sentence Completion

Sentence completion technique to find out the product Symbolism that users attach to a product. For this technique we use one word ”Symbolic” that refers to the image and associations that spring to mind in regard to a product. In this study system & product use as a symbol and give it to participant for exploring different features of system, after that, participant is handed a set of beginnings of sentences that need to be complete based on used system knowledge/experience.

In this case participants provide feedback to designers to give understanding about how users see their products. Symbolic meaning is not an easy and one-dimensional concept. It need to be evaluated as users point of view, need to understand its components and to know about how they can affect user experience. The following factors of symbolic meaning and relationship to product experience, they describe the nature of phenomenon:

  • Memory retrieval and associations
  • Support for identity, self-expression or status
  • Beliefs about the kinds of people who use the product
  • Support for user values and social relatedness

This consider as a qualitative and quantitative approach and single user at a time. Also required trained researchers and functional prototypes or products on market during development phase.

Strengths

  • respondents use their own words to describe their situation
  • more spontaneous and honest answers compared to traditional questionnaires
  • uncover conflicted attitudes, values and associations
  • support the positive meanings and correcting features that create negative reactions

Weaknesses

  • hard to anticipate other people’s reactions
  • designers and users may attach different meanings to a product
  • Symbolic meaning is challenging to evaluate because of its intangible nature
  • interviewing is time demanding and only a limited number of users can participate
  • intangible nature

UX Evaluation Method for iBeacon Project

On the basis of my study and nature of our project I will select Exploration test method for user experience evaluation. We have already performed competitive research/review not in very detail and professional way but a small effort to identify and get to know about our competitors in current market.

https://ibeacontlu.wordpress.com/2016/02/11/competitive-review/

References

  • Anticipated eXperience Evaluation by Gegner, L. and Runonen, M. 2012
  • Observing user experience by Elizabeth Goodman, Mike Kuniavsky and Andrea
  • Sentence Completion for Evaluating Symbolic Meaning by Sari Kujala and Piia Nurkka
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s