Testing the paper prototypes

After creating paper prototypes we have evaluated those prototypes by conducted four different testing sessions with users. We have selected 3 main tasks for these sessions and provided to users to perform those tasks.

Here the list of our selected three main testing tasks scenarios: Testing Tasks and paper prototypes screenshots.

Summary

Overall, the testing procedure was quite good, the three tasks were easy, self descriptive, presentable and have some common functionality as well. All the tasks were performed by users with minor difficulties, we have also received lot of small suggestions and 1 big suggestion/change. It was also challenging to perform these tasks on paper prototypes by acting like blind user.

Search Location, was our first task it was more a like filtering task to find particular room. All users performed it but felt that it is very hectic and laborious job for blind people to navigate on multiple screen for selecting/searching desired place.

This was the main concern of users, and it poked us to rethink for another approach to perform this functionality as more reliable and perform-able for blind users.

Our second testing tasks, Bookmark location was simple and performed easily, because all the steps were same and we only need to add 1 more step to perform this task, a Bookmark Location popup window on search results.

The main concerned was raised by users is that why popup window?  What would be if I want to listen to the voice messages again? These were valid concerns because popup window appeared immediately after last instruction and then instruction disabled and users cannot perform until they click on back button or bookmarked this location button.

Our last task, Selection from Bookmarked Locations, all the users performed without any difficulties, but 1-2 users were concerned about a delete feature for old bookmarked locations which was not included. This called for modification to include a delete option to remove already bookmarked location.

Most commonly appeared issues

  1. Too much time taking process for searching a particular location.
  2. Popup window for location bookmarking
  3. No autoplay for voice messages
  4. Delete bookmarked locations
  5. What happens when a user doesn’t know the room number?
  6. Back and home buttons on top of the screen

Post Test

After our testing with 4 users we have also performed one post testing after implementation of an issue #1 (search location). We added speak-able keypad with big numeric buttons like below image and the user said it is good, easy and more applicable for blind users

5

User also suggested one more thing to add voice recognition for searching particular area/location, but this is a big and more complex change but we plan to have this included in our future tasks list.

Test #1 Feedback

  1. Remove navigation screens for filtering location
  2. Add search location by entering room/area address
  3. Add voice recognition
  4. Autoplay instructions
  5. Add map on screen with all locations and user know about location by clicking on map
  6. Current location indicator when user lost or want to know surroundings

Test #2 Feedback

  1. Remove navigation screens for filtering location
  2. Add search location by entering room/area address
  3. Autoplay instructions
  4. Remove popup window
  5. Add button on top or bottom (we have done this by adding the bookmark option in instructions list at the end)

Test #3 Feedback

  1. Remove navigation screens for filtering location
  2. Add search location by entering room/area address
  3. Remove navigation screens for filtering location
  4. Add search location by entering room/area address
  5. Add voice recognition
  6. Delete bookmarked locations

Test #4 Feedback

  1. Announce initial location
  2. Search by map by tapping on screen
  3. Remove navigation screens for filtering location
  4. Add search location by entering room/area address
  5. Add voice recognition along filtering process
  6. Delete bookmarked locations
  7. Generate voice messages on hover and long press for selection
Advertisements

Testing Tasks

  1. Search Location Task

User intends to move from the lobby at Astra building to S303.

  • User opens the iBeacon navigation app
  • Request for connection
  • Screen will open with 2 large buttons and 2 small buttons

               1 Large button will be for search location

               2 Large button will be for bookmarked locations

               1 small button for Disconnect

              2 Small button for starting current location navigation

  • Taps on search location
  • The next screen opens a list of the major buildings user can navigate through left and right swipe

          Screen will contain two top buttons (Home and Back)

          Swipe left for moving up

          Right for down

          System will speak option

          Tap on desired option

          System ask tap again for making selection

          User will perform second tap for selection

  • Next screen opens a list of available classrooms, restrooms, offices, and other areas in this building section.

          User scrolls and gets audio message mentioning the name for each list

          Tap on desired location

           System ask tap again for making selection

           User will perform second tap for selection

           Screen will open with possible navigation options

           Audio message gives direction to selected location

           User can perform left/right swipe for listening again

  1. Bookmark Location Task
  • User opens the ibeacon navigation app
  • Request for connection
  • Screen will open with 2 large buttons and 2 small buttons

          1 Large button will be for search location

          2 Large button will be for bookmarked locations

          1 small button for Disconnect

          2 Small button for starting current location navigation

  • Taps on search location
  • The next screen opens a list of the major buildings user can navigate through left and right swipe

          Screen will contain two top buttons (Home and Back)

          Swipe left for moving up

          Right for down

          System will speak option

          Tap on desired option

          System ask tap again for making selection

          User will perform second tap for selection

  • Next screen opens a list of available classrooms, restrooms, offices, and other areas in this building section.

          User scrolls and gets audio message mentioning the name for each list

          Tap on desired location

          System ask tap again for making selection

          User will perform second tap for selection

  • Screen will open with possible navigation options

         Screen will also contain bookmark button on top

         User will tap on this bookmark button

         Location will be bookmarked

         System will generate audio message

  1. See/Selection from Bookmarked Locations
  • User opens the ibeacon navigation app
  • Request for connection
  • Screen will open with 2 large buttons and 2 small buttons

         1 Large button will be for search location

         2 Large button will be for bookmarked locations

         1 small button for Disconnect

         2 Small button for starting current location navigation

  •  User taps on bookmarked locations
  • The next screen opens a list of the bookmarked locations

         Screen will contain two top buttons (Home and Back) 

         Swipe left for moving up

         Right for down

         Tap on desired location

         System ask tap again for making selection

         User will perform second tap for selection

         Screen will open with possible navigation options for selected place

         Audio message gives direction to selected location

         User can perform left/right swipe for listening again

Sorting techniques for eliciting people’s experiences

Repertory Grid Technique

To get useful feedback and diverse opinions on different stages of the design from different users before doing final product UX evaluation. This process do by building early prototypes and creating all possible scenarios. It benefits to fulfill the actual user needs and desires.

In this study, structured interview was built by using repertory grid technique and divided into two main phases. In first phase, researcher created three different bipolar elements and presented to participants and in second phase asked for rate those elements on own opinions.

The same approach was considered in this journal as well “Capturing The Design Repertory Space From Grid Technique a User Perspective” for finding, exploring, understanding design space in early prototypes, generate different views on the artifacts, embodying various individual needs and concerns in relation to the artifact.

Below are findings and suggested steps of this research after applying this approach by using 11 group of people.

  • Charting the design space
  • Exploring and understanding design space
  • Abstraction: Underlying topics made visible

At final, the best design alternative is generally the one the most people agree on. This view rooted in the quantitative research tradition.

A comparison of five elicitation techniques for elicitation of attribute of low involvement products

Compared five elicitation techniques on eight criteria derived from the theories of consumer buying behavior.

  • First triadic sorting for mapping cognitive structure.
  • Second is free sorting, form groups based on important aspects as compare to other products.
  • Third is not a sorting method, but use to find respondent relevant attributes products, called direct elicitation.
  • Another one is ranking for prioritizing products according to preference.
  • Last one, is picking from an attribute list.

The purpose of these approaches to find the theoretical conceptualisations of relationship between product attributes and consumer choice and motivation.

What aspects are you interested in improving (e.g. usability, beauty, satisfaction, …)

In our project I am interested to improve ease of use because it is related to blind people and trying to make this app as easy as we can do.

What design details you are unsure about (e.g. menu items, visual appearance, functions…)

I am not sure about the voice messages/commands like we need to evaluate our messages based on voice quality, rhythm, speed, pitch etc

Who would you like to evaluate the prototype: expert or users

In our study we are only considering users, first we have planned to do evaluation with actual user (to make them blind temporarily) by using paper prototype along scripted voice messages (scripted audio).

After that we will do our final evaluation with actual blind user and it could be available and participate before our final evaluation but it depends on his availability.

Do you need qualitative or quantitative data, or perhaps both? Later in class we’ll collect some data and do the analysis.

We required both qualitative and quantitative data to evaluate our first prototype.

Selection of suitable color scheme, menu items etc

About color scheme and design we are not putting much effort on these, we are focusing on audio interfaces and how can we provide ease of use to blind people. We have documented all the possible functionality and scenarios (in our previous group post) those will be addressed in this application.

References:

Karapanos, Evangelos, and Jean-Bernard Martens. 2008. “The Quantitative Side of the Repertory Grid Technique: Some Concerns.” UXEM Workshop in CHI’08, April 6th, 2008. http://arxiv.org/abs/0909.3158.

Hassenzahl, Marc, and Rainer Wessler. 2000. “Capturing Design Space From a User Perspective: The Repertory Grid Technique Revisited.” International Journal of Human-Computer Interaction 12 (3): 441–59. doi:10.1207/S15327590IJHC1203&4_13.

Bech-Larsen, Tino, and Niels Asger Nielsen. 1999. “A Comparison of Five Elicitation Techniques for Elicitation of Attributes of Low Involvement Products.” Journal of Economic Psychology 20 (3): 315–41. doi:10.1016/S0167-4870(99)00011-2.

 

Methods for Evaluating Early Prototypes

Object-based

Object based techniques help to understand how people come to think, feel, and know about their lives. Object-based technique categories in three main parts:

Photo Elicitation

Participants respond to a set of images, discuss images and on those responses we try to understand what they are thinking, feeling and see in them, after analysis we apply those interpretations & knowledge in our own project.

Collage and Mapping

Provide platform to participants to externalize emotions, expressing attitudes and desires. And mapping provide relationship between people and these objects.

Card Sorting

Participants sort cards with words or phrases in them into groups and uncover how people organize information, relate and categorize concepts. We can perform open and closed sorting based on the desire needs.

Multiple Sorting

This technique is used to evaluate and understand the user experience and explore people’s conceptualization. People ask for classification and then interviewed with them to know the reason of these categorisations.

Contextual laddering

One-to-one interviewing, qualitative data gathering and quantitative data analysis technique. Helps researchers and designers to understand how concrete product attributes benefit personal values for end users and decision making.

UX laddering is under development but have several advantages for this technique:

  • Greater emphasis on concrete attributes,
  • Suitable within a user-centered design approach
  • Answering why – questions

References:

Kuniavsky, M. (2003). Observing the User Experience. Observing the User Experience. http://doi.org/10.1016/B978-155860923-5/50041-3 (chapter#8)

World, D., Frohlich, D., & Wilson, M. (2008). User Experience : A Multiple Sorting Method based on Personal Construct Theory. Chi, 1–5.

Vanden Abeele, V., & Zaman, B. (2009). Laddering the User Experience! User Experience Methods, Interact 2009, (July). Retrieved from http://www.kuleuven.be/facdep/social/com/mediac/cuo/admin/upload/Laddering the User Experience.pdf