Usability Testing Some simple tests and techniques that can be used to find out about your designs and your users Pen Lister MSc PGCert Nov 2009

Which usability test? There are many tests to choose from Think about your goal(s) and WHO you are testing Think about resources Think about variation Think about COMPARING test results

How many people to test? Up to 5 people in each category sufficient, according to Nielsen (2000) http://www.useit.com/alertbox/20000319.html BUT this has been challenged: http://www.usabilitynews.com/news/article87.asp

What about questionnaires? Keep surveys SHORT http://www.useit.com/alertbox/20040202.html Number of people can vary, but try to include people who are REPRESENTATIVE of your users As many participants as are relevant to your study http://www.uie.com/brainsparks/2007/10/22/usability-tools-podcast-statistical-significance/

Some techniques Scenarios (task setting) Heuristic Evaluation (aka Usability Audit) Card Sorting (Information Architecture) Think Aloud (Verbal protocol analysis) techniques Observation techniques

Requirements What you may need: Some TARGET users! Tasks that suit the type of user you are testing Post it notes for card sorting A test environment (computer, quiet room) A stopwatch A notepad Questionnaires or list of questions to ask Sign off forms Team members to help

Scenarios The process of setting users a task (or tasks) to carry out using the system Observing the user carry them out Timing how long it takes to complete them Recording what they say or do while carrying out the task Comparing the results of several users doing the same task Important - researcher does not engage in a dialogue with subject - only speaks to clarify instructions

Scenarios / types Types of task to consider: Find X product on a website Choose a character to be ‘you’ in a game Register on a website Change profile details on a website or in a game Search ‘detail’ in a system Read ‘information’ on a page (literacy/font)

Scenarios / ‘design’ the task Designing the task to test needs thought What do you want to find out that will affect the DESIGN of the interface? How can you test SEVERAL users with the same test? How can you test user groups and COMPARE results meaningfully (with another test)?

Scenarios : Data Recording Number of clicks; time on pages; time to complete task Physical actions number, type of errors delays in carrying out tasks inference of poor/good design through gestural analysis Verbal behaviour mismatches between expectations & outcome paralinguistic utterances (“um”, “er”, tutting etc.)

Scenarios : Observation Techniques Behaviours - physical actions Purposeful: key presses object selection with mouse - Accuracy of recording very important here Involuntary: head scratching sighing drumming fingers on desk etc. - This can provide useful insights, but care needed with interpretation

Think Aloud ‘Verbal protocol analysis’ Suitability of users to be established by questionnaire/interview Speaking your thoughts aloud “…When subjects are asked to think aloud, some of their verbalizations seem to correspond to merely vocalizing “inner speech,” which would otherwise have remained inaudible… ” http://www.psy.fsu.edu/faculty/ericsson/ericsson.proto.thnk.html

Think Aloud: Data Recording Physical actions number, type of errors delays in carrying out tasks inference of poor/good design through gestural analysis Verbal behaviour mismatches between expectations & outcome paralinguistic utterances (“um”, “er”, tutting etc.)

Think Aloud: Observation Techniques Behaviours – verbal utterances Categories intentions “I’ll press F1” expectations “F1 should bring up Help” description “ok, the back button’s down here” background information, opinion “system X does it like…” comments on effects of actions “oh ****, I’ll have to start again” Assumption that they mirror underlying cognitive processes - open to debate (see previous link – do other research on this)

Think Aloud or Talk Aloud? Think Aloud Protocol asks users to say what they are doing, looking at, thinking and feeling while they carry out their task Talk Aloud Protocol is slightly different: it involves participants only describing their action but not giving explanations. - is this more objective? Ericsson, K., & Simon, H. (1987). "Verbal reports on thinking". in C. Faerch & G. Kasper (eds.). Introspection in Second Language Research. Clevedon, Avon: Multilingual Matters. pp. 24–54

Card Sorting Card sorting is concerned with ‘Information Architecture’ IA is a very important part of a system design IA defines how the system divides its information IA defines how navigation is designed

Card Sorting: why? Creating a ‘map’ of the knowledge Users may do this differently than you! Jacob Nielsen: Card Sorting http://www.useit.com/papers/sun/cardsort.html How many users to test? http://www.useit.com/alertbox/20040719.html Usability Engineering, 1993, p127

Card Sorting: how Create cards (or post it notes) with different knowledge topics on them Ask users to group according to topic OR Ask users to group according to tasks they might perform Can be: Open sorting – user decides how to group info Closed sorting – user is asked to group according to predefined categories

Card Sorting examples Ref: http://www.boxesandarrows.com/view/card_sorting_a_definitive_guide http://usability.de/en/services/card-sorting.html http://www.uxmatters.com/mt/archives/2009/04/analysis-plus-synthesis-turning-data-into-insights.php

Heuristic Evaluation Jacob Nielsen’s Ten Heuristics: Visibility of system status Match between system and the real world User control and freedom Consistency and standards Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose, and recover from errors Help and documentation

Heuristic Evaluation Example of how to apply the ten heuristics: http://www.furiant.com/materials/web/radar_heuristics.pdf

Heuristic Evaluation

Heuristic Evaluation - more You don’t have to use all the heuristics Ask users about factors that can be categorised by a heuristic Or gather data about user behaviour that you can categorise by a heuristic It may be helpful to refer to Quesenbery, W. (2003). “The five dimensions of usability” to match up to heuristic

The five second test Ask users to look at a prototype or screenshot of your design for 5 seconds and list max. 5 things they can remember This can help to streamline content and let you know what is working for users visually Helps the design to be ‘easy to learn’ because it is easy to recall important parts of design

The five second test - more This can be carried out on any section of your design, not just the homepage Variations on this test might be: 10 seconds to find ‘X’ link on a webpage OR 5 seconds to get back to the homepage from here Go here for more advice on this method: http://www.uie.com/articles/five_second_test/ (Jared Spool)

VERY useful advice! Guerrilla HCI - Discount Usability Engineering Scenarios Simplified thinking aloud Heuristic evaluation http://www.useit.com/papers/guerrilla_hci.html (Jacob Nielsen)

Good Links Task-Centered User Interface Design http://grouplab.cpsc.ucalgary.ca/saul/hci_topics/tcsd-book/chap-1_v-1.html (last accessed 8/11/09) Usability Demystified: http://www.alistapart.com/articles/usability-testing-demystified/ (last accessed 8/11/09) A five second test website http://fivesecondtest.com

References/Bibliography A List Apart (various) http://www.alistapart.com/topics/topic/usability/ Ericsson, 2002, Protocol analysis and Verbal Reports on Thinking http://www.psy.fsu.edu/faculty/ericsson/ericsson.proto.thnk.html Furiant Consulting Heuristic Checklist, Radar Project http://www.furiant.com/materials/web/radar_heuristics.pdf Nielsen, J, 1993, Usability Engineering, Morgan Kaufmann Nielsen, J, Useit.com Alert Box, various, http://www.useit.com/alertbox/ Spool J, et al, User Interface Engineering Articles http://www.uie.com/articles/ Usability News, 2001, Discount User Testing Under Fire http://www.usabilitynews.com/news/article87.asp NB all websites last viewed 8/11/09

End