Jenny Le Peuple London Metropolitan University Requirements Gathering and Analysis

Mini case studies – common factors? picture source: http://www.hospital.org.uk/page586.htm picture source: http://news.bbc.co.uk/1/hi/business/2230760.stm picture source: http://elections.fas.harvard.edu/election2000/butterfly.pdf picture source: The Independent on Sunday 31/1/99 1 2 3 4

Computerised system for ambulance allocation/dispatch Shut down in < 2 days Failed (in part) because: poor communication between management & staff associations; system imposed key staff ‘on the ground’ not consulted in early stages key parts of system built by Co. with no experience (cheap) very poor interface Cost (est. in 1992) overspend of >£3m + (allegedly) loss of life The cost of poor RE, mini case studies (refer to slide 2) # 1: LAS (London Ambulance System) 1992

Intended to centralise the trading of stocks & shares and create ‘paperless’ transaction system Failed (in part) because: did not take account of different corporate cultures ‘culling’ of Stock Exchange IT staff managed by a committee US package adapted Est. cost £400 million (http://www.practicallaw.com/scripts/article.asp?Article_ID=4003) The cost of poor RE, mini case studies (refer to slide 2) # 2: TAURUS (London Stock Exchange System)

Attempt to design an electronic version of the ‘red boxes’ that are used to store/carry official documents Failed because: “too complicated to use” “too heavy” “misinterpreted words” “the people who were supposed to be using it – the ministers, couldn’t even get in” “the James Bond Stuff slowed things down. It drove them mad” (Woolf, 1999) Development cost estimated as at least £10,000 (excluding staff salaries, opportunity costs etc) + unspecified number of laptops @ £2,400 each The cost of poor RE, mini case studies (refer to slide 2) # 3: Computerised Ministerial ‘Red Boxes’ 1999

The cost of poor RE, mini case studies (refer to slide 2) # 4: US election 2000, Florida ballot papers picture source: http://jerz.setonhill.edu/design/usability/use-ballot.htm (first published in the Orlando Sun-Sentinel) very widely alleged throughout the media that many voters in Palm Beach County inadvertently voted for wrong candidate because of poor design of ballot paper hence, it is claimed, Al Gore (running for president) lost the vote (to George Bush) Cost?

The underlying problem

What is ‘requirements analysis’? Requirements engineering (RE) what do people need from a system understanding what needs mean in terms of design (Sutcliffe, 2002) Related to software engineering detailed design of system Requirements “designing the right thing” Software engineering “designing the thing right” (Boehm, 1981)

RE – objectives Capture complete set of requirements from users Analyse requirements accurately & understand implications Specify how to meet requirements in the design Complete the process within resource (e.g. cost & time) restraints (Sutcliffe, 2002)

The design cycle - generic form 1 requirements and functional analysis 2 preliminary, high-level design 3 specification 4 design 5 testing and evaluation 6 production 7 maintenance

‘Traditional’ waterfall lifecycle model requirements analysis problems? ‘..essentially linear’

Waterfall model – some weaknesses… Process hinges upon specific, initial user requirements - requirements often very unclear at the outset; generally uncovered throughout whole process User/s often only involved at requirements analysis stage if at all Lengthy & inflexible process, deliverables often only ‘appear’ at end of implementation phase - so too late to make changes Le Peuple & Scane, 2003 Smith-Atakan, 2006

Waterfall model – some weaknesses… Linear in nature, little scope for iteration - waterfall is poor model of how designers actually work; in reality they switch back and forth to incorporate new insights &ideas Usability issues often ignored - focus more on functionality Interface often ‘tacked on’ as afterthought - interface is the only part of a systems that the user interacts with directly; needs careful design

Waterfall model – some weaknesses… Overall design tends to reflect the designer’s view - rather than the users of the system; obviously the designer knows how to use the system because she or he designed it! Essentially technology (rather than human) centred - focus on the system rather than the user/task/context Le Peuple & Scane, 2003 Smith-Atakan, 2006

Alternatives - “Star” model (Hartson & Hix, 1989) key features: iteration evaluation

Alternatives - “Spiral” model (Boehm, 1988) From Boehm, 1988 Main steps: Define project objectives Risk Assessment Engineering & production Planning & management Key feature: Prototyping; gradually refining the requirements

User-centred systems development Human-centred as opposed to technology-centred, with focus on people’s characteristics, tasks & context of use Emerging standards: product ISO 9241-11: Guidance on Usability process ISO 13407: Human-centred Design Processes for Interactive Systems

User-centered design, characteristics Users as central actors Appropriate allocation of tasks between user & system Takes into account user characteristics, tasks & context of use Early focus on users, tasks, context Iteration of design solutions Many different models e.g participative and contextual design Range of developmental practices, methodologies, techniques, tools Often multi-disciplinary design teams (i.e. technical and human factors personnel)

Theoretical models - DSDM model Dynamic Systems Development Method Software Project Management, Hughes and Cotterell, 2002

Contextual design methodology, an example (Beyer & Holtzblatt, 1995, 1998 etc) based on: http://www.incent.com/cd/cdhow.html (Incontext Enterprises)

Probably the first major user-centred design project The design problem: to enable 10,000 athletes and officials (12 languages) to send and receive recorded messages to each other and their families during the 1984 LA Olympic Games using either touch pad or dial phones. to develop a networked system of over 35 computers scattered over Los Angeles to be rapidly designed, tested and changed over 8 months by a small group of people, to be used by anyone in the world with minimal training, to be ready for the Olympic Games on time. Olympic Messaging System (1984)

Printed scenarios of user interface ‘look’ helped to define the signing on procedure at an early stage Early iterative tests of user guides Early simulations of the OMS series of experiments, e.g. tested number of audio menu options that were acceptable An Olympian athlete in the team to provide feedback Interviews & demos with Olympians outside US. Methodology/techniques: Olympic Messaging System (1984)

Overseas interface tests with friends and family; Free “coffee and doughnut” tests – 65 people; Usability tests with 100 participants; Hallway methods and ‘try to destroy it’ test; Pre-Olympic field-test at an international event; (57 different usability problems still recorded) Tests of the reliability of the system with heavy traffic. Methodology/techniques: Olympic Messaging System (1984)

Key principles of user-centred design that were developed from the design of the OMS (Gould, 1987): focus early in the design process on users and their tasks measure users’ reactions and performance to scenarios, manuals, simulations, and prototypes (observed, recorded and analysed) design iteratively: when problems are found in user testing, fix them and carry out more tests all usability factors should be under the responsibility of one control group. Olympic Messaging System (1984)

how do we elicit requirements? wide range of techniques/methods

Choosing a requirements elicitation method choice dependant on: phase in the process time & effort required expertise/skills required equipment/facilities required people required strengths/limitations of method

Interviews phase user context & early design time & effort required allegedly low, but requires careful thought & planning expertise/skills required communication + ability to remain neutral & ask non-leading questions equipment & facilities required as simple as pen/paper, although could record on audio tape/video strengths obtain individual opinions; reveal areas not thought of; opportunity to clarify, rich data limitations individual opinions; difficult to record; time-consuming to analyse (esp. if ‘open’ questions); what people say may not be what they really do/think

Interview process formulate high-level objectives (what exactly are you trying to find out?) structured? formal set of questions asked in systematic way (questions should be formulated & piloted in similar way to survey) unstructured? agenda open-ended (difficult to control) semi-structured? (‘flexible’) commence with pre-determined questions & follow up in depth with more open questions

Interview objectives, examples explore possibilities reveal areas of importance prepare for subsequent research understand user’s domain and language/vocabulary reveal unconscious motivations Le Peuple & Scane, 2003

4 main phases: Nurturing: the warm up; introductions etc Energising: outlining main topic areas Body: the peak phase of activity, asking questions, following them up etc. Closing: relaxing interviewee; summarising; subsequent actions; future planning http://www.usabilitynet.org/tools/interviews.htm Interview process

Interview guidelines Structured questions should be piloted in same way as survey questions (to avoid ambiguity etc) Conduct in friendly but business-like way if interviewing >1 person, be consistent Allow interviewee to elaborate, but also be prepared to probe (not prompt!)

Surveys phase user context & early design time & effort required high: requires careful thought, planning & piloting expertise/skills required survey design & analysis equipment & facilities required software to design questionnaire (can be as simple as WP package); software to analyse/summarise data (e.g. spreadsheet, stats package such as SPSS) strengths information relatively easy to summarise; consistency limitations time-consuming to prepare & analyse (esp. if ‘open’ questions) and requires skill to interpret ; what people reply may not be what they do or really think

Surveys 1. Many surveys carried out via questionnaire - generally produce quantitative data i.e. essentially numerical data (qualitative data is generally more subjective, emphasis on meaning rather than quantification) 2. Can be completed by: researcher, who poses the questions verbally respondent, who completes the questionnaire themselves 3. Can be supplemented with follow up interviews to ensure validity of responses

Surveys format depends on purpose but individual questions are generally open or closed - as with interviews open question can produce richer data, but more difficult and time consuming to analyse open question example “What do you like about the current system” may invite ambiguous or irrelevant responses closed question example “Do you find the current system difficult to use?”

Closed questions can take many formats Categorical (categories) ranging from inviting a simple “yes” or “no” response e.g. “Do you have access to the internet at home? Yes/No” to more complex, e.g. Surveys

Rank order, e.g. Useful for prioritising features Surveys

Scalar e.g. Likert scale (measures strength of agreement with statement) Surveys

Scalar e.g. multi-point rating scale Above illustrates design problem what does ‘frequently’ mean in this context? every hour? every day? Surveys

Skill/expertise required to design effective questionnaire Before use, they should be Checked for spelling mistakes, poor grammar etc Piloted, to: avoid ambiguous/unanswerable questions see if you are obtaining the right sort of information form estimate of resources required Presentation IS VERY important, attractive design invites response Surveys

Surveys – results Can be summarised in a table and/or chart as appropriate Quantitative data can be subjected to statistical analysis Analysis of quantitative data subject to interpretation, for example “90 % of respondents replied that they did not use the electronic phone directory” - does this mean they don’t have access to the directory, or that they prefer not to use it? (more research required)

Our Objective This is what the system designer should be aiming for

Further activities Read Ch 4 ‘User Requirements: elicitation and analysis’ in Le Peuple & Scane (2003). Have a go at the ‘end of chapter assessment’ and the ‘further reading and research’. Visit http://www.usabilitynet.org/tools/methods.htm which helps to select techniques and explains how to use them. Visit http://www.cs.ucl.ac.uk/staff/A.Finkelstein/las.html to read more about the LAS project. Write down how you think the requirements analysis process could have been improved and made more effective. Visit http://www.sun-sentinel.com/broadband/theedge/sfl-edge-n-butterflyballot,0,4467323.flash to use an animated version of the Palm Beach ballot paper

References /1 Smith-Atakan, S. (2006). Human-Computer Interaction. Thomson. Beyer, H.R. and Holtzblatt, K. (1995). Apprenticing with the customer. Communications of the ACM, 38, (5), 45-52. Beyer, H.R. and Holtzblatt, K. (1998) Contextual Design, Morgan Kaufman. Boehm, B. (1981). Software Engineering Economics . Prentice Hall. Boehm, B. (1988). A spiral model of software development and enhancement. Computer, May, 61-72. Brooks, F. P. (1975). The Mythical Man-Month: Essays on Software Engineering. Addison-Wesley. Bush, V. (1945). As we may think. The Atlantic Monthly, 176 (1) 101-108. Full text available at: http://www.theatlantic.com/unbound/flashbks/computer/bushf.htm Carroll, J. M. (Ed) (2002). Human-Computer Interaction in the New Millennium. Addison-Wesley.

Card, S. K., Moran, T. P. & Newell, A. (1983). The Psychology of Human-Computer Interaction. Erlbaum. Coad, P. and Yourdon, E. E. (1991). Object-oriented analysis. Yourdon Press. Communications Directorate, South West Thames Regional Health Authority (1993). Report of the Inquiry Into The London Ambulance Service. De Marco, T. (1978). Structured Analysis and Systems Specification. Prentice Hall. Gould, John D. (1987): How to Design Usable Systems. In: Bullinger, Hans-Jorg, Shackel, Brian (Ed.): INTERACT 87 - 2nd IFIP International Conference on Human-Computer Interaction. Hughes B & Cotterell M, (2002). Software Project management, McGraw Hill Hartson, H. R. & Hix, D. (1989) Towards empirically derived methodologies and tools for human-computer interface development. International Journal of Man-Machine Studies, 31, 477-494. References /2

Jacobson, I. et al (1992). Object-Oriented Software Engineering: a Use-Case Driven Approach. Addison Wesley. Jarke, M. Pohl, K. Sutcliffe, A. G. et al (1993). Requirements engineering: an integrated view of representation. In: Sommerville, I. & Manfred, P. (eds) Proceedings of 4th European Software Engineering Conference. Springer-Verlag. Le Peuple, J. and Scane, R. (2003). User Interface Design. Crucial. Maguire, M. (incorporating material by J. Kirakowski & N. Vereker) (1998). User-Centred Requirements Handbook v. 3.3. Deliverable D5.3, Telematics Application Project 2010. Ross, D. T. and Schoman, K. E. (1985). Structured analysis for requirements definition. IEEE Transactions on Software Engineering, 3, 6-15. Rumbaugh, J. (1991). Object Oriented Modelling and Design. Prentice Hall. Sutcliffe, A. G. (2002). User-Centred Requirements Engineering. Springer-Verlag. Woolf, M. – Political Correspondent. Ministers defeated by new hi-tech red boxes. Article in The Independent on Sunday, 31 January 1999. References /3