Human-computer interaction: lessons from theory and practice

University of Southampton Research Repository ePrints Soton Copyright © and Moral Rights for this thesis are retained by the author and/or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This thesis cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder/s. The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders. When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given e.g. AUTHOR (year of submission) "Full thesis title", University of Southampton, name of the University School or Department, PhD Thesis, pagination http://eprints.soton.ac.uk UNIVERSITY OF SOUTHAMPTON FACULTY OF BUSINESS AND LAW School of Management Human-Computer Interaction: Lessons from Theory and Practice by Saif Khalifa Rashed Alshaali Thesis for the degree of Doctor of Philosophy October 2011 UNIVERSITY OF SOUTHAMPTON ABSTRACT FACULTY OF BUSINESS AND LAW SCHOOL OF MANAGEMENT Doctor of Philosophy HUMAN-COMPUTER INTERACTION: LESSONS FROM THEORY AND PRACTICE by Saif Khalifa Rashed Alshaali This thesis explores the gap between theory and practice within the context of humancomputer interaction (HCI), specifically relating to effective implementation of HCI methods and frameworks within practice. The thesis is structured as follows: three connected but stand-alone papers are presented preceded by an introduction, and followed by a conclusion. The introduction defines HCI, discusses its history and evolution, and how it has been influenced by different disciplines. The first paper covers the usability of personalisation of Web sites and consists of three quantitative studies. The main measurements are efficiency, effectiveness, and satisfaction as indicators of usability. Two of the studies show a significant relationship between the amount of content on a page and time taken to find information. The third study shows that when users are only allowed 3-5 seconds to glance through the page (skim), the significance, found in the other two studies, disappeared. There is, however, no indication that subjective satisfaction is affected, regardless of the amount of time users take to find information in any of the studies. The second paper is a case study: a practical evaluation of how usability is implemented in commercial website design projects. It compares the difference between targeting usability issues early in the design and later in the development. The third paper conceptualises involving users early on in design projects, how this affects design projects in the context of Garrett’s (2002) framework and how it compares with current and optimal approaches. It shows that involving users early in the design process does not negatively affect time or effort and concludes by defining areas where research should concentrate to provide further evidence towards involving stakeholders in Web design through researchable propositions. Finally, the conclusion chapter summarises each of the paper’s limitations and conclusions. It links the three papers through a discussion on how they are related in addition to how this research could benefit the practitioner. Contents 1 Introduction 1.1 Aims . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Definition of HCI . . . . . . . . . . . . . . . . . . 1.2.2 History . . . . . . . . . . . . . . . . . . . . . . . 1.2.2.1 Chronology of Publications and Events 1.2.2.1.1 Foundation (1950-1980) . . . . 1.2.2.1.2 Development (1980-1995) . . . 1.2.2.1.3 Reflection (1995-2010) . . . . . 1.2.3 Research . . . . . . . . . . . . . . . . . . . . . . . 1.2.4 The Theory-Practice Split . . . . . . . . . . . . . 1.3 Structure of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 2 3 6 7 8 10 12 13 15 17 2 Personalization and Usability 2.1 Introduction . . . . . . . . . . . . 2.2 Literature Review . . . . . . . . 2.3 Research Question . . . . . . . . 2.4 Pilot Study . . . . . . . . . . . . 2.4.1 Methodology . . . . . . . 2.4.2 Results . . . . . . . . . . 2.5 Main Study . . . . . . . . . . . . 2.5.1 Methodology . . . . . . . 2.5.2 Results . . . . . . . . . . 2.6 Supporting Study . . . . . . . . . 2.6.1 Methodology . . . . . . . 2.6.2 Results . . . . . . . . . . 2.7 Discussion . . . . . . . . . . . . . 2.8 Limitations and Future Research 2.9 Methodological Reflection . . . . 2.10 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 20 23 27 29 29 30 31 31 34 35 35 35 36 39 40 41 . . . . . . . . . . . . . . . . A Usability Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3 Usability Methods and Frameworks in Commercial Projects 47 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.2 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.2.1 Usability Methods and Frameworks . . . . . . . . . . . . . . . . 50 v vi CONTENTS 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.2.2 Garrett’s Framework . . . . . . . . . . Research Problem . . . . . . . . . . . . . . . Methodology . . . . . . . . . . . . . . . . . . Case Study Background . . . . . . . . . . . . The Case Study . . . . . . . . . . . . . . . . . 3.6.1 Summary of the Case Study . . . . . . 3.6.2 Case Study Details . . . . . . . . . . . 3.6.2.1 Portal . . . . . . . . . . . . . 3.6.2.1.1 Strategy . . . . . . 3.6.2.1.2 Scope . . . . . . . . 3.6.2.1.3 Structure . . . . . . 3.6.2.1.4 Skeleton . . . . . . 3.6.2.1.5 Surface . . . . . . . 3.6.2.2 E-Services (Online Services) Feedback and Discussion . . . . . . . . . . . . Limitations and Future Work . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B Figures 4 Stakeholders in Web Design 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Current HCI Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Stakeholders in Web Design . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Stakeholders as Team Members . . . . . . . . . . . . . . . . . . . 4.3.2 Stakeholders and Garrett’s Framework . . . . . . . . . . . . . . . 4.3.3 Garrett’s Framework Time*Effort Paradigm . . . . . . . . . . . . 4.3.3.1 Garrett’s Original Time*Effort Figure . . . . . . . . . . 4.3.3.2 The Overlapping Planes . . . . . . . . . . . . . . . . . . 4.3.3.3 The Changing Planes . . . . . . . . . . . . . . . . . . . 4.3.3.3.1 Generic Scenarios . . . . . . . . . . . . . . . . 4.3.3.3.1.1 Increasing Relative Effort . . . . . . . . 4.3.3.3.1.2 Increasing Relative Time . . . . . . . . 4.3.3.3.1.3 Decreasing Relative Effort . . . . . . . . 4.3.3.3.1.4 Decreasing Relative Time . . . . . . . . Increasing Relative Effort while Decreas4.3.3.3.1.5 ing Relative Time . . . . . . . . . . . . Decreasing Relative Effort while Increas4.3.3.3.1.6 ing Relative Time . . . . . . . . . . . . 4.3.3.3.2 Stakeholders’ Scenarios . . . . . . . . . . . . . 4.3.3.3.2.1 “Nirvana” . . . . . . . . . . . . . . . . . 4.3.3.3.2.2 Current Practice . . . . . . . . . . . . . 4.3.3.3.2.3 Proposed Practice . . . . . . . . . . . . 4.3.4 Changes in the Stakeholders’ Involvement . . . . . . . . . . . . . 4.3.4.1 Nirvana . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.4.2 Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.4.3 Proposed . . . . . . . . . . . . . . . . . . . . . . . . . . 60 64 66 69 70 70 73 73 73 74 75 76 77 77 78 82 84 87 101 101 106 113 115 118 120 121 122 122 123 123 124 124 125 125 126 126 127 129 132 134 135 136 137 CONTENTS 4.4 4.5 4.6 vii Stakeholders in Web Design: Research Agenda . . . . . . . . . . . . . . 138 Stakeholders in Web Design: Practical Agenda . . . . . . . . . . . . . . 139 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 5 Conclusion 5.1 Personalization and Usability: How Google’s Personalization Scheme Affects Efficiency, Effectiveness and Satisfaction . . . . . . . . . . . . . . 5.1.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.2 Limitations and Future Research . . . . . . . . . . . . . . . . . . 5.2 Adopting Usability Methods and Frameworks in Commercial Web Projects. A Case Study Exploring the Benefits and Pitfalls . . . . . . . . . . . . . 5.2.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Limitations and Future Research . . . . . . . . . . . . . . . . . . 5.3 Stakeholders in Web Design: A Theoretical Framework . . . . . . . . . . 5.3.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Contribution to Research . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Implication for Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Bibliography 151 142 142 142 143 143 144 145 145 146 147 149 List of Figures 1.1 1.2 1.3 HCI Paradigms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How Fields in HCI relate (Saffer, 2009) . . . . . . . . . . . . . . . . . . . Timeline of Influential Publications and Events . . . . . . . . . . . . . . 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 Google News Item Block . . . Reuters News Item Block . . Weather Forecast Item Block Moving of Item Blocks . . . . Moving of Item Blocks . . . . 6 Item Blocks . . . . . . . . . 3 Item Blocks . . . . . . . . . 9 Item Blocks . . . . . . . . . . . . . . . . . 21 21 21 30 31 32 32 32 3.1 Garrett’s (2002) Framework Illustration . . . . . . . . . . . . . . . . . . 61 B.1 B.2 B.3 B.4 B.5 B.6 B.7 B.8 B.9 B.10 B.11 B.12 B.13 Initial Portal Structure Design . . . . . . . . . . 1st Phase of Changes in Portal Structure Design 2nd Phase of Changes in Portal Structure Design 3rd Phase of Changes in Portal Structure Design Final Portal Structure Design . . . . . . . . . . . Initial Portal Skeleton Design . . . . . . . . . . . 1st Phase of Changes in Portal Skeleton Design . Final Portal Skeleton Design . . . . . . . . . . . Sub Pages Skeleton Design . . . . . . . . . . . . Initial Portal Surface Design . . . . . . . . . . . . 1st Phase of Changes in Portal Surface . . . . . . Initial Online Services Structure Design . . . . . Final Online Services Structure Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 89 90 91 92 93 94 95 96 97 98 99 100 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 Human Centered Design . . . . . . . . . . . Stakeholders in Web Design . . . . . . . . . The Three Stakeholders’ Scope . . . . . . . ISO 13407 . . . . . . . . . . . . . . . . . . . ISO 13407 as SDLC . . . . . . . . . . . . . Stakeholder Typology (Mitchell et al., 1997) Stakeholders in Nirvana . . . . . . . . . . . Stakeholders in Practice . . . . . . . . . . . Stakeholders in Proposed Scenario . . . . . Garrett’s Original Time*Effort Figure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 103 105 108 109 113 118 119 120 121 . . . . . . . . . . . . . . . . . . . . . . . . ix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 5 7 x LIST OF FIGURES 4.11 4.12 4.13 4.14 4.15 4.16 4.17 4.18 4.19 4.20 4.21 4.22 4.23 4.24 4.25 4.26 4.27 4.28 Increasing Relative Effort . . . . . . . . . . . . . . . . . . Increasing Relative Effort . . . . . . . . . . . . . . . . . . Decreasing Relative Effort . . . . . . . . . . . . . . . . . . Decreasing Relative Effort . . . . . . . . . . . . . . . . . . Increasing Relative Effort while Decreasing Relative Time Decreasing Relative Effort while Increasing Relative Time Varying Relative Effort and Relative Time . . . . . . . . . Varying Only Relative Effort . . . . . . . . . . . . . . . . Varying Only Relative Time . . . . . . . . . . . . . . . . . Varying Relative Effort and Relative Time . . . . . . . . . Varying Only Relative Effort . . . . . . . . . . . . . . . . Varying Only Relative Time . . . . . . . . . . . . . . . . . Varying Relative Effort and Relative Time . . . . . . . . . Varying Only Relative Effort . . . . . . . . . . . . . . . . Varying Only Relative Time . . . . . . . . . . . . . . . . . Varying Nirvana Stakeholders’ Involvement . . . . . . . . Varying Practice Stakeholders’ Involvement . . . . . . . . Varying Proposed Stakeholders’ Involvement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 124 125 125 126 126 127 128 129 130 131 131 132 133 134 136 136 137 List of Tables 1.1 Tiers of Research According to Parush (2006) . . . . . . . . . . . . . . . 16 3.1 ISO 13407 Usability Methods in Different Stages of Development (Maguire, 2001) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Usability.gov Usability Methods for Different Stages of Development . . UsabilityNet Usability Methods for Different Stages of Development . . Usability Engineering Life-Cycle Usability Methods in Different Stages of Development According to (Mayhew, 1999) . . . . . . . . . . . . . . . Methods that are Common to at least 3 of the 4 Frameworks . . . . . . Similar Methods used in ISO 13407 and UsabilityNet . . . . . . . . . . . Advantages and Disadvantages of Usability Methods (Adapted from Holzinger (2005) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparison between Different Stages in Frameworks . . . . . . . . . . . Mapping Methods to Garrett’s Framework . . . . . . . . . . . . . . . . . Summary of Benefits and Pitfalls of Frameworks in Commercial Projects in Relation to Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 xi 51 52 52 53 54 55 56 59 71 83 Acknowledgements My mother, Maitha, you represent everything that is good in this world to me. Thank you for your prayers and support. My father, Khalifa, my idol and friend. Thank you for always believing in me even when I did not believe in myself. You are the person I aspire to be one day. My wife, Hessa, for you are truly my significant other. Thank you for your unconditional love, for being strong, and taking over much of my responsibilities so that I could focus on my studies. My children, Khalifa, Mohammed, and Maitha. I hope you grow up to be great, I love you dearly. My brothers and sister, Rashed, Saeed, and Noura. I always look up to you. My supervisors, Dr. Mel and Dr. Jonathan. Dr. Mel for always pushing me to read, write, and articulate. This would have not been possible without your diligent guidance from the start to the very end. Dr. Jonathan for always asking the hard questions. Thank you for your contribution to this thesis. I am truly grateful to my family and friends, whose support and encouragement was something I could rely on. xiii Chapter 1 Introduction 1.1 Aims The aim of this thesis is to explore the gap between theory and practice within the context of HCI, specifically relating to effective implementation of HCI methods and frameworks within practice. The thesis is structured as follows: three connected but stand-alone papers are presented preceded by this introduction, and followed by a conclusion relating to recommendations for practice and future research. This introduction defines HCI, discussing its history and evolution, and how it has been influenced by different disciplines. The first paper covers the usability of a Web technology, namely personalisation of Web sites, and consists of four studies, evaluating the standard definition of usability and issues relating to it. Second, we present a case study; a practical evaluation of how usability is implemented in commercial website design projects and compare and contrast differences between targeting usability issues early in the design or later in the development. Finally, the third paper conceptualises involving users early on in design projects, how this affects design projects in the context of a current framework and how this approach compares with current and optimal approaches. 1 2 1.2 Chapter 1 Introduction Overview Although there is a plethora of research on the role of HCI at a theoretical level of understanding, there is still fairly limited evidence on how HCI as a discipline has been embedded into the creation, design and evaluation of Web sites at an operational level (Carroll and Rosson, 1992; Wixon, 2003). Such a lack of evidence may be because of the complexity of the whole arena of HCI, its multidisciplinary nature, and its complexity at the operational level. This complexity and multidisciplinary nature are two related issues. Due to the varied nature of the history of the theoretical foundation in HCI, such theories from psychology, human factors, ergonomics, and design were the foundation upon which HCI started. Commencing its life from such diversity and focus of research, it is not surprising that there is often little consistency in the application of HCI. The complexity of HCI at an operational level arises from the fact that it exists in a complex and fast moving environment where time and budget are the two key drivers in any commercial project. This I believe has resulted in a gap between theory and practice. In this thesis, the aim is to examine this gap through a more concrete lens. Specifically this thesis aims to analyse how usability evaluation is undertaken in commercial projects, explore the barriers that affect usability work in commercial projects, conceptualise how to involve users earlier in design projects, and explain the benefits of this in each of the project stages as well as the organisation’s return on investment. To be able to acquire a better understanding of HCI, however, a definition coupled with how it relates to other disciplines, together with a brief overview of the history of the discipline is required. This, of course, cannot be done without pointing out important milestones of personal computers, graphical user interfaces (GUI) and the Internet. Next, influential research within other disciplines and HCI will be surveyed. This will be followed by a review of research within HCI domains that has helped in the development of theories, models, and frameworks. Finally, a discussion about the gap between theory and practice will be presented. Chapter 1 Introduction 1.2.1 3 Definition of HCI Human-computer interaction (HCI), which also used to be known as “interaction design” and “knowledge media design” (Baecker, 2008), is a discipline that appertains itself with how computers and humans interact. There are several definitions of HCI. For example, Myers (1998, pg. 974) defined HCI as “the study of how people design, implement, and use interactive computer systems and how computers affect individuals, organizations, and society”. Booth (1989, pg. 4) offered an oversimplified definition of HCI: “the study of interactions between humans and computers” after which he offered five more in-depth definitions that target different areas within HCI: research into organisational impact, research into design, research at a task level, research into matching models, and research into interactional hardware and software. The most encompassing definition however is “Human-computer interaction is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them” (Hewett et al., 1992, pg. 5). Being an applied field, HCI was, and still is, influenced by many different disciplines and fields, and from both sides: the human and the computer. Disciplines such as psychology, computer science, human factors and ergonomics have had a substantial impact on HCI. General principles and theories of these domains were utilised in designing and developing more usable interfaces. This was particularly true in the early days of HCI when there was a lack of domain theory. HCI is a paradigm concerned with both humans and computers, how they interact with each other and how design focuses on solving practical problems in order to enhance the user experience. Hewett et al.’s (1992) definition is general enough to include other means of human-computer interaction mediums such as virtual reality, wearable computers, and others, but specific enough to target the main issues, namely design, evaluation, and implementation (Figure 1.1). These issues are notably important in the Internet and World Wide Web era. New interfaces and design ideas are rapidly emerging on the Web and in technology generally and there are vast differences between them. Graphic and website designers invent creative ways to display information and content, sometimes focusing too much on their own vision rather than facilitating accessibility for users. This is particularly true with personal Web sites where the owner is the artist. In such cases user experience, usefulness, accessibility, and usability of the website are subjacent to the vision of the designer. 4 Chapter 1 Introduction Conversely, when the website is business-related, targeting e-commerce or a more prominent online presence to the largest audience possible, then usability and user experience becomes extremely important. This is because in personal Web sites, the owner could discard sections of the population for the sake of the vision they have, however, businesses are likely to suffer great loss by following the same idea. HCI is also related to several other fields including interaction design, interface Human design, usability engineering, visual design, and information architecture (Saffer, 2009, Figure 1.2). All of these fields, according to Saffer, fit within a larger construct called ‘user experience design’. Saffer, however, does not consider the whole field of Figure 1.1: HCI Paradigms HCI to fit within what he describes as ‘user experience design’ even though he agreed that the overall field of user experience is related to different disciplines such as mechanical engineering, electrical engineering, architecture, and content creation. However, it is believed by this author that HCI is more involved with other disciplines such as interface design and visual design. This is clear from the methods available within HCI to design, evaluate, and implement interfaces such as design guidelines and standards (Maguire, 2001), heuristics evaluation (Chattratichart and Lindgaard, 2008), and rapid prototyping (Bevan, 2003), respectively. Other researchers (e.g., Karvonen et al., 2010) attempted to differentiate between the related fields through a framework of five dimensions: the scope of target technology, design vs. research orientation, technology-driven vs. human-driven orientation, intuitive vs. scientific practice in analysing humans, and meta-scientific rigour in investigating users. Two of these dimensions are of particular interest: design vs. research orientation and technology-driven vs. human-driven orientation. The design vs. research orientation dimension discussed what is the purpose of the research and whether it had any practice-oriented goals. Research, the authors argued, has less practice-oriented goals (Benbasat, 2010; Carroll, 1997). The 5 Chapter 1 Introduction technology-driven vs. human-driven orientation dimension discussed how the product is built; either based on technical solutions or human requirements. Generally, the products and systems are built according to technical solutions with limited input from users and therefore result in poor products. Benbasat (2010) differentiated between design sciences and HCI (in information systems) as a difference of focus. Design sciences, which is where most of Saffer’s (2009) related fields fall, focus on the attributes of design itself and how to do design, well. HCI, on the other hand, concentrates on the evaluation of the effectiveness of the different designs from a managerial perspective rather than demonstrating the quality of that design. This is an important distinction - the interest in design as an artifact, or the interest in the design as an instrument to advance managerial/organisation goals (Benbasat, 2010). USER EXPERIENCE DESIGN Signage MECHANICAL ENGINEERING ARCHITECTURE INFORMATION ARCHITECTURE INDUSTRIAL DESIGN ELECTRICAL ENGINEERING CONTENT CREATION (Text, Data,Graphics) INTERACTION DESIGN VISUAL DESIGN Ubicomp Controls Interactive Environments Info Viz USABILITY ENGINEERING HUMAN-COMPUTER INTERACTION Navigation INTERFACE DESIGN Figure 1.2: How Fields in HCI relate (Saffer, 2009) The discussion of HCI through its definition and how it is related to other disciplines showed that there is confusion as to what HCI really is. This author believes that this is caused by how HCI actually started and evolved. For a better overview of HCI, a brief history is presented in the next section to show how HCI evolved and why there is such discrepancy in understanding HCI as a discipline. 6 1.2.2 Chapter 1 Introduction History The start, and history, of HCI is not entirely clear due to the fact that it started in an ad hoc manner more than 50 years ago. Additionally, there are many different perspectives that researchers have adopted when discussing the history of HCI. Some researchers, for instance, have concentrated on the technological inventions within the context of HCI (Myers, 1998). Myers argued that virtually all advances and concepts now in use in computers and the Internet had their beginnings in universities. One example is the ubiquitous graphical interface which was prominent in Windows 95. This was based on the Macintosh, emerging from research done in Xerox SPARC, which in part was based on early research undertaken by Stanford Research Institute, then called Stanford Research Laboratory, in the 1960s. Others (e.g., Grudin, 2005) provide a view of HCI in relation to other influential disciplines such as Management Information Systems and Human Factors and Ergonomics. Grudin discussed the three roles in early computing: operation, management, and programming. He detailed the chronology of the evolution of computers through these three roles and the transformation of the computer users from non-discretionary to discretionary users. Baecker (2008) presented a more general view of the evolution of HCI. The author not only introduced influential publications in HCI but also pivotal events such as the first conference in HCI which was held in 1982. Shackel (2009) undertook an in-depth analysis of the field through discussing where the influence first emerged, how it evolved through breaking it down into three sections: the foundations of HCI, the development of HCI as a field, and continuities from the past and perspectives into the future. Shackel also offered a considerable amount of general historical information such as when the first publication directed towards HCI was published. The author, however, pointed out that the paper has more of a UK/EU emphasis than a US one. This historical overview will concentrate on influential publications and events within HCI and publications that discussed HCI within other disciplines. The overview will also show how the publications moved from other disciplines of research to focus, for the most part, on HCI. Chapter 1 Introduction Figure 1.3: Timeline of Influential Publications and Events 7 8 1.2.2.1 Chapter 1 Introduction Chronology of Publications and Events Figure 1.3 illustrates the chronological order of influential publications. It also shows the timeline for important technological advances such as the invention of the personal computer and the invention of the World Wide Web. The figure also maps the transition of how publications that are considered in the realm of HCI moved from being published in other venues to being published predominantly in HCI through the subscript next to the publication. The discussion of the history will adapt two of the three sections of Shackel’s (2009) classification of the history of HCI. He defined the first two eras: foundation (1950-1980) and development (1980-1995). Shackel’s third section entitled “Continuities from the past and perspectives into the future” discussed where he thinks computing and HCI should head. This author, however, created an alternative section called reflection (1995-2010) to highlight research that focused on the direction HCI needs to take and why this is important. This section will include other reflectional research including Shackel’s. 1.2.2.1.1 Foundation (1950-1980) Earliest work in HCI is considered by many such as Shackel and Richardson (1991) and Grudin (2005) to be “Ergonomics in Computers” by Shackel in 1959. The paper disclosed a redesign effort of the EMIAC II with emphasis on the redesign decisions. Contrary to previous redesign attempts reported, there was observation of the user while using the device, and user input in the redesign. In his paper, Shackel thrashed out the essence of HCI: “ease of operation and avoidance of operator [read user] mistakes” (1959, pg. 36). The author gave these two issues - ease of operation and avoidance of operator mistakes - top priority over other issues such as loss of speed or machine inaccuracy. The author shifted the requirements from the hardware to that of the user. Licklider’s (1960) “Man-computer symbiosis” followed a year later where he explained the requirements of human-computer interaction. He detailed the requirements for a successful “symbiosis” [read interaction] between user and computer. Licklider listed what he considered to be the main barriers to the integration. First, the speed of the computer is much faster than that of the user. Second, language was a Chapter 1 Introduction 9 major barrier in communication even with strides of advancement in the area. Finally, input and output equipment was the most undeveloped of the three. Licklider had essentially described artificial intelligence. After research identified what HCI was and what its requirements were, its application came next. Engelbart (1962) wrote a comprehensive technical report describing how to “increase the capability of a man to approach a complex problem situation” (p. 1). The author explained the framework of how technology should be used to aid an individual in understanding complex situations and solving them. He detailed what he calls “augmentation means” (p. 9) by mentioning four classes of technology: Artifacts, Language, Methodology, and Training. These four classes form the basis upon which humans rely to extend their capabilities. The author then discussed how these classes fit and create general knowledge. Engelbart and his team effectively invented multi-window display and WYSIWYG word-processing, among other high-impact inventions (Shackel, 2009). In 1963, Sutherland (1963) presented many of the ideas that are widely used in current-day personal computers through his PhD research at the Massachusetts Institute of Technology through Sketchpad. Sketchpad was a program that allowed the user and the program to collaborate to solve a problem visually. Sketchpad laid the foundations for Computer-Aided Design (CAD) software and had to wait for hardware to improve to be able to truly utilise it. A couple of years later, in 1965, Nelson coined the word “hypertext” (Nelson, 1965) and theorised about a system that would be able to provide much more information through the interconnection of pieces of information such as text and pictures. Only a small part of Nelson’s vision was captured through the invention of what is currently known as hypertext. Hypertext, as a technology, was invented in 1968. In 1969, and through computer programming, Kay published his PhD thesis entitled “The Reactive Engine” (Kay, 1969) which aimed at developing an interactive dialogue through the creation of a computer programming language called “FLEX” which is an acronym for “Flexible Extendable”. He based his thesis on the notion that form and abstraction, in relation to people requiring mental models of the environments before it could be comprehended, is the basis for creating the engine. Kay’s language and depictions paved the way for the creation of many currently used technologies such as bitmap screen, overlapping windows, and icons. In 1970, the first book in psychology regarding psychology methods in programming was published by Sackman (1970) was based on an extensive amount of quantitative data. In 1972, however, what is considered a more influential psychology book (Baecker, 1995) was 10 Chapter 1 Introduction published by Weinberg a couple of years later. Weinberg’s (1972) book, “Psychology of Computer Programming”, had a single purpose; focusing on computer programming as a human activity. It discussed programming through three broad issues: as a human performance, as an individual activity, and as a social activity (Baecker, 1995). In 1973, Martin moved the discussion from programming and the communication between the person and the user through his book “Design of Man-Computer Dialogues” (Martin, 1973), which is one of the first publications that concentrated on the design of dialogues in human-computer interaction. It covered different aspects of dialogue in a terminal computer and issues with designing for different types of users. In Kay and Goldberg (1977) paper, the authors developed the notion, through the invention of smalltalk language and its implementation on the Dynabook, that computers will not be used for their capacity to compute but rather as a medium for communication. The smalltalk language offered many of the ideas of today’s operating systems. 1.2.2.1.2 Development (1980-1995) In 1983, and in the first effort to produce domain theory in HCI, which is required to establish the discipline as an academic one (Corley and Gioia, 2011), Card et al. (1983) presented what is considered to be one of the first domain theories in HCI: the Goals, Operators, Methods, and Selection rules (GOMS) in their book “The Psychology of Human Computer Interaction”. GOMS is a predictive model based on the human information processing theory. It allows designers to predict the amount of time users will take to complete a task. In the same year, Shneiderman (1983), as with Card et al. (1983) diffused the importance of the computer as a device and rather concentrated on how it can help users with their tasks. This was a change of thought in HCI because of the focus turning to discretionary use (Grudin, 2005). Card et al. concentrated on graphical user interfaces rather than text-only interfaces, while Shneiderman focused on visual representation and direct manipulation of items presented on the screen of a computer. Also in 1983, Edward Tufte, while a statistics professor at Yale University, published a revolutionary book called “The Visual Display of Quantitative Information” (Tufte, 1983). He pointed out many of the mistakes in representing data and explored new ways to increase its effectiveness and efficiency. Even though Tufte’s book was not targeted towards interface design, his ideas are still used by designers today. However, after 26 years of Shackel’s first definition of HCI, the world was still finding it difficult Chapter 1 Introduction 11 to implement usability-oriented design as pointed out in Gould and Lewis’s (1985) paper. They targeted the issue by providing three basic principles to create a simple and easy to use computer system: early focus on users, empirical measurement, and iterative design. The paper, as the authors describe it, is both theoretical and empirical: theoretical through the proposition of the three principles and empirical through the presentation of data regarding the three principles in relation to how designers perceive them, the arguments designers provide for not using these principles and the answers to these arguments through examples of where each of the principles has been used successfully. In the same effort but on a bigger scale, Norman and Draper (1986) published a book entitled “User Centered System Design; New Perspectives on Human-Computer Interaction” which is a seminal work that focused on how systems can aid users with their mental load. Their focus was towards a more cognitive basis and they discussed the trade-offs users and designers are faced with when designing interactive systems. Norman and Draper created the concept of user-centred design within the book. Also, in the effort to understand how users act, Suchman (1987) introduced the term ‘situated action’ in her book “Plans and Situated Actions: the problem of human-machine communication” as an alternative approach to purposeful action in response to recent developments in the social sciences. The purpose was to explore the relationship between knowledge of a certain situation and the action that is generated in response. Suchman took a different approach to HCI than earlier work that concentrated on the psychology and cognitive aspect and rather focused on the sociological aspect of HCI. In 1989, managerial aspects and the efficiency of computer systems being used attracted more focus. In an effort to define how users (employees in Davis’s Paper) use computer systems, Davis (1989) defined usability within information systems through the Technology Acceptance Model (TAM) as two factors: perceived usefulness and perceived ease of use as the precedents for intention to use and actual use. TAM is an adaptation of the theory of reasoned action (Ajzen and Fishbein, 1973) which has its basis within behavioural sciences. In 1992, HCI had grown over the preceding 33 years and many schools had either courses or programmes that taught HCI. These courses and programmes, however, were not unified, which prompted the Association of Computer Machinery (ACM) to create a curriculum development group in 1992 (Hewett et al., 1992). This was an effort to produce a curriculum of recommendations for education in human-computer 12 Chapter 1 Introduction interaction, and it aimed to be beneficial to anyone trying to either create a single course or a programme in HCI. In 1994, and with the desire to add to domain theory within HCI, Newell (1994) published a book entitled “Unified Theories of Cognition” that tried to formulate the Unified Theory of Cognition (UTC) through the introduction of the concept of Soar, a symbolic cognitive architecture, in an Artificial Intelligence (AI) context. Soar, however, was not intended to be the unified theory of cognition but rather an example of the UTC to help focus on the bigger picture rather than concentrating on small isolated parts of the theory. With the increase of GUI use and the invention of different operating systems, many interfaces and tools were then developed to build interfaces. In an effort to categorise available interface tools, and similar to what Martin did back in 1973, Myers (1995) undertook an extensive survey of user interface tools and defined the different types and classifications. Myers discussed why some tools failed whilst others succeeded. The author then discussed research issues and the need for new programming languages, increased depth, increased breadth, and end-user programming and customisation. 1.2.2.1.3 Reflection (1995-2010) HCI has gone through a great deal from its foundation and development eras, and since the mid nineties researchers in the field have had to stop and reflect as to where HCI came from, where it is currently and what the future holds for the discipline. The four notable papers (Baecker, 2008; Benbasat, 2010; Grudin, 2005; Shackel, 2009) attempted to undertake this task from relatively different perspectives. Grudin (2005) studied three foci of HCI: human factors and ergonomics (HF&E), information systems (IS), and HCI. He provided an in-depth analysis of the history of HCI through these three foci starting from as early as 1911. He argued that despite the common focus of these three disciplines, only minor interaction between them has taken place. He reasoned that the lack of interaction was because of the different academic approaches to the three fields: HF&E, IS and HCI. The first two were well established when the latter emerged. In addition, and still within an academic context, the first two fields consider conferences to be the venue for work in progress whilst HCI was considered to be the final destination of the academic work. Grudin supported this argument with a table Chapter 1 Introduction 13 showing that whilst research in HF&E and IS has a relatively high acceptance rate (80% and 60% respectively), HCI has a much lower acceptance rate (20%). This issue is also shown earlier through Baecker and Buxton’s 1987 work; they argued that most of the influential publications in HCI were in conferences, technical reports, and other venues which are harder to access. Another reason is the language and nomenclature that is used across the three disciplines Conversely, Baecker (2008) reflected on the history of HCI through seven phenomena: hypertext, direct manipulation, GUIs, design, usability testing, workplace context, and other HCI (research that did not fit comfortably with the other phenomena), which he considered to be the starting point for researchers. He also argued for the urgent requirement of a comprehensive history of HCI. Shackel (2009), as mentioned earlier, approached the history of HCI by sectioning it into three main eras: foundation (1950-1980), development (1980-1995), and future. Through an IS route, Benbasat (2010) focused on the challenges of HCI and the direction of future research. He argued that HCI research should be relevant to practice in order to be beneficial to both research and practice. He also discussed how HCI research (in IS) can differentiate itself from other disciplines such as computer science, information science, and marketing. He pointed out that HCI should be concerned with how it can use the design as an instrument to advance an organisation’s goals and objectives rather than becoming embroiled in the design itself. 1.2.3 Research As mentioned earlier, many fields played a role in HCI development. Rogers (2005) described the contributions made to HCI from different perspectives such as ecological, activity theory, external cognition, distributed cognition, situated action, ethnomethodology, hybrid, and overarching approaches. The research here is categorised as work that emanated from either outside HCI or research from within HCI. In the former category, four pivotal areas of research are presented: Fitts’ Law, the Hick-Hyman Law, Guiard’s Model, and Task Analysis. In the last category, two pivotal research areas are presented: GOMS and KLM. In research that emerged from outside HCI, Fitts’ Law (Fitts, 1954; Fitts and Peterson, 1964) is “one of the most robust and highly adopted models of human movement” (Carroll, 2003, pg. 35). It is a predictive model developed through experimental 14 Chapter 1 Introduction psychology that predicts the time required to move to a target area depending on the distance and size of the target area. The drawback of Fitts’ Law is since it is a one-dimensional model it does not map properly to two-dimensional target acquisition. Researchers have been working on extending Fitts’ Law to incorporate advances in interfaces and include a second dimension. For example, MacKenzie and Buxton (1992) included a second dimension with hand and head movement, and Jagacinski and Monk (1985) enhanced the model of bivariate pointing, while Accot and Zhai (2003) included a third dimension. The Hick-Hyman Law (Hick, 1952; Hyman, 1953) is similar, in essence, to Fitts’ Law, in the sense that they are both based on information theory, both address rates and limits of human performance, and both received substantial support in research (Seow, 2005). The Hick-Hyman Law, however, was more difficult to implement and therefore received less less in the HCI realm (Seow, 2005). Guiard’s 1987 Model, on the other hand, is a descriptive model detailing the division of labour between a person’s hands. The rationale behind the model was that it has been found that not only are people right-handed or left-handed, but they also use their hands differently (e.g., Kelso et al., 1979; Porac and Coren, 1981; Wing, 1982), and the lack of understanding regarding how people assign tasks to hands. Task Analysis was developed “in response to the need for a rational basis for understanding the skills required in complex non-repetitive operator tasks” (Diaper and Stanton, 2003). It deals with the decomposition of a task done by a user into smaller pieces to better describe and understand the task. The level of granularity of the decomposition is left to the analyst depending on the complexity of the task and how familiar the elements that make up the task are. An example can be given regarding clicking on a hyperlink. The Task Analysis can be as simple as: 1. The user move the mouse pointer over the link. 2. The user clicks on the hyperlink. This could also be decomposed further if required as such: 1. The user grabs the mouse. 2. The user moves the mouse pointer. 3. The user stops the mouse pointer over the hyperlink. Chapter 1 Introduction 15 4. The user presses the left-mouse button, clicking the link. Task Analysis, however, is a general term. Hierarchal Task Analysis (HTA) is a more defined method of Task Analysis that allows for the creation of a hierarchy in order to group tasks into plans: a group of tasks that completes a certain function. In research that had its origins in HCI and was developed by Card et al. (1983), GOMS is described as a Task Analysis technique that incorporates cognitive science with a grounding in Human Information Processing (Proctor and Vu, 2006) and “has been one of the most widely known theoretical concepts in HCI ” (John and Kieras, 1996a). GOMS, in essence, is a decomposition of the user’s interaction with the computing system to its basic steps, or actions. There are several other derivatives of GOMS including Keystroke-Level Model (KLM), CPM-GOMS, Natural GOMS Language (NGOMSL), and CPM-GOMS. KLM is a restricted model of GOMS. This method was presented in Card et al.’s 1983 work along with the original GOMS. The main difference between GOMS and KLM is that the latter does not include Goals or Selections rules but only the sequence of Operators and Methods to accomplish a task making default assumptions. This makes the model easier to implement but it does attract criticism in that it is not suitable for long experiments and the fact that it measures expert usage rather than novice ones. 1.2.4 The Theory-Practice Split HCI has focused on rigorous research and conceptualisation to establish itself as a legitimate academic discipline, which is common with relatively new practical fields. Since the early 1950s for example, management sciences have moved away from anecdotal evidence and eclecticism (Corley and Gioia, 2011). This, in management sciences as well as HCI, however, created two ideologies: one that is concerned with rigour and the other with relevance. This is not to say that all of the rigorous research is without relevance to practice, and that relevant research (to practice) is without rigour. Furthermore, the overwhelming expansion of HCI as a discipline has now created a wealth of knowledge that has forced practitioners to isolate themselves from sections of the discipline’s original foundation (Carroll, 2003). A good portion of research has been done, in HCI and other fields, which shows a discrepancy in how 16 Chapter 1 Introduction Tier 1 2 3 4 Objective Examine usability of a given project Explore/compare various products and contextual impact on usability Derive guidelines that can be applied to other designs Question Does it work for the user? When, where, compared to what does it work? Outcomes Information on the usability of a product Comparative information on products and contexts How should it work with other similar designs, contexts, and systems? Derive theoretical im- Why does it work? plications to better understand human behavior with interactive systems Design guidelines based on behavioral research methods General principles, models, and/or theories accounting for various phenomena Table 1.1: Tiers of Research According to Parush (2006) theorists and practitioners view academic research (e.g., Buie et al., 2010; Lee et al., 2002). Research more recently describes the ’gap’ between theory and practice in HCI and ways to bridge that gap (Benbasat, 2010; Dray, 2009; Grudin, 2005; Parush, 2006; Wixon, 2003; Carroll, 2003). These research studies have highlighted the difference in approach between practitioners and theorists in formulating the problems and finding solutions. Parush (2006), for instance, described a 4-tier structure each corresponding to a different type of research as shown in table 1.1. Dray (2009) went further in the differentiation between theory and practice of HCI when considering rigour; she stated that for research to be rigorous, the product, as a whole, cannot be investigated; but rather an aspect of that product is evaluated in an incremental fashion. Research can also take the direction of describing the relationships between abstracts of a product. Practitioners, on the other hand, do not need to extend the understanding between different abstracts for the value of the knowledge but as a mean to enhance the product development in some manner. This knowledge they acquire should benefit the development directly. They also focus on the development of a whole product rather than a small aspect of it. Later enhancement of the product might look at how specific aspects can be improved but that rarely happens when the initial product is being developed. Wixon (2003) Chapter 1 Introduction 17 expanded on what actually happens in the applied usability work. He argued that in product development, a product is built to meet certain criteria, whilst balancing between trade-offs within a given context and constraints. He also pointed out that in commercial projects, schedule and resource dominate all discussions of design, including being the definitive argument one rather than another design decision. Benbasat (2010) discussed how any research, and especially HCI, should be “new, true, interesting, and relevant (to practice)” (p. 16). He argued that any HCI research should be coupled with a design component. The coupling of the design component in HCI research results in a more practical relevance and unique identity. The focus here will be on implementing HCI and usability methods within practice, which has been studied by many researchers (e.g., Blandford et al., 2006; Furniss et al., 2008). To be able to bridge the gap, however, the author believes this thesis has to be extended in order to better understand practice; how HCI and usability work are being implemented into natural contexts with the focus on time and budget constraints. Before this can be done, an example of how usability is being implemented has to be provided. This is explained in this thesis through chapter two. Next, providing evidence on how Web design and development is happening in practice and the barriers of implementing usability analysis is articulated through chapter three. Finally, conceptualising how to involve users from the commencement of website design projects and how such involvement affects the project is explained through the adoption of Benbasat’s 2010 notion that design should be undertaken to promote and advance organisational goals. Adopting Benbasat’s notion allows for two things: the implementation of usability work, and the definition of the benefits to the organisation. 1.3 Structure of the Thesis The thesis is structured as follows. Chapter Two, entitled “Personalisation and Usability: How Google’s Personalisation Scheme Affects Efficiency, Effectiveness, and Satisfaction”, tried to measure how personalisation affected usability through measuring efficiency, effectiveness, and satisfaction. Findings showed that whilst efficiency was related to the amount of information on a Web page, satisfaction did not fall. It also discussed the fit of the usability factors: efficiency, effectiveness, and satisfaction to personalisation Web sites. Chapter Three, entitled “Adopting Usability 18 Chapter 1 Introduction Methods and Frameworks in Commercial Web Projects. A Case Study Exploring the Benefits and Pitfalls”, focused on the difficulties experienced in implementing usability methods into a ’real-world’ situation where limited resources and tight deadlines are the norm. It utilises Garrett’s 2002 framework as a basis for implementing usability work within the commercial project. The paper compares how doing usability work in two distinct instances at the beginning of design (through the portal) and at a later stage (through the online services). The case study showed decreased implementation of usability recommendations as the project progresses and critically evaluated Garrett’s framework’s benefits and shortcomings in a commercial project. Chapter Four, entitled “Stakeholders in Web Design: A Theoretical Framework”, evaluates the current and available HCI frameworks, used in Web design. This paper analyses the effect of adding different stakeholders at different stages of the design and theorised how such additions would affect the project as a whole in terms of effort/time and bottom-line ROI. It compares three possible alternatives; generic/nirvana, current practice, and an alternative approach which proposes that the user can be included from the beginning of the design process without greatly affecting each plane of the overall project progress and without using an excessive amount of relative effort. Finally, Chapter Five entitled “Conclusion” summarises each chapter’s contribution and limitations. It also provides direction for future research. Chapter 2 Personalization and Usability: How Google’s Personalization Scheme Affects Efficiency, Effectiveness and Satisfaction Abstract: Personalising a website seems likely to affect its usability, especially when a user changes the amount of content of a Web page as part of the personalisation process. This paper explores the effects of explicit Web page personalisation, where the user is the main source of personalization for both the interface and the content, on usability. Specifically, the studies reported here examined what effect changing amount of content to a page via personalisation had on usability. There are a total of three studies. The pilot study measured efficiency. It found a significant relationship between visual search time and the amount of information on the page. The main study measured efficiency, effectiveness and subjective satisfaction and also showed a significant relationship between the amount of content on a page and time taken to find information. The supporting study showed that when users were allowed three to five seconds to glance through the page (skim), the significant effect of the amount of content on the page on visual search time diminished. There was, however, no indication that subjective satisfaction was affected, regardless of the amount of time users took to find information in either study. 19 20 2.1 Chapter 2 Personalization and Usability Introduction With ever-increasing competitiveness in e-commerce and online presence, website personalisation has become one of the major ways to encourage repeat visits, improve customer satisfaction (Chellappa and Sin, 2005; Kwon et al., 2010) and increase e-loyalty (Chang and Chen, 2008; Koren et al., 2009). Broadly, Web personalisation leverages personalisation technologies to provide the relevant content in the correct format to the right person at the right time. The objective is to provide customised services and to maximise business opportunities (Tam and Ho, 2005). Customer information is usually either previously obtained or provided in real time (Wu et al., 2003). Previously obtained information can include demographic information, hobbies, and interests. This is usually gathered when a user registers for an account at a website. Real-time customer information is gathered while the user is browsing a website. The website monitors what the user is looking at and clicking on and makes comparisons with the behaviour of other users. Such data are used to provide recommendations such as similar products that other users have looked at or purchased. Systems that do this are often referred to as recommender systems (Koren et al., 2009). Personalisation can be either implicit, explicit, or a combination of both, and can be directed towards the interface, the content, or both. Additionally, owners of Web sites usually add the capability of personalisation to their Web sites in order to aid users in achieving their goals and needs; goals that are usually measured by attractiveness, control, efficiency, affect, learnability, and satisfaction (Kuan et al., 2005). Personalisation is not only used on Web sites; intranets are also offering personalised pages for their employees (Bellas, 2004; Urbach et al., 2009). Personalisation tools are one of the ways in which technology is used to give users the capability to manipulate Web pages to fit their needs (Wu et al., 2003). Capabilities include changing the layout of the page and adding short-cuts to information that is most interesting to the user. Providing personalisation capabilities is expected to continue to grow as institutions and organisations provide more practical access to information which is quickly found and is useful to the person concerned. Such provision is beneficial, as it avoids the user having to look though a whole range of irrelevant content (Bhide et al., 2007; Harris and Lessick, 2007), thereby perhaps enriching the user’s experience. In 1996, Yahoo! was one of the first portals to incorporate personalisation in users’ 21 Chapter 2 Personalization and Usability pages. This enabled users to add and modify specific sections of interest, or “item blocks”, on their personal pages (Manber et al., 2000). These item blocks can include hundreds of items such as news, stock prices, and weather forecasts. A majority of users, however, according to Manber et al., did not use the functionality due to one or more of three reasons: 1. Good default personal page, 2. Difficulty of using the personalization tool, and 3. Lack of need for complex personalization. In addition, the authors pointed out the lack of standards by which to summarise content found elsewhere on the internet at the time, so users were limited to only choose from the content and services Yahoo! offered. However, over 14 years later, users are no longer Figure 2.1: Google News Item Block limited to one specific provider of content because of the invention of Really Simple Syndication (RSS) (Ward, 2007). In May 2005, Google Figure 2.2: Reuters News Item Block launched a service that allowed users to personalise the default Google homepage (http://www.google.com/ig). This service enabled users to add content to their default homepage and also allowed for the direct manipulation of the layout of the page after content was added to it. Figure 2.3: Weather Forecast Item Block The service is made up of what are called “item blocks” similar to a typical window in an operating system, albeit smaller. The system allows the user the capability to move these item blocks around the Web page. In addition, the service allows the user to add any RSS feed to the homepage, either from internal (e.g. Google News, Figure 22 Chapter 2 Personalization and Usability 2.1), or external sources (e.g. Reuters, Figure 2.2). Users can also add widgets, such as weather forecasts (Figure 2.3), a calendar, or their own email inbox. At the time Google launched their personalisation service, other sites such as Yahoo! and Microsoft, were already offering some sort of personalisation service. Yahoo! offered its personalisation service through my.yahoo.com. Yahoo’s personalised service design, however, was ranked low in aesthetic design (Pousman and Stasko, 2006). Microsoft offered its own personalisation service through my.msn.com. Other sites offered personalisation services, not for the sake of the technology, “but as a strategy to make their naive users’ experience richer ” (Bhide et al., 2007, pg. 1) and hence gain market share. These Web sites include netvibes.com, which features more aesthetically pleasing elements when customising, moving, adding, or deleting item blocks. For instance, when a user moves an item block, it collapses into a mini-block until moved to the new location. When released, the item block expands back to the previous size. Overall, however, personalisation services tend to be similar when it comes to displaying content; they all use boxes to hold content and a title bar that typically include “edit” and “x” (close) buttons. Even Web sites that do not fall into the category of portals and search engines have started to include personalisation services. One of these Web sites is WordPress. This website is a “state-of-the-art publishing platform with a focus on aesthetics, Web standards, and usability” (Wordpress, 2011), more commonly known as blogging. The WordPress website allows both owners and users (through plug-ins) to personalise how they see and interact with the website. The owner is allowed the capability to change how and where the items on the Dashboard are displayed. The users are able to see specific posts, view the site in their own language, and take advantage of a number of other features. Such personalisation services offer an innovative way to allow the user to include content that is of specific interest to the user rather than just generic content; however, there is a lack of research on how changing the amount and layout of content effects the usability of a Web page. The research described in this paper aimed to ascertain the relationship between the amount of content and what affect it had on other usability factors; namely efficiency, effectiveness and satisfaction. This paper is structured as follows: First, a literature review examining three relevant Chapter 2 Personalization and Usability 23 disciplines is presented. These disciplines are personalisation as a technology, human-computer interaction, and visual search as part of cognitive psychology. Second, the research question is presented and the hypotheses defined. Third, the methodologies of each study are presented followed by the results of that study. Finally, discussion, limitations, and conclusions are presented. 2.2 Literature Review The main focus will be personalisation and what affect it has on usability. First, personalisation is defined and an explanation given as to how it is implemented. Next the advantages and disadvantages of the technology in relation to the user experience as well as the various methods of implementation used will be discussed. Then, the definition of usability and how it relates to personalisation will be presented. Finally, visual search, which is a subsection of cognitive psychology that deals with how people generally search for information, is discussed as a measure of efficiency; one of the factors of usability the study focused on. With the explosion of sites on the Internet, personalisation of Web sites has become an important tool to overcome information overload Shahabi and Chen (2003), increase switching costs and satisfaction (Chellappa and Sin, 2005), and make the Web a friendlier environment (Pierrakos et al., 2003). Wu et al. (2003) define personalisation on the Web as the use of technology and customer information to tailor interactions, content, and navigation for individual users. Personalisation can be either implicit, explicit, or a combination of both. In implicit Web page personalisation, the computer is the main source of personalisation for both the interface and the content. This allows the website to provide the user with information it categorises as relevant, depending on the information it captures or that which is provided by the user whilst browsing the website. The more a person uses the website, the more the website learns about that person, and the more the implicit personalisation becomes relevant. A good example of this is Amazon.com, where one is offered suggestions of products based on the past choices of other users of the website. Such suggestions are presented after viewing or buying the current product of interest and so avoid unnecessary interference to the user. 24 Chapter 2 Personalization and Usability In explicit personalisation, the user is the main source of personalisation of both the interface and the content. This type of personalisation allows the user to change the layout of the website and its displayed content. The user is provided with sources of content and widgets which they can incorporate into the website. Additionally, a user can incorporate additional content from any website that offers its content using RSS technology. Google’s personalisation, where the user plays an active role in personalising the page to their preference, falls into this category. There are some inconsistencies in the use of terms relating to personalisation. For instance, Treiblmaier et al. (2004) refer to implicit personalisation as “personalisation”, but term explicit personalisation “customisation”. Customisation, from their point of view, is “a user-initiated and user-driven process. It uses adaptable system-components which users can tailor to their specific needs” (p. 2). In this paper, however, the term (explicit) personalisation will be used to describe the actions users take to personalise a Web page. Personalisation technologies are not without their drawbacks. Criticism is targeted towards personalisation because the technology collects data about the user (in the case of implicit personalisation, possibly without the user’s awareness) and, in conjunction with the personal information provided by the user, poses a risk to privacy and security. These technologies can also collect data such as clicks, time spent on a specific product page, and previous products a user has looked at and compare such data with that of other users to offer recommendations. If users worried about privacy refused to share this type of information, then the personalisation system would be rendered useless (Sackmann et al., 2006). This revolved around the affect of personalisation on usability. Usability is an important factor in the overall satisfaction with any system. There are several definitions of usability. Nielsen (2003) defines usability as: “a quality attribute that assesses how easy user interfaces are to use ... how easy it is for a user to manipulate the interface and how easy it is to use after the manipulation”. Traditional usability evaluation measures several attributes relating to the user’s behaviour on the site. One such method measures attractiveness, control, efficiency, affect, learnability and satisfaction (Kuan et al., 2005). Another, more dominant, usability evaluation definition is the “extent to which a product can be used by specified users to achieve Chapter 2 Personalization and Usability 25 specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (ISO9241-11, 1998, pg. 2). The ISO 9241-11 also defined the three factors as follows: ❼ Effectiveness: Accuracy and completeness with which users achieve specified goals. ❼ Efficiency: Resources expended in relation to the accuracy and completeness with which users achieve goals. ❼ Satisfaction: Freedom from discomfort, and positive attitudes towards the use of the product. Several methods exist that allow for the evaluation of a system or a website. These methods range from allowing exploration of a certain feature of a system to exploration of the entire system. Holzinger (2005) grouped usability methods into two broad categories: inspection methods and test methods. Inspection methods include heuristic evaluation, cognitive walkthrough, and action analysis. Test methods include thinking aloud, field observation, and questionnaires. Nielsen (1994) described four basic ways used to evaluate software and website human-computer interfaces: automatically, empirically, formally, and informally. Automatic website evaluation is still a work-in-progress. However, the other three methods are currently used to inspect and evaluate Web sites. Andreasen et al. (2007) described five methods that relate to empirical interface evaluation when dealing with remote usability testing. The authors categorised the methods into two broad categories: synchronous and asynchronous. Synchronous methods include usability evaluation and usability inspection. Asynchronous methods include self-administered Web study, self-reporting of critical incidents, and logged user pattern. Palanque et al. (2007) concentrated on formal usability testing, grouping them into two forms: formative and summative. Formative usability testing is done during the development of a system and includes heuristic evaluation. Summative usability testing is done at the end of the development as a validation assessment, to confirm that the system is fit for purpose and to provide empirical data to evidence this. The distinction between formal and informal usability methods is somewhat blurred: for example, Palanque et al. (2007) mention heuristic evaluation as a formal testing method, but Holzinger (2005) describes heuristic evaluation as “the most common informal method ” (p. 72). 26 Chapter 2 Personalization and Usability The three factors defined in ISO 9241-11, namely: effectiveness, efficiency and satisfaction, will the basis for the studies carried out in this paper to measure the effect of personalisation. These were chosen because they are considered to be the general definition of usability (Jokela et al., 2003; Naumann et al., 2009). To measure the effect of personalisation on usability, visual search will be used. Users search through a page regardless of whether it is personalised or not to find information. Visual search was used as a tool in relation to website usability and is considered a norm when conducting usability studies (e.g. Pearson and van Schaik, 2003; Ling and van Schaik, 2007; Lu et al., 2010). Previous research has explored many aspects, including the effect on visual search of colour (Ling and van Schaik, 2002; Pearson and van Schaik, 2003), fonts (Ling and van Schaik, 2006, 2007), information forms (Li et al., 2009a,b), and information layouts (Buscher et al., 2009; Halverson and Hornof, 2004). Other studies examined the effect of quantity of information (Lu et al., 2010) and layout size (Hornof, 2001; Hornof and Halverson, 2003) on visual search time. Lu et al. (2010) concentrated on search time, fixation duration and fixation count and found out that adding more content to a Web page increases the visual search time. Hornof and Halverson (2003) found similar results when extending the layout of Web pages; namely that increasing the size of the layout increased visual search time. Visual search is affected by several factors, including the density of information, both overall and local. Local density, as defined by Tullis (1983) is “the number of filled spaces near each character, often manipulated by altering line spacing” (p. 662). Overall density, Tullis (1983) argued, is “the number of characters displayed, often expressed as a percentage of the total character space available” (p. 662). It is possible to have two displays with the same overall density, but different local density. Density should also be distinguished from grouping, which is often related to the relevance of items on a display in HCI literature Stewart (1976). One of the core models in visual search is Guided Search 2, which “is used to predict search times for different stimulus conditions by means of detailed computer simulations” (Hübner, 2001, pg. 549). However, visual search is primarily used to measure visual search time using geometric shapes and letters. The author is more interested in measuring the visual search time related to words. Word search, a variant of visual search, is a process of finding a target word in a set of distracting words (Ojanpaa et al., 2002). Visual search time therefore is the amount of time users take to find a piece of information on a Web page, Chapter 2 Personalization and Usability 27 in this paper, visual search time is synonym for efficiency as one measure of usability. In the definition of usability in the ISO 9241-11, it describes efficiency as the ”resources expended” which is defined here as time. 2.3 Research Question Although previous research has explored many aspects of website usability, including the effect of amount of content on visual search time (Lu et al., 2010), there is still a lack of research when it comes to evaluating the usability of a website when the user is responsible for the amount of content on the Web page, specifically through personalising the Web page. By being able to personalise a Web page, a user is able to arrange it to fit their requirements rather that adhere to the default offering. The research question this paper explores is “What is the effect of changing the layout and amount of content via personalisation on usability?” Personalisation is the act of the user controlling the amount of content on the Web page and moving it according to what they want. Usability is measured by three distinct variables as defined by the ISO 9241-11: efficiency, effectiveness, and satisfaction. These factors were found not to be correlated (Frøkjær et al., 2000); measuring each one independently of the others and observing its effect on usability was important because it allowed the researcher to measure the effect of each variable while holding the other variables constant. Effectiveness was measured by whether the participants were able to complete the task or not. Efficiency was measured by the time participants took to complete the task, or visual search time (VST). Satisfaction was measured by a known satisfaction survey (Lewis, 1995) after participants completed the experiment. Based on the above, the following hypotheses were formed: ❼ Hypothesis 1a: Increasing the amount of information on a Web page will increase the amount of time to find specific information. ❼ Hypothesis 1b: Increasing the amount of information on the page will decrease the number of tasks completed successfully. ❼ Hypothesis 1c: Increasing the amount of information on the page will decrease the perceived satisfaction with the page. 28 Chapter 2 Personalization and Usability There were a total of three different studies: a pilot study, the main study, and a supportive study. A variation of Google’s personalisation page (iGoogle) was chosen because, even though all of the surveyed personalisation services were similar, iGoogle adopted a minimalist approach to personalisation; it was done in a manner that did not involve other items on the page other than the content blocks. The personalisation scheme was also easy to adapt to the research at hand; it did not require changes to the default personalisation procedure users would normally go through when personalising a Web page. An experimental approach was chosen for the three studies because this allowed for the study of the relationship between variables which are, in this case, efficiency, effectiveness, and satisfaction (Hopkins, 2000). Additionally, an experimental approach was chosen to enable quantifying the relationship (Hopkins, 2000). It also allowed the researcher to conduct an experiment; in this case, varying amounts of content on the Web page, and to observe the response of the participants, by measuring the time it took participants to complete the task. Imposing of a treatment requires retaining other variables as constant, to be able to accurately measure the effect. Additionally, the experimental approach was chosen because the research question: “What is the effect of changing the layout and amount of content via personalisation on usability?” required the development of hypotheses which needed to be tested (Robson, 2002). Finally, the explanatory approach in the design of the experiments evolved from other studies that used the same methods in evaluating usability; therefore repeating other studies’ methodologies was considered a valid approach (e.g. Pearson and van Schaik, 2003; Ling and van Schaik, 2007; Lu et al., 2010; Ling and van Schaik, 2002; Pearson and van Schaik, 2003; Ling and van Schaik, 2006). To address the inclusion of several experiments in this paper, the next sections will describe the methodology of the each experiment, followed by the results. Chapter 2 Personalization and Usability 2.4 Pilot Study 2.4.1 Methodology 29 A pilot study was completed in order to evaluate if having more information on a website had an effect on the visual search time. There was a total of 5 participants recruited through convenience sampling. A convenience sample might affect the external validity of the results (Robson, 2002). Fulfilling a requirement of a representative sample, in this case, is quite difficult as with much of the research involving the web (Robson, 2002). A Lenovo Thinkpad T60p with a 15.4 inch monitor and an external mouse was used in the experiment. A modified page of iGoogle, after all identity had been removed, was used, allowing users the same control over the item blocks. There were 33 item blocks for participants to choose from. The item blocks were only made of news-type item blocks similar to those in Figures 2.1 and 2.2. These were chosen because they do not require configuring, such as the weather item block (Figure 2.3). Participants were allowed to choose item blocks they found interesting and move them around. Moving of the item blocks is easily done by grabbing the top bar of the item block and moving it where required (Figure 2.4). They were then instructed to choose 3, 6, or 9 item blocks and lay them horizontally on the page (Figures 2.6, 2.7, and 2.8 respectively). The main reason for the choice of the variation of item blocks and the horizontal orientation is that according to Ojanpaa et al. (2002), four to five words could be identified vertically in a single fixation as opposed to around two words horizontally. This will enable users to look at a single block at a time with minimal distractions because each of the item blocks in the test Web page is around four to five words vertically. The users were then asked to answer three questions with answers within the item blocks they had chosen. There was no need for participants to click through to find answers to the questions. This allowed for the measurement of the amount of time participants took to find answers to questions when content varied on a Web page. A time-to-task approach to usability testing (MacLeod et al., 1997; Sauro and Kindlund, 2005) was adopted in this experiment. Even though there are several other approaches to usability testing, such as eye-tracking, the time-to-task approach was an adequate way of gathering data and, as mentioned earlier, a norm in usability studies 30 Chapter 2 Personalization and Usability (e.g. Pearson and van Schaik, 2003; Ling and van Schaik, 2007; Lu et al., 2010). Eye-tracking in the current form of non-invasive tracking is relatively new. Additionally, and more importantly, the eye-tracking accumulates an extensive amount of non-relevant data for this study (Jacob and Karn, 2003). Also, eye-tracking apparatus is generally used to aid in improvement of the design of an interface, something that is outside the scope of this research (Poole and Ball, 2005). Finally, there are no standards regarding the eye-tracking metrics currently used by eye-tracking apparatus (Poole and Ball, 2005). Figure 2.4: Moving of Item Blocks 2.4.2 Results The pilot study had a total of five participants and a total of 15 observations. Of the participants, 80% were female; and 80% of the participants have used some sort of personalisation previously, but not iGoogle. Regression analysis on the pilot study data, using visual search time as a dependent variable and the number of item blocks as an independent variable showed that the relationship was significant (F = 5.04, p = 0.042). The regression model explained 27.9% of the variation in visual search time. The pilot study supports hypothesis 1a. There were no measures of hypotheses 1b and 1c in the pilot study. Chapter 2 Personalization and Usability 31 Figure 2.5: Moving of Item Blocks 2.5 Main Study 2.5.1 Methodology After the results of the pilot study had confirmed the existence of a relationship between the amount of content and visual search time, a more rigorous main study was designed by alleviating some of the vagueness and mistakes that was identified in the main study. One of the main mistakes identified in the pilot study was that participants started searching for an answer before the end of the questioning, which complicated the way of calculating the amount of time users took to find information. Participants in the main study were instructed not to look at the screen until the question is completely read out to them. Another mistake was that some participants did not know how to personalise a Web page. This was remedied by a mini-presentation given to participants showing them how to use the Web page. A sample question was also posed for the participants to answer so that the researcher could ensure that they understood the task and were aware of how to use the Web page. 32 Chapter 2 Personalization and Usability The main experiment used a 17 inch monitor and an external mouse. The screen and mouse were the only objects visible to the participant of the study. A laptop was connected to the 17 inch monitor and displayed the same screen to the tester and the participant. It allowed Figure 2.6: 6 Item Blocks the researcher to demonstrate how to interact with the item blocks and to observe how the user interacted with the Web page. The same modified Web page of iGoogle as the one used in the pilot study was used for Figure 2.7: 3 Item Blocks the main experiment. To ensure randomisation, participants were instructed to pick a number from a bowl that corresponded to a random mix of 3, 6, and 9 item blocks. For example, if the participant picked a number Figure 2.8: 9 Item Blocks that corresponded to a 6-3-9 mix, the participant chose the first group of item blocks (i.e. 6) and arranged them in three columns with 3 item blocks per column on the page (Figure 2.6). The participant was then asked three random questions. The questions comprised of reading part of one of the headlines in one of the item blocks and omitting the final word for the participant to find. The time taken to find the word corresponded to visual search time the participant took rather than how long it took participants to understand the question and then find the answer. The time the participant took to answer each question were noted. Next, the participant was instructed to choose the second group of item blocks (i.e. 3: Figure 2.7). As with the pilot study, a simple time-to-task approach was implemented. A Chapter 2 Personalization and Usability 33 time-to-task approach to usability testing (MacLeod et al., 1997; Sauro and Kindlund, 2005) was adopted in this experiment. Even though there are several other approaches to usability testing, such as eye-tracking, the time-to-task approach was an adequate way of gathering data. Eye-tracking in the current form of non-invasive tracking is relatively new. Additionally, and more importantly, the eye-tracking accumulates an extensive amount of non-relevant data for this study (Jacob and Karn, 2003). Also, eye-tracking apparatus is generally used to aid in improvement of the design of an interface, something that is outside the scope of this research (Poole and Ball, 2005). Finally, there are no standards regarding the eye-tracking metrics currently used by eye-tracking apparatus (Poole and Ball, 2005). The next group of item blocks were differed from the first group because any used item blocks were omitted from subsequent experiments with each participant. This was done to ensure no carry over effect from the previous item block combination. The participant was then asked three random questions. The time the participant took to answer each question was noted. Finally, the participant was instructed to choose the final group of item blocks (i.e. 9: Figure 2.8). These item blocks were different to the two previous groups of item blocks. The participant was then asked three random questions. As done earlier, the time the participant took to answer the questions was noted. After completing the experiment the participant was asked to complete a survey that measured their subjective view regarding efficiency, effectiveness, and satisfaction. However, the main interest of the survey lay in the self-reported satisfaction of participants. The survey was adapted from Lewis (1995) which was designed to measure users’ satisfaction with the usability of computer systems. A copy of the survey is available in Appendix A. Data were collected at the United Arab Emirates University. Participants were recruited in a convenience sample from undergraduate classes in the UAE University. A convenience sample might affect the external validity of the results (Robson, 2002). Fulfilling a requirement of a representative sample, in this case, is quite difficult as with much of the research involving the Web (Robson, 2002). In order to recruit participants, the researcher presented overall aims of the research in three separate seminar groups and briefed the students regarding the experiment, after which participation was requested. An office was dedicated to the experiment and experiment time-slots were published for students willing to participate. A total of 54 students 34 Chapter 2 Personalization and Usability volunteered, filled in the precondition statement and participated in the research at the UAE University. Designs of both the experiment and the survey to be used were submitted to the scientific research ethics committee at the UAE University along with a briefing on how the experiment was to be conducted, screen shots and the address of the Web page to be used, a copy of the questions being asked during the experiment, and a copy of the tentative research paper. Both the experiment design and the survey were reviewed by the committee in its meeting on February 16th, 2009 and was approved as is. 2.5.2 Results The main experiment had a total of 54 participants and 486 observations. Of the participants, 63% were females. There was no significant difference between males and females (F = 0.7466, p = 0.388). Sixty four per cent of the participants used some form of personalisation, and 44% of them had previously used iGoogle. A linear model with repeated effects was used to fit the cross-over design data. Visual search time was used as the dependent variable and number of item blocks, sequence, gender, mother language, computer experience, and age were used as independent variables with number of item blocks within subjects being the repeated effect. The test of significance of the predictors indicated that only the number of item blocks was significant (F = 76.017, p < 0.000). None of the other independent variables had a significant effect. There was no reliable measure for effectiveness. The task seemed simple for all of the participants as 100% success rate was reported; however, the task could not have been made more complex because of the nature of personalisation. There was no significant effect of the number of item blocks on the subjective satisfaction of participants. Satisfaction was based on the score of overall satisfaction question which was measured by the usability questionnaire question 16. Additional analysis was done on other survey questions that relate to satisfaction (questions 1, 6, 13, 14, and 15). There were no differences between the satisfaction levels of males and females (Mann-Whitney test, p = 0.870). There were also no difference in satisfaction between those who used personalisation and those who did not (Mann-Whitney test, p = 0.275). Chapter 2 Personalization and Usability 2.6 2.6.1 35 Supporting Study Methodology A supporting study was designed to mimic the main study with one difference: participants were allowed to skim through the personalised page for three to five seconds; they were allowed to quickly look through the page after it was personalised. This was done to imitate the real-world scenario where users usually know the general location of the item blocks and what they generally contain. For instance, having an item block from Reuters, a user knows that it contains latest news from Reuters even though it is not known what exact headlines are contained within. 2.6.2 Results The supportive study comprised a total of 17 participants and 51 observations. The sampling of participants, as with the main study, was a convenience one. A convenience sample might affect the external validity of the results (Robson, 2002). Fulfilling a requirement of a representative sample, in this case, is quite difficult to obtain as with much of the research involving the Web (Robson, 2002). A class at the University of Southampton was approached and the experiment explained. Participation was requested and asked to fill a precondition statement. The research followed all the ethical research procedures required by the University. Of the participants, 59% were females and ages ranged between 21 and 53. Sixty four per cent of the participants had previously used some form of personalisation, and 41% of them previously had used iGoogle. Visual search time was used as the dependent variable and the number of item blocks, sequence, carry-over, gender, mother language, computer experience, and age were used as independent variables, with number of item blocks within subjects being the repeated effect. Number of item blocks did not have a significant effect on visual search time (F = 1.26, p = 0.27). Furthermore, there was no significant relationship between number of item blocks and subjective satisfaction of participants (F = 1.08, p = 0.39). Satisfaction was based on the score of overall satisfaction question which was measured by the usability questionnaire question 16. Additional analysis was done on other survey questions that relate to satisfaction (questions 1, 6, 13, 14, and 15). As with the pilot study and main study, a simple time-to-task 36 Chapter 2 Personalization and Usability approach was implemented. A time-to-task approach to usability testing (MacLeod et al., 1997; Sauro and Kindlund, 2005) was adopted in this experiment. Even though there are several other approaches to usability testing, such as eye-tracking, the time-to-task approach was an adequate way of gathering data. Eye-tracking in the current form of non-invasive tracking is relatively new. Additionally, and more importantly, the eye-tracking accumulates an extensive amount of non-relevant data for this study (Jacob and Karn, 2003). Also, eye-tracking apparatus is generally used to aid in improvement of the design of an interface, something that is outside the scope of this research (Poole and Ball, 2005). Finally, there are no standards regarding the eye-tracking metrics currently used by eye-tracking apparatus (Poole and Ball, 2005). 2.7 Discussion The pilot study showed that there was a relationship between number of item blocks and the time it takes participants to find information. This means that the more information a user is presented with, the longer it would take to find information. The main experiment provided additional assurance that there is a very strong relationship between the amount of content on a Web page and the time it took participants to find information on that page. Users of personalisation vary in the amount of information they have on their personalised page. However, the follow-up interviews showed that even when participants took longer to find information, even when considerable, participants perceived that they saved time overall because they did not need to visit every website the content is aggregated from. This, as mentioned by Chellappa and Sin (2005), ensures repeat visits and increases customer satisfaction because, as opposed to a generic home page, a user is presented with something that better suits their preferences, and that they are used to. Additionally, decreasing the amount of content users have to sift through in order to find what they want through personalisation helps to reduce information overload; a major issue with the Internet and the World Wide Web (Lu et al., 2010; Shahabi and Chen, 2003). Users, in general, rarely seek to personalise a page presented to them. This is especially true in Google’s case where users visit the site with a different mental model than the to that offered by the personalisation service; the website is a search engine rather than a portal. The findings also support Frøkjær et al.’s (2000) research, that the three usability factors as Chapter 2 Personalization and Usability 37 defined by ISO 9241-11 (ISO9241-11, 1998) - efficiency, effectiveness, and satisfaction are not correlated. It seems that there is a deeper relationship between these three factors than a simple linear relationship. Other studies have shown that adding more content in a typical Web page will increase the amount of time users take to find information (Lu et al., 2010). Other studies varied different aspects of a Web page and found this increased the length of time users took to find information (Hornof, 2001; Hornof and Halverson, 2003). However, the relationship between the amount of content and personalisation in the context of usability has not been researched. The pilot and main study measured the visual search time and found similar results; adding more content to a Web page will increase the visual search time. This is not how personalisation is carried out however. Personalisation varies the type of content on the Web page in addition to the amount, not only by adding content but by removing content as well. Since users are the source of the personalisation, however, they are aware of the general location of content on the page; not the specific content but rather the content group such as headline news or sports news. This promoted the supporting study. It was interesting to find that when participants were allowed – three to five seconds to skim the personalised page before the experiment began, the relationship between amount of content and visual search time diminished. It seems that the amount of information on the Web page plays a lesser role in the time users take to find information on the personalised Web page than familiarity with the Web page and the content within. Additionally, regardless of the time users took to find information, there was no change in their subjective satisfaction with the Web page because of perceived usefulness, such as time reduction in looking for information on the personalised Web page. Users’ perceptions were that they would take less time finding information on the personalised Web page than searching through several Web pages to the find the same information. Users visit Web sites with a reason in mind, and enabling them to be able to achieve their target effectively, efficiently, and with an acceptable level of satisfaction is a common goal. However, the context of use of a website or system should be taken into consideration when discussing usability. Non-discretionary use of systems may yield different results in usability measures than discretionary use. In general, the common use of the time-to-task approach to usability testing (MacLeod et al., 1997; Sauro and Kindlund, 2005) proved complicated to implement when it 38 Chapter 2 Personalization and Usability came to personalisation. This was due to several reasons: the most important is that when information specific to each participant is involved, evaluation becomes extremely difficult through the general measurements of usability, efficiency, effectiveness, and satisfaction (Bellotti et al., 2004, 2003; Bellotti and Smith, 2000; Bellotti et al., 2009). The time-to-task approach is commonly used in Web sites where users are not allowed to change the content or context of the Web page and where the users are asked to seek specific information from a Web page. The measurement is then done on how long users took to find the information required. This approach is viable when the extent of improvement to a new website design needs to be measured. When it came to personalisation, however, it seems that more factors play a role than those measured in these experiments. It is observed by other researchers that conducting any type of rigorous research in HCI is difficult (Dray, 2009), which inclines researchers to narrow their objective thus making it less practically relevant (Parush, 2006). Other definitions of usability, as described in the literature review, are available and could have been used. However, these are not agreed upon and therefore add another dimension of complexity when dealing with usability evaluation in terms of defining and measuring the agreed factors. Hence, in this case, a pre-agreed on definition was adopted. It seems that usability evaluation based on the three factors, efficiency, effectiveness, and satisfaction, is not suitable for all types of Web sites. It is evident here and within other research mentioned earlier (e.g., Bellotti et al., 2004, 2003; Bellotti and Smith, 2000; Bellotti et al., 2009) that Web sites, especially those that allow for the manipulation of the content and context and include information related to the user, are harder to measure through evaluation methods and therefore require another set of measurements. Perhaps a longitudinal rather than a snap-shot evaluation is better suited in measuring similar Web sites; a method that evaluates the interface over a period of weeks rather than at one point in time (Bellotti et al., 2009). 2.8 Limitations and Future Research The author considers that measures of effectiveness may not have been appropriate in this study because of the simplicity of the task given to participants. The pilot, main, and supporting studies asked participants to find information on the page rather than click through the links to find answers, which would have been a better design in terms Chapter 2 Personalization and Usability 39 of effectiveness. Simplicity of the experiment is believed to be the reason why participants achieved the task easily. In the experiments, the effectiveness measure was described as if the users were able to complete the task successfully. A better measure might have been breaking down the effectiveness measure into sub-categories of 25%, 50%, 75%, and 100% effectiveness according to the time taken to find answers. This would have allowed the visual search time to influence the effectiveness measure as well as the efficiency measure. Future research might also concentrate on creating a more complex experiment where effectiveness, as well as efficiency and satisfaction, could be measured more appropriately. Such research would also have to take into consideration a more complex website design and finding information deeper within the structure of a website rather than on the same Web page. Future studies could also concentrate on Web sites other than portals and search engines; more specifically sites that require time-critical decisions to be made, for example, taking longer to find information in a brokering interface might have a negative effect on the performance of the broker and therefore affect satisfaction differently. There are instances, on the Web, where time is of the essence; for example when checking-in online for a flight. In such a context some passengers would want to check-in as soon as the online check-in opens and go through it as fast as possible to ensure they get the best possible seats. The design and execution of the studies could have been improved by focusing on the factors of interest: efficiency, effectiveness, and satisfaction, starting from the pilot study and ensuring the validity and reliability of the measures involved (Robson, 2002). Ensuring that everything, regardless of how minor, was well-documented reduced potential anomalies in writing up the experiments and explaining how they were undertaken. Video recording as well as voice recording would have given added reliability to the study, both as a referent in what participants were experiencing throughout the experiment, and also validity of the users’ responses to questions; unfortunately the researcher was not granted permission for this. 2.9 Methodological Reflection Based on the successful use of these measures to test usability by other studies (e.g. Lu et al., 2010; Ling and van Schaik, 2006, 2007), the author designed and executed a pilot study testing one of the three factors proposed: efficiency. The pilot study found 40 Chapter 2 Personalization and Usability that the efficiency measure was measurable and significant enough to promote a full-study design. On reflection it would have been beneficial to also measure the other two factors, effectiveness and satisfaction in the pilot study. Ensuring that these two factors were also measurable and significant within this context would have made testing them in the main and supporting study easier. In the design of the experiments, the two factors, effectiveness and satisfaction, were considered to be poor measures of what the researcher was actually attempting to measure: the effect of changing the layout and content via personalisation on usability. This might be due to two possible reasons that subsequently arose in the pilot and supporting study. The first reason was that the measures were not appropriate for personalisation, because of the personal nature of personalisation; what is considered satisfactory for an individual could be unsatisfactory for another. Therefore, the experiments did not produce the results on these two factors that correlated with the results found for the efficiency factor. The second reason was that the way the factors were measured was incorrect. The three factors are easily measured individually, but placed in an experiment within such a context of personalisation, effectiveness and satisfaction became highly subjective. This was because the users were the source of personalisation; therefore they were bound to find the information they were looking for on an interface that they personalised, and thus be satisfied with the interface because they had personalised it themselves, regardless of how minor the change was. Other studies have used the efficiency factor specifically but not the other two factors (e.g. Pearson and van Schaik, 2003; Ling and van Schaik, 2007; Lu et al., 2010; Ling and van Schaik, 2002; Pearson and van Schaik, 2003; Ling and van Schaik, 2006; Li et al., 2009a,b; Buscher et al., 2009; Halverson and Hornof, 2004; Lu et al., 2010; Hornof, 2001; Hornof and Halverson, 2003). This might have been because the research was narrow enough to promote such use of a specific measure rather than all three; or that the three factors - efficiency, effectiveness, and satisfaction - as defined by the ISO 9241-11 were found not be correlated (Jokela et al., 2003) or have a more complex relationship that is not easily defined. If the research had only been based on the measure of efficiency, as undertaken in other studies (e.g. Ling and van Schaik, 2007; Lu et al., 2010; Ling and van Schaik, 2002; Pearson and van Schaik, 2003), significant results would have been generated. This, Chapter 2 Personalization and Usability 41 however, was not the intention of this study. The author intended to measure the effect of personalisation on usability per se, and not only efficiency. When the measures of usability were surveyed, the ISO 9241-11 emerged as the most agreed-upon measure (Jokela et al., 2003; Naumann et al., 2009) and therefore was chosen, holistically, as a measure in this study. If the research was only based on the measure of efficiency, as done in the aforementioned researches, this research would of showed significant result. This, however, was not the intention of the this research. The author intended to measure the affect of personalisation on usability and not only efficiency. When the measures of usability were surveyed, the ISO 9241-11 came up as the most agreed upon measure of usability (Jokela et al., 2003; Naumann et al., 2009) and therefore was chosen, as a whole, as a measure of usability. It would have beneficial to look at more complex personalisation activities where the user is involved in more personalisation than only changing a single Web page. In such cases, the measures of effectiveness and satisfaction could have yielded significant results similar to that of efficiency. 2.10 Conclusion This paper attempted to investigate how personalisation affects usability - specifically efficiency, effectiveness, and satisfaction. Results from the pilot study (Section 2.4) and main study (Section 2.5) have shown that the amount of information on a Web page affects the time it takes users to find information on that Web page. This is also supported by previous research (e.g. Pearson and van Schaik, 2003; Ling and van Schaik, 2007; Lu et al., 2010; Ling and van Schaik, 2002; Pearson and van Schaik, 2003; Ling and van Schaik, 2006). When users were given the opportunity to quickly look through the personalised web page for 3 - 5 seconds, however, the efficiency significance found in the main study diminishes, as was evidenced in the supporting study (Section 2.6). Relating to the measure of satisfaction, results from the three studies showed that even when participants took longer to find information on the Web page, this did not have an effect on their subjective satisfaction with that Web page as measured by surveys administered after the experiments. Finally, the validity of the measurements 42 Chapter 2 Personalization and Usability of efficiency, effectiveness, and satisfaction is considered inappropriate for all types of Web sites, especially those that allow users to change the layout and include personal information (Bellotti et al., 2004, 2003; Bellotti and Smith, 2000; Bellotti et al., 2009). Appendix A Usability Questionnaire 43 44 Appendix A Usability Questionnaire Usability Questionnaire Section 1 (Demographics): Age: Gender: [F] [M] Level of Education: High School College Under- Post- Graduate Graduate Other Mother Language: Country of Origin: Where are you living now (Country)? How many years have you lived there? I am computer savvy: Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree Have you ever used personalization on the Internet? [YES] [NO] Personalization is the capability of being able to change the layout and content of a web site to fit your needs If you answered yes to previous question, then have you evern used Google personalization service? [YES] [NO] 45 Appendix A Usability Questionnaire Section 2 (Usability): Overall, I am satisfied with how easy it is to use this web page. Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree -1 0 1 2 3 Strongly Agree It was simple to use this web page. Strongly Disagree -3 -2 I could effectively complete the tasks and scenarios using this web page. Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree I was able to complete the tasks and scenarios quickly using this web page. Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree I was able to efficiently complete the tasks and scenarios using this web page. Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree -1 0 1 2 3 Strongly Agree 0 1 2 3 Strongly Agree I felt comfortable using this web page. Strongly Disagree -3 -2 It was easy to learn to use this web page. Strongly Disagree -3 -2 -1 I believe I could become productive quickly using this web page. Strongly Disagree -3 -2 -1 0 It was easy to find the information I needed. 1 2 3 Strongly Agree 46 Appendix A Usability Questionnaire Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree The information provided for the web page was easy to understand. Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree The information was effective in helping me complete the tasks and scenarios. Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree The organization of information on the web page screen was clear. Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree 0 1 2 3 Strongly Agree 0 1 2 3 Strongly Agree The interface of this web page was pleasant. Strongly Disagree -3 -2 -1 I liked using the interface of this web page. Strongly Disagree -3 -2 -1 This web page has all the functions and capabilities I expect it to have. Strongly Disagree -3 -2 -1 0 1 2 3 Strongly Agree 0 1 2 3 Strongly Agree Overall, I am satisfied with this web page. Strongly Disagree -3 -2 -1 Chapter 3 Adopting Usability Methods and Frameworks in Commercial Web Projects. A Case Study Exploring the Benefits and Pitfalls Abstract: This paper presents a case study utilising Garrett’s (2002) framework to implement usability methods within a commercial website development project. After extensive critical evaluation of several other usability frameworks, Garrett’s was adopted as it was considered to be the easiest to adapt within real-world situations. Primarily it enabled users to move in stages from an abstract mental model to a concrete operational level. This cognitive process enables all aspects of the user experience to be included within the design process. The case discusses the work undertaken by a team designing and developing a large-scale website for a governmental entity in the United Arab Emirates (UAE), and the barriers to integrating these methods and frameworks within a previously planned approach. The structure of the project was divided into two distinctive sections, enabling this case to take a unique view of the process. The case compares the approach and acceptance of usability recommendations at the beginning of the process of the redesign and 47 48 Chapter 3 Usability Methods and Frameworks in Commercial Projects development, and again when the majority of the design and development were close to completion and evaluates the differences between them. As well as examining Garrett’s framework, the case study also examined the usability analysis as part of the whole development of a portal and online services for a governmental entity. Finally, the case study allowed greater insight into the workings of commercial website design and development and offered suggestions of how to better integrate usability methods and frameworks in commercial projects. 3.1 Introduction Many usability methods, including inspection and evaluation, are developed to measure the level of usability of a system or website. Such methods may vary in their formality; for example a cognitive walkthrough (Wharton et al., 1994) may be less formal than a Task Analysis For Error Identification (TAFEI) (Stanton and Baber, 1996). They may also differ in their specificity, where some are used to concentrate on a particular aspect of a system whilst others are more effective when a broader evaluation is required (e.g. a complete website). These methods are generally helpful when focusing on creating a usable system and are efficient at identifying usability problems. Issues arise, however, when these methods are implemented into a real-world project environment. For example the lack of time and budget greatly limit the efficiency of usability methods (Blandford et al., 2006; Carroll, 2003; Wixon, 2003). The effect is that such methods are either ignored or treated as a non-crucial aspect of the design and development of the system or website. This paper reports on the work of one of the researchers as a usability consultant in a commercial environment. It explored the complexities of integrating usability methods into a real-world situation and the main barriers to undertaking usability activities in the context of a commercial project. It starts with an overview of usability methods and frameworks that will be important as the context for later discussions. Next the research problem and methodology is presented. The case study followed with a detailed report and account of the usability consultant’s findings. Next, records of feedback are provided and discussed, followed by limitations and future research. Finally, we concluded by identifying the strengths and weaknesses of Garrett’s framework and a greater insight to the workings of the website design and development, as well as providing suggestions for alternatives to better Chapter 3 Usability Methods and Frameworks in Commercial Projects 49 integrate usability methods and frameworks in commercial projects. 3.2 Literature Review In order to critically discuss and evaluate usability methods and frameworks, a review of the common methods and frameworks is presented. Usability methods are more practical than usability frameworks. This might be due to the fact that usability methods deal with single issues and narrower oversights which can be used as a model and added to a project-in-progress. They are considered a viable tool for when website developers are concerned with the usability aspect of the website. Usability methods can be used separately with added benefits to the overall usefulness of the website. This is also obvious when comparing scholarly papers published that discuss usability methods versus those that discuss usability frameworks (e.g., Folmer and Bosch, 2004; Gray and Salzman, 1998; Holzinger, 2005). Furthermore, usability frameworks require more commitment to the user-centred mindset which is side-tracked when budgetary and time limitations create pressure on any project. There are several differentiations that need to be made to narrow the scope of this paper. Design, in this author’s mind, is the process by which an end product is produced, and the end product itself. Development, on the other hand, has a notion of including coding; the actual process of creating the end product. This differentiation is similar to Lang and Fitzgerald (2007) understanding. They describe design as: “the target system is a purposefully designed solution, but the means of arriving at that solution is also designed ” (p. 204). They describe development as “the term development bears connotations of coding, construction and back-end software engineering” (p. 204). The case study described here fixates, primarily, on the design aspect. Also discussions might relate to other design processes such as in software development; the emphasis here being on Web design as part of an overall Web development effort to launch a large-scale website. In the next sections, usability frameworks and methods will be discussed. Usability frameworks allow for the efficient use of methods, which are, in turn, used for collecting data regarding the usability of a product or interface. Usability frameworks can be used to break down the design/development cycle into distinctive parts that 50 Chapter 3 Usability Methods and Frameworks in Commercial Projects focus the efforts on specific requirements of that phase of design/development. Several analyses will be undertaken regarding differences in currently available frameworks. First, analyses of the common methods used in currently available frameworks are examined. Second, differences in the use of methods at different phases of design/development are presented and discussed. Third, the advantages and disadvantages of methods are set out. Fourth, how different frameworks align will be shown and discussed. Finally, a section dedicated to a critical evaluation of Garrett’s framework will conclude the section. 3.2.1 Usability Methods and Frameworks There is a plethora of usability methods available from both a theoretical and practical perspective that can be used throughout different stages of the website development process. These methods are spread across different stages of the development process and their categorisation changes according to the framework. More discussion about the frameworks and their respective stages will follow later, but the discussion here examines the methods through how the different frameworks section them. What is important is how usability methods fall into different stages of the frameworks because these methods are the way to collect data and move forward with the design. Researchers vary in the ways they categorise methods. Maguire (2001) lists a total of 36 usability methods that were mapped to the ISO 13407 different stages (Table 3.1). Usability.gov, a “primary [US] government source for information on usability and user-centered design” (Usability, 2011) lists a total of 13 methods broken down into the three stages: analyse, design, and test. The Usability.gov site emphasises a relatively small number of methods and identifies which of these methods can be used in several different stages (Table 3.2). It is interesting to see an additional stage called plan listed in the guidelines but this disappears when displaying methods. This might be because of the lack of usability methods that fit within the plan stage. UsabilityNet (Bevan, 2003) lists a total of 32 usability methods through its six stages of development: planning and feasibility, requirements, design, implementation, test and measure, and post release. What is interesting is that UsabilityNet lists ISO 13407 as a part of the planning and feasibility stage (Table 3.3). Chapter 3 Usability Methods and Frameworks in Commercial Projects Planning Context of Requirements Design Use Usability plan- Identify stake- Stakeholder Brainstorming ning and scop- holders analysis ing Usability cost- Context of use User cost- Parallel design benefit analy- analysis benefit analysis sis Survey of exist- User require- Design guideing users ment interview lines and standards Field Focus groups Storyboarding Scenario of use study/user observation Diary keeping Personas Affinity diagram Task analysis Existing Card sorting systemcompetitor analysis Task/function Paper protomapping typing Allocation of Software protofunction typing 51 Evaluation Participatory evaluation Assisted evaluation Heuristics or expert evaluation Controlled user testing Satisfaction questionnaires Assessing cognitive workload Critical incidents Postexperience interviews User, usability Wizard-of-Oz and organiza- prototyping tional requirements Organizational prototyping Table 3.1: ISO 13407 Usability Methods in Different Stages of Development (Maguire, 2001) Finally, the usability engineering life-cycle (Mayhew, 1999), a framework more oriented towards software development but still relevant to Web development, defines a total of 20 methods through its five stages of development: requirement, design, code, test, and deployment (Table 3.4). The usability engineering life-cycle is similar to the Usability.gov in categorisation where it defined how methods are used through different stages of development rather than identifying each method and categorising it in only one stage of development. Many of the methods listed in the previously discussed frameworks overlap. Table 3.5 shows the methods that are used in at least three of the four frameworks. These are commonly used methods with generic naming and are being used in different stages of 52 Chapter 3 Usability Methods and Frameworks in Commercial Projects Usability Method Card Sorting Contextual Interviews Focus Groups Heuristic Evaluation Individual Interviews Parallel Design Personas Prototyping Surveys (Online) Task Analysis Usability Testing Use Cases Writing for the Web Analyze ∗ ∗ ∗ ∗ ∗ Design ∗ Test ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ Table 3.2: Usability.gov Usability Methods for Different Stages of Development Planning and Feasibility Stakeholder meeting Requirements Design Implementation Test and Measure User surveys Style guides Diagnostic evaluation Analyze content Interviews ISO 13407 Contextual quiry Planning User observation Context Design guidelines Paper prototyping Heuristic evaluation Parallel design Storyboarding Competitor analysis in- Focus groups Brainstorming Evaluating existing systems Evaluate prototype Wizard of Oz Interface design patterns Rapid prototyping Post Release Post release testing Performance Subject testing assessment Subjective evaluation User surveys Heuristic evaluation Critical incidence technique Pleasure Remote evaluation Card sorting Affinity diagramming Scenarios of use Task analysis Requirements meeting Table 3.3: UsabilityNet Usability Methods for Different Stages of Development 53 Chapter 3 Usability Methods and Frameworks in Commercial Projects Proactive Field Study Pluralistic Walkthroughs Teaching Method Shadowing Method Co-discovery Learning Question-asking Protocol Scenario-based Check-lists Heuristic Evaluation Thinking-aloud Protocol Cognitive Walkthroughs Coaching Method Performance Measurement Interviews Retrospective Testing Remote Testing Feature Inspection Focus Groups Questionnaires Field Observation Logging Actual Use Requirement ∗ Design ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ Code Test Deployment ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ Table 3.4: Usability Engineering Life-Cycle Usability Methods in Different Stages of Development According to (Mayhew, 1999) development. Some of these stages are comparable; for instance, context of use in ISO 13407 and requirements in UsabilityNet; however, some of the methods are being used in entirely different stages that are not comparable, i.e. the Affinity Diagram method is used in the design stage of the ISO 13407 and the requirements stage in the UsabilityNet. Some of the methods mentioned in the frameworks are similar in function but named differently. Other methods seem to be unique to specific frameworks. Table 3.6 compares the same methods available in both ISO 13407 and UsabilityNet, and differentiates between what stage they are used in. There seem to be a variation in where the methods are proposed to be used. Some of the methods fall within relatively similar stages, others however, differ greatly, e.g. brainstorming is mentioned in the design stage in ISO 13407 and in the requirements stage of the UsabilityNet. There seems to be some agreement on the definition of the usability methods used in Web development; however, there seems to be some disagreement as well regarding how these methods are defined and where they are placed in the development stage. It is out of the scope of this paper to review each method individually but the intention here is to show the scope of methods available as well as the different way they are 54 Chapter 3 Usability Methods and Frameworks in Commercial Projects Usability Method Card Sorting Contextual Interviews Focus Groups Heuristic Evaluation Individual Interviews Prototyping Surveys (Online) Usability Testing Use Cases Task Analysis User Observation Usability Frameworks Mentioning the Method ISO 13407 Usability.gov UsabilityNet ISO 13407 Usability.gov UsabilityNet Usability Engineering ISO 13407 Usability.gov UsabilityNet Usability Engineering ISO 13407 Usability.gov UsabilityNet Usability Engineering ISO 13407 Usability.gov UsabilityNet Usability Engineering ISO 13407 Usability.gov UsabilityNet Usability Engineering ISO 13407 Usability.gov UsabilityNet Usability Engineering ISO 13407 Usability.gov UsabilityNet ISO 13407 Usability.gov UsabilityNet ISO 13407 Usability.gov UsabilityNet Usability Engineering ISO 13407 Usability.gov Table 3.5: Methods that are Common to at least 3 of the 4 Frameworks Chapter 3 Usability Methods and Frameworks in Commercial Projects Method Brainstorming Field Study/User Research Affinity Diagram Design Guidelines Parallel Design Storyboarding Existing System Competitor Analysis ISO 1307 Design Context of Use Design Design Design Design Requirements Requirements 55 Stage Used UsabilityNet Requirements Requirements Requirements Design Design Design Requirements Planning and Feasibility Table 3.6: Similar Methods used in ISO 13407 and UsabilityNet viewed. Some frameworks try to couple methods to a single stage of development while others extend the use of these methods to more than one stage. Holzinger (2005) concentrated on later stages of development and differentiated between HCI methods by categorising them into inspection methods and test methods. Inspection methods include heuristic evaluation, cognitive walkthrough, and action analysis. Test methods include thinking aloud, field observation, and questionnaires. Holzinger drew a comparison between six different methods using categories: phase applicability, required time, required number of users, evaluators, equipment, expertise and intrusiveness. Table 3.7 lists the advantages and disadvantages of these methods, according to Holzinger. Some of the methods categorised by Holzinger as part of the inspection and test stages are also categorised by other researchers in earlier stages of the development process. For instance, heuristic evaluation in Usability.gov and UsabilityNet is part of the analysis and test stages. In the usability engineering life-cycle, it is part of the design, code, test, and deployment stages. This seems to imply that there is no agreement about which methods are more appropriate to use in each stage. It might also be a definition issue, as each of the frameworks look at the methods in a different context. Earlier works, such as Nielsen and Phillips (1993) compared three evaluation methods from a cost/benefit perspective with regard to arriving at estimates of user performance with interface alternatives. The three methods were heuristics; labelled as informal, GOMS (Goals, Operators, Methods, and Selection rules); labelled as formal, and user testing, labelled as empirical. The authors found that user testing was the best method in order to attain a good estimate; however they warn that results from such a test may not be appropriate to ’real-world’ applications as laboratory testing is 56 Chapter 3 Usability Methods and Frameworks in Commercial Projects Name Advantages Inspection Methods Disadvantages Heuristic Evaluation ❼ Incorporation of usability ❼ Lack of user input. early in the design stage and throughout the development ❼ Inability to identity unknown users’ needs process. ❼ Unreliability in domain❼ Intuitiveness. specific problem identification. ❼ Easy detection of major and ❼ Lack of a mechanism to evalminor usability issues. uate the whole design. ❼ Application of accepted usability principles. ❼ Rapidity. Congnitive Walkthrough ❼ Fully functional prototype. ❼ Possible bias depending on task selection. ❼ Recognising problems that might arise from interacting ❼ Concentration on minor dewith the system. tails. ❼ Lack of user involvement. Action Analysis ❼ Accurate prediction of time ❼ Extra time taken to design taken to complete a task. and execute. ❼ Insight into user behaviour. ❼ Requires high-level expertise. Test Methods Thinking Aloud ❼ Insight into users’ behaviour. ❼ Lack of adherence to most type of performance measures. ❼ Reveal how users use a system in a normal context. ❼ Time consumption. ❼ Non-analytical learners generally feel inhibited. ❼ Unnatural for users to concentrate on advantages and disadvantages at the same time. Field Observation ❼ Simple to implement. ❼ Context might render false results. ❼ Does not require extensive ❼ Only captures major usability analysis of recordings. problems. Questionnaires ❼ Subjective user preference, ❼ Low validity. satisfaction, and possible anxieties can be easily identified. ❼ Requires sufficient responses to be significant. ❼ Can be used to compile statis❼ Identifies fewer problems than tics. other methods. Table 3.7: Advantages and Disadvantages of Usability Methods (Adapted from Holzinger (2005) Chapter 3 Usability Methods and Frameworks in Commercial Projects 57 not generally a good measure of real-world performance. Conversely, John and Kieras (1996a) concentrated on GOMS and compared four variants of the method: Keystroke-Level Model, the original GOMS formulation, NGOMSL, and CPM-GOMS. Several papers detailed each of these methods and when they should be used (e.g., John and Kieras, 1996b; Nielsen and Molich, 1990). There are many well-developed usability methods available that can be employed to enhance the usability of a website with varying degrees of efficiency (e.g., Gray and Salzman, 1998; Hartson et al., 2001; Wixon, 2003). These methods can be used during the different development stages and therefore require frameworks to provide structure and direction and assist the decision on the most suitable time for using a certain method in a project’s timeline. Methods are the actual means of gathering data regarding the usability of the design and the usability of the interaction later on in the development. However, methods require a gathering technique which shows when these methods are usable. This can only be done through a comprehensive and easy-to-use framework to gather and categorise these methods. The usability frameworks reviewed identify different stages as important in order to capture and implement user requirements. The ISO 13407, which defines User-Centred Design (UCD), breaks down the process into five stages: planning, context of use, requirements, design, and evaluation. There are, however, other categorisations of the stages of development which might be as simple as three stages: analyse, design, and test (Usability.gov) or a more complicated six stages: planning and feasibility, requirements, design, implementation, test and measure, and post release (UsabilityNet). The Usability Engineering Life-cycle breaks down the process into five stages: requirements, design, code, test, and deployment. Finally, Garrett’s (2002) framework breaks the process down into five stages as well: strategy, scope, structure, skeleton, and scope, but is actually a different approach to the development in that it moves it from the more abstract to the more concrete. A comparison of the different stages each of the aforementioned frameworks follow is provided in Table 3.8 which also shows the breadth of what each of the frameworks actually cover. Garrett’s framework, for instance, is more concerned with the design stage of development whilst including the previous stages common to other frameworks including requirement gathering and planning. ISO 13407 and Usability.gov extend to the evaluation/test stage. Finally, UsabilityNet and the usability engineering life-cycle also include post 58 Chapter 3 Usability Methods and Frameworks in Commercial Projects release and deployment stages which are not mentioned in other frameworks, which gives a sense that the breadth of these frameworks is greater than the other frameworks. These two frameworks also include a stage between the design and evaluation: implementation (UsabilityNet) and code (usability engineering life-cycle), which further indicates that it is concentrating more on the overall development process as opposed to just one or a few aspects. Differences in the frameworks’ breadth also seem to arise from the varying levels of concentration on usability with some concentrating solely on usability while others emphasise the development process whilst incorporating a usability aspect. Other frameworks are currently available that have emerged from research and which focus on the development of Web sites. Although not all of the frameworks are considered to be from the usability field per se, some have examined website design and included usability as a major aspect of the process (e.g., Holsapple et al., 2005), whilst others solely concentrated on usability as the core of the framework to develop Web sites (e.g., Yeung and Law, 2004; Fang and Holsapple, 2007; Cockton, 2005). The major theme, however, in all the frameworks reviewed, was incorporating an interface aspect; sometimes referred to as the visual aspect, and a functional aspect within the frameworks defined. Some of the reviewed frameworks, such as Yen et al. (2005), approached website design from an accessibility perspective by breaking down the process into structural and functional sections, and then converting the design problem into a generic problem that can be analysed and solved using analytical techniques through defining key objectives and constraints. The authors argued that this approach would allow the evaluation of Web sites through quantitative rather than qualitative data. Another framework used by Holsapple et al. (2005) broke down Web development into three main sections: usability, visualisation, and functionality. This framework was constructed to help the development of Web sites targeted towards people with disabilities. It approached website design from an accessibility perspective and followed guidelines that addressed this particular issue. Other examples are frameworks that enabled aspects of website design to be expanded to a more granular level (Yeung and Law, 2004; Fang and Holsapple, 2007). Yeung and Law (2004) described five modules that interconnect to provide website usability: language usability, layout and graphic usability, information architecture usability, user interface and navigation usability, and general usability. They extended the heuristics evaluation method and measured the differences between Planning Planning and Feasibility Usability Engineering Garrett Strategy Context Requirements Analyze Requirements Design Design Design Implementation Requirements Design Code Scope Structure Skeleton Surface Table 3.8: Comparison between Different Stages in Frameworks Evaluation Test Test and Measure Test Post Release Deployment Chapter 3 Usability Methods and Frameworks in Commercial Projects ISO 13407 Usability.gov UsabilityNet 59 60 Chapter 3 Usability Methods and Frameworks in Commercial Projects independent and chain hotels’ Web sites. In contrast Fang and Holsapple (2007) approached the usability issue from a features perspective describing task, user, provider, system, and environmental features whilst investigating the navigational structures and its impact on usability. Yet other frameworks lie somewhere in between. Cockton (2005), for instance, proposed adding a fourth dimension to previous dimensions that were, and still are, considered important: system, context of use, and the user. The fourth dimension to be addressed was value and Cockton discussed its importance in designing usable Web sites. He also introduced four processes through which value could be added: opportunity identification, design, evaluation, and iteration. Cockton argued that even though it seemed that HCI was moving towards value-based evaluation, this was still in its infancy and would require considerable work to build an effective framework of use. In other research concentrating on physical aspects of Web design, Ivory and Hearst (2002) suggested using four dimensions: information, navigation, graphics, and experience to help the non-professional designer to develop more usable Web sites. The authors then proceeded to develop a prediction model to evaluate the design of the website. 3.2.2 Garrett’s Framework Garrett’s (2002) framework discussed a five-level development process: Strategy, Scope, Structure, Skeleton, and Surface (Figure 3.1) . This is a step away from the generic planning, requirements, design, development, and evaluation process usually followed by other frameworks. Additionally, each stage in this framework moves from the abstract to the concrete. These stages, which the author calls “planes”, are articulated to ensure that all aspects of the user experience are covered when designing. The first plane, strategy, identifies what the owners and users want from the website. The scope plane converts the strategy objectives into specific requirements in respect to both content and function. The structure plane develops a conceptual structure of the website. This is where group members begin to see how content fits on to the website and how everything is related. The skeleton plane defines how the pages will look in an illustrative format - or “wireframes” as the author describes it. Layouts of page elements become clear in this plane; navigational elements, labelling elements, and content are all given an area in this plane. The final plane, surface, is what a user first sees when they visit the website: fonts, colours, and other visual elements are detailed Chapter 3 Usability Methods and Frameworks in Commercial Projects 61 in this plane. Figure 3.1: Garrett’s (2002) Framework Illustration The levels of abstraction is a concept that has been used in many fields to reduce the complexities of the problem at hand. This is done by moving from a more abstract to a more concrete level of abstraction and is done to reduce complexities, to focus on a wider view in higher level of abstractions, and to move to more specific details in more concrete levels. This removes unnecessary details from the overall view of the conceptual design where details are handled at the appropriate time and necessary details are sure to be dealt with and not left out of the design process (Zahniser, 1983). For example Rasmussen (1986) Level of Abstraction Hierarchy defined five levels of abstraction: functional purpose, abstract function, generalised function, physical function, and physical form. The most abstract level is the functional purpose that defines the overall meaning of the system and the strategy it carries. The physical level, on the other hand, gives an accurate representation of the system including the components and connections. Another example is Zahniser’s (1983) software engineering system development life cycle where the life cycle is mapped to an 62 Chapter 3 Usability Methods and Frameworks in Commercial Projects abstraction framework. The author defined seven levels of abstraction:“Nirvana”, Functions, Data, Boxes, Control, Builds, and ending with Problem Solution. As with Rasmussen’s hierarchy, Zahniser’s life-cycle start with the most abstract level called “Nirvana” where a concept of a system only exist in a user’s mind. The most concrete level is Problem Solution where the produced system is installed in the user’s environment that alleviates the problem the user had conceptualised in the “Nirvana” stage. Garrett’s framework articulates the design aspect of the development process but includes the previous steps before the design stage, i.e. feasibility, planning, and requirements. This allows the development process to continue from the end of the framework easily utilising the output of the process; the design is implementable with most, if not all, of the questions answered. Abstraction, through Garrett’s framework is beneficial to the stakeholders involved in the design of the website; it allows for the discussion and input of different stakeholders without using overly technical terms, either HCI or development. Choosing not to cover the later development process is also beneficial in limiting the complication of the discussions. The end design details what the stakeholders require from the website and leave the underlying hardware and coding to the developers. This also simplifies development because developers are presented with a clear indication of what the stakeholders require from the website. Evaluation on the nearly-completed website is also done from a design perspective rather than an overall evaluation of the system; as long as the hardware and coding is able to provide the desired outcome. For example, if the loading speed and compatibility with different Web browsers is as required, then the designer is concerned less by the underlying coding and more with how it is being presented to the user. In Garrett’s framework, the design is undertaken in a linear fashion with overlapping areas where work still being finalised in one plane coincides with work starting on the next level. This allows changes to be made in the previous level without greatly affecting the next level whilst incorporating changing requirements within the design process. These levels overlap at around 30% in adjacent levels and 10% with the following one. Overlapping the levels attempts to overcome the shortcomings of the original system development life-cycle where work on the next stage only starts when the work on the previous stage finishes. Many issues arise from such an approach, such as non-acceptance of changing requirements among other issues (Ragunath et al., 2010). Chapter 3 Usability Methods and Frameworks in Commercial Projects 63 Garrett’s (2002) framework provides technical and non-technical team members with a common ground, namely the five planes, to facilitate discussions using a common language and concentrate on usability issues that the website is currently experiencing or likely to in the future through abstraction. This, also, is a step away from the overly technical discussions in the generic development process that are imposed by the stages they fall within. It also give a broader perspective of Web design as all stakeholders are able to give attention to the five planes in a more general sense rather than being concerned with specific technical or managerial details. These five planes also direct the effort towards user requirements and needs rather than technical aspects of design and development, which means that this framework can be easily embedded into a business context and is therefore likely to be accepted by potential users. However, the framework lacks some important aspects of website development from a user perspective. For instance, Hodgetts et al. (2005) referenced Garrett’s framework while discussing implementation of user experience design practices into agile processes. He explained that the framework was beneficial in agile development but did not incorporate the evaluation of usability methods, which is a crucial part of any developmental design efforts, including Web sites. This can be overcome by defining what types of usability methods can be used in each stage of design. This, of course, is influenced by several constraints including time and budget; and therefore having a group of methods from which to choose will be beneficial. In the context of the ISO 13407, this is done through UsabilityNet’s method table; where one can select one of the three conditions: limited time/resources, no direct access to users, and limited skills/expertise. After selecting one of the conditions, the recommended methods are highlighted. This was not attempted in other frameworks though. 3.3 Research Problem All the aforementioned methods are relatively efficient in estimating usability problems, even though some researchers take the stance that the amount of methods available is, in part, to be blamed for the lack of integration of usability methods into practice (Bellotti et al., 2009). Other issues are also to blame. For example more recently as researchers have begun to explore the difficulties of using HCI methods in a ’real-world’ setting (e.g., Blandford et al., 2006; Nørgaard and Hornæk, 2006; Spencer, 2000; 64 Chapter 3 Usability Methods and Frameworks in Commercial Projects Wixon, 2003), they have suffered implementation, time management and social constraints, as well as observer bias. Blandford et al. (2006) for instance detailed a case study of working with developers on digital libraries to produce usability tools that support design practice. The authors’ aim was to provide usability evaluation tools to help developers create a more usable library and not to implement a new design process. The authors found that even though the claims analysis method they used was effective in helping the development process, it took much longer to learn and apply, and it was much harder to introduce the method into practice than anticipated. One of the problems associated with implementation was the lack of any effective communication methods between the authors and the developers of the digital library. Another example is when Spencer (2000) applied the streamlined cognitive walkthrough method in a sizeable software development company where there was a large development team involved in the development process. The author found that it was often difficult to keep team members on task. Such users were usually ‘pressured for time, tended to lapse into lengthy design discussions, and were sometimes defensive about their user-interface designs’ (p. 1). Spencer refers to these three factors as ‘social constraints’ and proposed an alteration to the cognitive walkthrough method to work around such obstacles by teaching the team to perform cognitive walkthrough, modifying the cognitive walkthrough method itself, and by having strong leadership during the method implementation. He subsequently undertook a small case study as proof of the concept and concluded that although streamlining the cognitive walkthrough method may focus on the big picture; such a method loses some relevant details in the process. Yet another example is the research of Nørgaard and Hornæk (2006). The authors concentrated on the ‘think aloud’ method, and explored how it was used in practice. They identified several crucial issues with how the method was being implemented. One particular problem was how evaluators tended to expect users to confirm their perception of what difficulties may exist rather than allowing users themselves to identify their own problems. They also criticised evaluators for asking users hypothetical questions rather than listening to users’ real experiences. Yet another issue was that evaluators focused mainly on usability problems of the system but failed to understand the actual utility of the system being used. Yet another example is Wixon (2003) who posited a simple question that is important to both theorists and practitioners alike: how many users are required to actually Chapter 3 Usability Methods and Frameworks in Commercial Projects 65 detect an appropriate number of problems that would lead to a development of a usable website? Wixon used this controversial question to investigate a broader topic, namely the goals of the methods used and how the quality of each one is determined. He also introduced the notion of applied usability work and compared it to engineering. He stated that, in ’real-world’ engineering “the goal is to produce, in the quickest time, a successful product that meets specifications with the fewest resources, while minimizing risk ” (p. 4). Finally, the author concluded that there was a need for a case study approach in order to enhance the understanding of the usability practice and to evaluate the methods by applying them to real, rather than simulated or hypothetical situations; he then gave two examples of such case studies. Carroll (2003) also identified that, even when the HCI practitioner is skilled enough to pursue its application, “external issues such as schedule, budget, and compatibility with standard solutions often prevail ” (p. 6). This, in turn, created pressure to develop methods that could be easily implemented and could be understood by novice Web designers. The usability literature therefore argued for a case study approach to HCI (Wixon, 2003). Several papers published within HCI that discuss case studies concentrate on digital libraries and universities. Those that focus on digital libraries explore a variety of issues including the techniques and principles suitable for evaluating the digital library’s usability (e.g., Blandford et al., 2001; Battleson et al., 2001), how search engines influence students’ behaviour when searching for information in the digital library (Augustine and Greene, 2002), and what usability issues users face in a natural setting (Blandford et al., 2001). Studies that focus on universities usually discuss redesign initiatives on the university’s website (Chisman et al., 1999; Corry et al., 1997). Other case studies that relate to HCI have focused on a variety of other subjects including the design of a public access for a community photo library (Kules et al., 2004). However, most case studies discuss the case study from an end-user perspective rather than through the stakeholder approach concentrating on the whole design process. Furthermore, many Web sites developed in these case studies cannot be considered real-world per se because they lack the commercial aspect of the Web development. It is clear, from the literature review, that scholars have found that although methods of HCI may be effective in theory and in controlled environments, when introduced into the ’real-world’, the process of evaluating and correcting usability issues directly 66 Chapter 3 Usability Methods and Frameworks in Commercial Projects conflicts with deadlines and resource budgeting, which are two crucial elements that dominate any practical project. Research confirms that HCI methods are difficult to implement and hard to adapt to current Web design and development efforts. Furthermore, in most cases these methods are seen as a complementary addition rather than a crucial part of any website development. If it is not possible for HCI methods to be effectively integrated into the social and business context of a real life project and fit within actual timeline and budgetary requirements, then usability evaluations will always be treated as being a parallel and cumbersome process that could be completed if required and with little, if any, feedback that influences the design and development process. Such an assumption is likely to significantly weaken the input of HCI methods into design and development, or even worse, lead to them being omitted as a value-adding activity to the project. In summary the aim of this research is to explore the barriers to developing a commercial website and implementing Garrett’s framework to overcome these barriers using a case study approach. This includes observing the overall design process and the stakeholders that took part in the design process. 3.4 Methodology The available literature, research question, and contextual circumstances allowed the researcher the opportunity to approach this research problem as a single case study with an exploratory interest as part of the usability company commissioned to evaluate the Web development effort and provide recommendations on making it more user-friendly. The case study approach was a viable alternative here because of its applicability to “sticky, practice-based problems where the experiences of the actors are important and the context of action is critical ” (Bonoma, 1985). A case study approach is also beneficial when the intent is to capture knowledge from practice and practitioners (Benbasat et al., 1987). Benbasat et al. (1987) suggested three criteria to apply in a case study approach in research: first, to be capable of researching the system in a natural setting; second, understand the complexity of the processes and interactions happening, and third, as a viable tool to research an area with little prior research. All of the data collected were qualitative. The author was a participant observer and directly involved in the design and development of the website through the usability company as a usability professional. Participant observer is defined as ”a Chapter 3 Usability Methods and Frameworks in Commercial Projects 67 researcher who participates in social activities with the subject of the study over an extended period of time” Whyte (1979, pg. 56). Participant observation is a cornerstone of anthropology and less objectivity is sometimes an issue when researchers are directly involved in what is being researched. On the other hand, when the researchers are directly involved, they have access to a wealth of experience that is otherwise not accessible (Poltrock and Grudin, 1994). The roles of usability professional and researcher in this case was intertwined but data collected extended beyond that of the commercial work at hand; observations and follow-up interviews collected data discussing the attitudes throughout the work period. Assertions to the quality of the website and the work done was left to the team members of the project rather than the usability professional to decrease the issue of researcher objectivity. The data gathering was made up of actual documents produced and correspondence related to the project. These included emails, brainstorming sessions, meetings, conversations, and interviews at the end of the consultation period. This allowed for the interaction with individual team members working on the project and also helped in the understanding of complexity in relationships whilst working on a commercial design project. The case study also gave the researcher the opportunity to observe the reactions and attitudes of team members towards usability when implementing a structured framework, rather than just gathering information about what participants say about usability at various times throughout the consultation period. There were a total of 20 people involved throughout the project with different groups involved at different times. The number of people involved does not include those who worked on the project but had no direct contact with the usability consultancy which included people working on coding and hardware required for the website. The case study was a unique opportunity that allowed the author to interact with two sections of the project. The first section did not start until the usability consultancy’s involvement (the portal) and the other section was nearing the completion of the project; at least from a development perspective where much of the development had been done (the online services). This situation allowed the author to observe and record other team members’ reactions and interactions with recommendations brought up by the usability analysis. Working with two distinct sections of the project also allowed the author to implement Garrett’s framework on one section (the portal) and try to do so on the other section (online services); and to compare and contrast the 68 Chapter 3 Usability Methods and Frameworks in Commercial Projects differences when a framework is implemented at the beginning and mid-project. Garrett’s (2002) framework was selected for use in this project because of the considered benefits to the real-world user not found in other frameworks, as discussed above. One of the most critical benefits is that it allowed communication between stakeholders without the use of overly technical terms through abstraction and with a concentration on end-user needs rather than what the developers can develop in the given amount of time or how much the budget can buy. The technical issues are then handled by the developers after the requirements from a usability perspective have been agreed upon by stakeholders. In the first meeting, the stakeholders were briefed on Garrett’s framework and the stages of design were laid out. Each of the planes was explained and inputs and outputs of each plane described in detail for each of the attending stakeholders. Afterwards, data collection and discussions were targeted towards each of the planes by being announced in the beginning of each meeting and summarised at the end. At the beginning of the next meeting, the summary of the discussions and design updates was presented to the stakeholders. Also, when, albeit rarely, changes had a major impact on other planes, this was also outlined, and changes in the document were circulated to the stakeholders. This allowed stakeholders to mainly concentrate on each of the planes. 3.5 Case Study Background It is important that the context of the case study is discussed so that it can be understood and analysed correctly. The case study took place in the United Arab Emirates and the duration of the work took around 45 working days, excluding follow-up questions and related issues. There were four main parties involved in the case study: The governmental agency, project management company, development company, and The usability company. The governmental agency is one of the major governmental agencies in the country and plays a vital role in the rapidly growing and dynamically changing economy of the United Arab Emirates (UAE). The UAE is a relatively small country with a total area of 83,600 square kilometres. The country has seen a population boom: from 557,887 in Chapter 3 Usability Methods and Frameworks in Commercial Projects 69 1975 to 4,229,000 in 2006 to an estimated 8,132,491 in 2010. The GDP rose from 321.8 billion in 2003 to 599.2 billion in 2006 (UAE Dirhams). The UAE has one of the highest GDP per capita in the world. The governmental agency’s minister was personally involved in the development of the website and identified usability problems that the development company and internal team were not able to resolve. She therefore commissioned a usability company to be involved in the development of the website. The vice minister was then involved in the day-to-day development effort. The project management company is one of the major telecommunication companies in the UAE and one of the largest telecommunication companies in the world. The project management company took steps to diversify their revenues and expand into IT and specifically IT project management. The usability company’s involvement was twofold. On one hand, it was commissioned after the project had started; this was particularly true of the online services section, where whole parts of the actual code-base of the system had already been developed and the design was partially implemented. On the other hand, the portal itself - all of the other elements of the website other than the online services section - was still not developed, hence allowing the usability company’s involvement to be more concrete. 3.6 The Case Study The case study details the interactions between the usability company, the project management company, and the governmental agency to develop a usable website. The next section is broken down into two parts: a summary and a detailed account of the case study. During the consultation period, and because of limited timeframes, minor formal end-user input was captured through the usability company. However, alternative methods of evaluating usability were used, including context of use, competitor analysis, brainstorming, prototyping and heuristics analysis, to evaluate the design. Methods are the way to collect data and move forward with the design and therefore a crucial part of the design process; however, they need to be organised in an overarching 70 Chapter 3 Usability Methods and Frameworks in Commercial Projects Garrett Planes Strategy Scope Structure Skeleton Surface Methods Used Identify Stakeholders Competitor Analysis Brainstorming Stakeholders Analysis Evaluate Current System Context of Use Analysis Brainstorming Design Guidelines Prototyping Heuristic Evaluation Design Guidelines Prototyping Heuristic Evaluation Design Guidelines Prototyping Heuristic Evaluation Method Adapted From (Maguire, 2001) UsabilityNet UsabilityNet ISO 13407 ISO 13407 and UsabilityNet ISO 13407 ISO 13407 and UsabilityNet ISO 13407 and UsabilityNet All All ISO 13407 and UsabilityNet All All ISO 13407 and UsabilityNet All All Table 3.9: Mapping Methods to Garrett’s Framework structure to be able to be utilised fully. This is why a framework is required. Additionally, these methods need to be used in an appropriate manner depending on the stage they are being used in. There is no previous research on how methods fit Garrett’s framework; therefore, the researcher had to rely on how other frameworks categorised methods. The methods used in each plane are detailed in Table 3.9. These methods were adapted from previous frameworks and incorporated with Garrett’s planes in accordance with the mapping of the frameworks in Table 3.8. As described in Table 3.8, only methods used in different stages of design/development in other frameworks that overlapped with Garrett’s framework were used in this research. Additional methods could have been used, but due to limitations introduced by the project such as lack of time, budget, and user involvement, these particular methods were ultimately chosen. Specifically, they allowed for the input of usability methods without expending large amounts of time and budget. If the project had allowed it, other methods could have proved useful in the different planes. The methods were tested in relation to Garrett’s planes and evaluated in a real-world situation. Chapter 3 Usability Methods and Frameworks in Commercial Projects 3.6.1 71 Summary of the Case Study There were two main sections of the consultation work undertaken by the usability company: 1. Design a Web portal for the governmental agency. 2. Evaluate the current e-services and state of the templates for online services and propose changes to enhance the usability of the services. Designing the Web Portal In commencing consultation the brief was to evaluate the strategy and brainstorm how it will outline what the website should provide to customers and employees. Recent decrees and laws were considered as part of the strategy of the website and requirements from the federal government were also considered. The primary requirement was that the website should be entirely new and without relevance to the previous website. The owners of the website requested starting from a clean slate, and evaluating what was required and what was not before committing it to the design. This required a preliminary competitor’s analysis to figure out what was currently being offered by similar governmental agencies in other countries. A list of requirements and generic content that is commonly available in other similar Web sites was constructed and used as part of the scope definition. Several brainstorming sessions followed to prioritise the content and define what should be part of the website. When the content and its importance had been agreed upon, a structure was provided that showed how the content would fit together in the overall structure. A simple hierarchical structure was used. The structure was iterated several times until a viable alternative was created. The scope and structure plane outcomes helped to create a skeleton of the pages for the website. Several alternative skeletons of the homepage were created and then compared and discussed in meetings in order to present options for the best suited alternative. These alternatives were similar from a usability perspective but different in the layout of required content and function. The prioritising of requirements in the scope plane helped contrast between the presented alternatives. After the skeletons of both the homepage and sub-pages were decided upon, a graphic designer was brought in from the development company to design the 72 Chapter 3 Usability Methods and Frameworks in Commercial Projects majority of the surface plane. The colour theme, fonts, and general aesthetics were created in this plane with the help of the graphic designer. Designing the Online Services When the usability consultancy became involved in the design and development of the website, work on the online services section had already started. There had been some research done with expected end-users but only from a business perspective rather than a user-experience one. Since previous Web sites of the governmental agency had not included online services, surveyed users considered it a usability enhancement to just have services offered online. Additionally, even though the project management team had an overall plan and tried to foresee many of the issues, other issues came up that were not in the plan that changed the project time schedule. Finally, much of the code for the online services section had been done. All of the aforementioned issues created a situation where the recommendations made by the usability analysis required changes to the code-base which had already been created. These changes would be taxing, both on time and budget. Additionally, project members in charge of the online services section resisted any changes recommended to enhance the usability, arguing that it was not very important or that there was no time to implement the changes. Only minor structural changes were implemented to adhere to the recommendations. Other planes were neglected due to the lack of time; the project was already late and over-budget. The work done in the strategy and scope planes was relevant to both sections and allowed for the creation of the structure of the online services. This had to be detailed to be able to support the design of online Web services required by a recent decree from the federal government. To be able to alter the structure, understanding how the services are offered in the physical structure and their details were essential. The project management team had already created a detailed workflow analysis of the services the governmental agency offers the public. They had also enhanced the workflow so that it could be used online. This was the basis for redesigning the structure. After the structure was completed, the skeletons were discussed, but only minor changes were recommended. The surface required more extensive changes which was communicated through documents and during meetings. Chapter 3 Usability Methods and Frameworks in Commercial Projects 3.6.2 73 Case Study Details 3.6.2.1 3.6.2.1.1 Portal Strategy The strategy of the portal was to extract useful information from the governmental agency’s strategy. It focused on a recent decree that required all the governmental entities to provide at least 50% of their services online at by the end of the year the website was redesigned and to increase that to 75% the following year. The portal was required to provide the means to reach the services offered in an easy manner. Additionally, the portal should be a place where information is made available to all interested parties. Comprehensive information had to be provided as to laws and regulations and general procedures relating to how to do business in the UAE, how to establish a business within the UAE, and other relevant information. Additionally, the governmental agency was responsible for protecting the consumers’ rights and monitoring the prices of key products. Most of the information recorded here, however, was confidential and therefore could not be disclosed. However, the design recommendations of the portal followed the strategy of the governmental agency, which was agreed upon in subsequent meetings after minor modifications to the draft website strategy. 3.6.2.1.2 Scope In order to move forward with the design of the website, a list of required functionality and content was needed. This was drawn from the strategy already agreed upon and can be summarised as follows: Allow users with minimum computer knowledge to access relevant information and services with the minimal time and frustration. This included allowing users to be able to finish what they came to the website to do easily, enabling users to find more information about the governmental agency or start the process of a certain service online, learning more about the service and understanding the requirements. The scope was therefore to concentrate on making the website as usable as possible. The scope was defined to limit the possibility of changing requirements later in the project timeline. Defining the scope also limited the possible 74 Chapter 3 Usability Methods and Frameworks in Commercial Projects expansion of the scope of what is actually being designed and developed, which is considered a problem in projects. The governmental agency’s team members wanted an entirely new website that did not relate in any way to the previous one. The discussions concentrated on what was actually required for the strategy to be implemented through the portal. This led to a form of competitor analysis being undertaken: a list of similar Web sites were visited and analysed. A list of most common content found on Web sites was drawn up in a document. Also, interesting functionalities available on similar Web sites that might be beneficial to the governmental agency was noted. Subsequent meetings analysed the list of contents; this was changed accordingly based on the content available through the governmental agency and additional functionality was added to the list. Functionality, such as ’consumer price index’, was deemed necessary to establish a link between the governmental agency and citizens of the UAE by providing consumer support services such as supplying price lists of essential products sold in specific outlets. This was all made available in a simple spreadsheet that allowed consumers to easily compare and contrast the prices in each outlet. Two major types of users were identified through the scope: new visitors and returning visitors. New visitors were users that were new to the website and required general information such as information about the country, and regulations and laws when related to doing business, or establishing a business in the UAE. Returning visitors were identified as users that wanted more concrete information and wanted to complete tasks through the website; they were broken down into two categories: individuals/companies and agents. This included people wanting to establish a new business in the UAE or companies/agents wanting to register a trademark. One of the functionalities that was proposed by the usability consultancy allowed users faster access to services through the portal. The functionality was composed of the construction of three drop-boxes in the middle of the page. The first drop-box allowed users to access specific services offered to users through knowing the exact service name. The second drop-box allowed the users access to services by just knowing what they need to do but not knowing the exact name of the service. The third drop-box allowed the users access to frequently used forms for specific services that currently cannot be completed online but will reduce the user’s time while visiting the Chapter 3 Usability Methods and Frameworks in Commercial Projects 75 governmental agency. This functionality was directed mostly towards returning visitors to the website who are accustomed to the governmental agency and the services such an organisation provides. Concentration in the design of the homepage was on fast and easy access to information and services. This was evident in the strategy and is carried forward to the scope. All of the functionalities proposed met that basic requirement. 3.6.2.1.3 Structure The structure of the website needs to incorporate two main concepts: interaction design and information architecture. The structure plane is where the move from the more abstract to the more concrete takes place. Interaction design is concerned about how people use the website as a whole; information architecture, on the other hand, is concerned with how the information is organised and presented. Several iterations were done while designing the information architecture of the portal. After deciding what functionality and content was required in the scope plane, the effort proceeded to define the relationship between the functionalities and content defined. The competition analysis done earlier also helped in defining specifics regarding the most common structure of content and most common functionalities (Figure B.1). Following this, several brainstorming sessions and email communications were scheduled between the usability company, the governmental agency, and the project management company. The meetings enabled the team members to define and iterate the design of the structure (both architecture and interactivity) and produce the most suitable design. The information architecture is shown in Figures B.2, B.3, and B.4. The final structure was then agreed upon (Figure B.5). Interaction design was discussed in the context of what users want and require from the website and how the design should facilitate task completion. Due to the lack of project resources, formal end-user input was not considered in the structure section; however, extensive heuristic evaluation and informal end-user input were recorded. Some of the recorded end-user inputs were discussions about how they liked the aesthetics of the website and where they thought a specific link would take them. This was done on earlier versions of the website and on a small scale. Once the final structure had been finalised and accepted by the governmental agency, a 76 Chapter 3 Usability Methods and Frameworks in Commercial Projects skeleton design of the website was developed. Minor changes were made to the structure throughout the project that did not conflict with any previous analysis. 3.6.2.1.4 Skeleton The skeleton plane is where the concentration turns to interface, navigation, and information design. In the structure plane, the concentration was on the overall website. In the skeleton plane the concentration moves to individual Web pages. In this plane, the elements that will provide all the aforementioned functionalities will be implemented in the website. Elements such as navigation, labelling, and content areas are designed into the pages and discussed. The integration between these elements and the previous plane’s analysis is essential. As with the structure, several alternatives for the skeleton of the website were designed (Figures B.6 and B.7), changing various navigational and contextual elements of the website. Those skeletons were based on usability best practice and manipulated to adhere to the requirements of the governmental agency One of the designs was chosen by the governmental agency, followed by a brainstorming session on that decision by the usability company, the governmental agency, and the project management company to analyse the benefits and drawbacks of the chosen skeleton; it was concluded that, with minor changes, it would be the most suitable skeleton. The brainstorming sessions resulted in including more of the content on the homepage and targeting what the the governmental agency identified as important information. The skeleton of the home page was then finalised (Figure B.8). The sub-page skeleton was also designed to closely follow the design of the main page to decrease vagueness in the design (Figure B.9). Following such procedures, the designers and the client were able to agree on the most suitable design for the sub-pages. After the skeletons of both the home page and sub-pages were finalised, a transition to surface design was possible. Chapter 3 Usability Methods and Frameworks in Commercial Projects 3.6.2.1.5 77 Surface The surface plane is concerned solely with visual design. The layout in the skeleton plane is taken and then presentation issues are dealt with. Much attention is paid to the consistency of the visual design. This includes standardising colours and fonts used throughout the website. Working closely with the designer, the skeleton was used as a base to develop the look and feel of the website. The governmental agency identified colours that it considered to be suitable for the website; these were examined to ensure that the usability of the colour scheme was not in conflict with other principles of usability. The usability company also stipulated that there was a consistent design theme throughout and that there was no major deviation from the default design. There were some changes to the design implemented by the designer which affected the usability of the website which was brought up at the next meeting. Subsequently, all of the changes were recommended through the brainstorming sessions that followed to iterate the design of the surface to should be changed from that depicted in Figure B.10 to the design in Figure B.11. 3.6.2.2 E-Services (Online Services) This section of the design and development was more complicated than that of the portal. As mentioned earlier, the development of the online services section was nearly completed, so the usability company proposed to evaluate the current state of the online services and recommend changes using heuristic evaluation. This is useful for identifying usability issues, and recommending fixes that would integrate well with the portal. After documenting the current status of online services (Figure B.12) and evaluating all aspects of the services from a usability perspective which included navigational, labelling, and search elements, recommendations for changes to the current structure of the online services were proposed (Figure B.13). The structure had minor changes to be done. The structure only required minor changes which did not pose any major issues with other team members. Other recommendations, however, that related to the skeleton and surface of the Web pages did require major changes. The most important reason why team members responsible for the online services section resisted change was the fact that the project was over budget and behind schedule. Any type of change requested by either the governmental agency or 78 Chapter 3 Usability Methods and Frameworks in Commercial Projects the usability company was considered a burden. This was quite evident in the skeleton plane where most of the recommendations were concentrated. The majority of these recommendations were left out and omitted. This practice followed through to the surface plane where only minor recommendations were implemented; however, the majority were ignored. 3.7 Feedback and Discussion When the usability company was awarded the project, the usability consultation was actually broken down into two major parts: one started at the beginning of the project, and the other was undertaken at later stages in the overall development of the online services section as it came as a direct order from one the top officials in the governmental agency that recognised the lack of usability of the website. It was therefore requested that a usability company became involved to evaluate the current stage of development and recommend changes where considered necessary. This was done by the top management without consent from the other members of the governmental agency and other entities such as the project management company. This was evident from the lack of cooperation, especially at the start of the project. The lack of understanding of the importance of a user-centred approach was also evident in actions taken by individuals, such as the development company’s personnel trying to bypass the usability analysis team and go directly to the decision maker. It was fortunate, however, that the decision maker was a champion of the user-centred approach and therefore required input from the usability consultant in all aspects. This posed two problems. First, coming late into the development phase, in regards to online services, meant that the usability company would recommend changes to the design that might have already been developed, which would have been time and budget consuming, especially when the rationale behind the changes was not evident. Second, resistance to change was a major barrier from team members in the online services section and would have been present even if the usability company had been commissioned earlier in the design and development process. This is the case not only because it is a process where the agencies involved were not aware of the value of user-centred design methodologies in website design, but also because the management enforced the involvement without consent. This was less of a problem in regards to the Chapter 3 Usability Methods and Frameworks in Commercial Projects 79 portal design and development. The usability company proved to be a major asset to the project in providing easy solutions and recommendations to problems facing the design and development teams, which subsequently came to rely on the usability company for feedback and suggestions. Being able to lay down a design path and follow it through helped keep the project on time and within budget and, together with the help of an excellent project management company, the finished product was usable. To overcome issues surrounding changing what had already been developed and resistance to change, the usability company took an additional step to actually rationalise all aspects of the design proposed to all the parties involved in the weekly meetings which were opened up for general discussions. This allowed other members of the project to know about, and understand where the changes were coming from and why they were important. Those members involved in the design and development of the governmental agency’s website eventually changed their view towards usability when they saw how usability analysis and design enhanced the user experience through the design and development of the website by thinking about the user in every decision being made regarding the website. They realised that the input from the usability company also focused on user satisfaction - making sure that everyone shared a common goal, and agreed that the website actually satisfied that common goal. Project managers from the governmental agency and project management company agreed. A senior manager from the project management company commented that ”being involved with usability allowed an understanding of some of the requirements that we could have addressed in previous projects and that I am consistently thinking of usability not only in this project but all of the projects I am involved in.” Members from the governmental agency group said that ”laying out the process in this manner helped us to concentrate on the task at hand rather than worry about something that is ahead or something that is already decided upon through the process.” This was due to the fact that the decisions made were discussed and agreed upon and supporting evidence had been provided, and that focusing on the task at hand correctly would help develop the next phase. Also, it was not necessary to design and develop the website from scratch because of the availability of a usability design document that 80 Chapter 3 Usability Methods and Frameworks in Commercial Projects gathered the aforementioned information which helped to minimise, or in some instances, even eliminate the guesswork usually present in Web development efforts. Feedback was requested from involved members at the end of the usability consultation. Members of the governmental agency’s team had a positive experience when the design and development was user-centred. The IT manager in the governmental agency said that when the usability emphasis was introduced, the design and development was “more consistent, more focused, and easier to use” when asked how this development effort compared to the several attempts they had made previously. Additionally, he considered not taking usability into consideration in previous development efforts a “major mistake”. One of the webmasters of the governmental agency’s website said that this project differed from previous projects in that it was based on a “clear hierarchical structure” provided by the usability document which allowed the development to progress more smoothly than in previous development efforts. Another webmaster involved in previous projects felt that most were concerned with what the departments wanted rather than what the end-user required. She went as far as to say that without usability, “perfection will never be achieved ” when developing a website. She said that ”while developing the website, usability allowed for the defining of a target and then pursue it so that performance can be measured and designs can be iterated accordingly, this could not have been done without a precise usability evaluation of the requirements of users and stakeholders.” It would have been beneficial to be able to capture the perception of members regarding the usability process at the beginning of the project and compare that to the feedback captured at the end of the project. Since the researcher was only a participatory observer and was only brought in at the mid-project stage, biased attitudes were already in play because of the project taking longer and costing more than expected. People involved in the project were resistant to any change or any enhancements that occurred regardless of its benefit. Also, when the project was completed, most of the people who participated in the project did not fill in a survey sent to them, even after numerous attempts to encourage them to do so. This might have been due to the fact that many are tied to other projects and considered this project completed. These comments highlight the need to introduce HCI methods early on in the project development stage in order to engage all parties in the significance of Chapter 3 Usability Methods and Frameworks in Commercial Projects 81 usability and what a difference it can make to the end product. What is obvious from this small-scale usability case is that there still exists a lack of ‘usability culture’ within the commercial world, especially in relation to gathering opinion and feedback from actual users. Early communications regarding the value of usability methodologies to all parties involved is important in order to secure commitment from users, designers and decision makers alike; an issue that HCI specialists need to address if methods and evaluations are to be utilised more effectively in the future of Web design. Garrett’s (2002) framework proved to be a major facilitator of communication between the usability consultant and other teams because it provided a common base using the five planes where more technical issues were omitted for the sake of user-centred design and development. Members from the IT department perceived more hierarchy and focus on users and stakeholders, through the use of Garrett’s framework, which allowed for building a web site that catered to the stakeholders needs, something that was not done in earlier web site design projects that the department had overseen. The project management team members saw a concentration on important usability requirements that aided in the success of the project, something that was not part of the requirements prior to the usability consultancy’s involvement. Owners of the website which included the managers and other high-ranking personnel saw a clear structure in the method provided through Garrett’s framework, which allowed the website design to focus on each task in a systematic and consistent manner, concentrating on each plane at a time. Utilising this framework therefore enabled those involved to follow a logical process that was consistent with their own mental model but without them worrying about the next step, as Garrett’s framework followed a logical process that guided the thinking of those involved from the abstract to the concrete. The technical issues were managed within the requirements that emerged from the discussions and usability analysis. If the requirements were hard to implement through discussions and usability analysis in this phase of development, then they were moved to the next phase, and even though they were not implemented they were documented for future use, allowing stakeholders the opportunity to improve the website with less effort. Even though Garrett’s framework has its shortcomings as mentioned previously, it helped all the members to become involved in understanding the requirements of the end-users, regardless of the complexities of the underlying technologies. The usability consultant was not able to fully implement the framework as described by Garrett 82 Chapter 3 Usability Methods and Frameworks in Commercial Projects because of late entry as the usability consultant. However, even with the partial implementation of Garrett’s framework, its value was obvious as it helped in decreasing the level of miscommunication because of the lack of common foundation. This was achieved by using a unified language in clarifying each stage of where the project was at each point in the discussion, so that everyone’s concentration was directed towards that particular stage. For instance, when the project was at the structural stage, all discussions were about the structure rather than the colours and fonts to be used, which are elements of the surface stage. 3.8 Limitations and Future Work It is clear that often, in the real-world, usability is an after-thought; therefore it is necessary for researchers to develop methods of analysis and evaluation that are both rigorous and valid as well as being easily utilised and implemented. This is a complicated task and requires concerted effort to immerse oneself into a project that is commercial in nature so that observations can be made in the actual context of use rather than a laboratory setting. The author intends to extend Garrett’s (2002) framework to make it more adaptable to real-world situations. The problem at hand is twofold. First, Garrett’s framework should be extended to include usability methods if it is to be of any real benefit. This is due to the fact that usability methods are not integrated within Garrett’s framework (Hodgetts et al., 2005). There is a plethora of methods available, and categorising these methods within Garrett’s framework will facilitate the successful selection of the most appropriate method for each stage. Several steps could be taken to attain the goal of enhancing usability through the application of Garrett’s framework: 1. Selection of appropriate methods used in this case study as shown in Table 3.9. 2. Evaluation of Garrett’s framework against other currently available frameworks to ensure its compatibility with the development aspect of Web design/development from the outset. Benefits Pitfalls Users Designers Developers Shown an overall design pro- Part of the usability analy- Part of the usability analysis Allowed to develop a web- cess and where the design is sis effort as representatives effort and therefore are part site with a clear design doc- currently at. This allows the if possible. If not possible of the design process from ument that satisfies the re- owner to see how the design then user intentions are used a usability perspective. De- quirements of users and stake- process is progressing, the de- through methods that does sign decisions are then made holders. cisions made within each of not require users involvement to satisfy the requirements of ment is then used as a ref- the five phases, and the ratio- which will aid in developing users and stakeholders rather erence when changes to the nal behind these decisions. than other criteria. website is required. more user-centred Web sites. The design docu- Might consider usability anal- Might lack the time or bud- Concentrate on more techni- Design document might not ysis and evaluation a sec- get to take users requirements cal design issues and decrease be specific enough to develop ondary objective and there- from a usability standpoint as the importance of usability the entire website because of fore give less priority to it. part of the design process and because of the lack of time incomplete requirements. Re- This results in decisions being therefore end up with a less and budget. Try to overpass turning to the usability team made that does not take into usable website. usability consultation for the might be required to alleviate sake of speediness. any vagueness. consideration the users and Chapter 3 Usability Methods and Frameworks in Commercial Projects Owners stakeholders requirements. Table 3.10: Summary of Benefits and Pitfalls of Frameworks in Commercial Projects in Relation to Stakeholders 83 84 Chapter 3 Usability Methods and Frameworks in Commercial Projects 3. Evaluation of the reliability of Garrett’s framework as a facilitator of incorporating usability methods within Web design projects. These three steps, if addressed, would allow the development of more user-friendly Web sites without negatively affecting the budget and time constraints of the project because of the integration of usability methods from the beginning of the project. The author considers such a process would be an improvement to the current status of usability which is often thought of and implemented as an afterthought. If those involved worked through a systematic and consistent process as described above early on in all Web design projects, then all requirements would be seamlessly integrated and considered a part of the design/development efforts. 3.9 Conclusion The current case study utilised Garrett’s (2002) framework whilst developing a large-scale website. It focused on the difficulties experienced in implementing usability methods in a ’real-world’ situation that had limited resources and tight deadlines. It also discussed the barriers of entry regarding usability analysis. Furthermore, this case study examined the usability analysis as part of the whole development of a portal and online services for a government agency as well as examining Garrett’s framework, which identified many of its strengths and weaknesses. Finally, the case study allowed a greater insight of the workings of the website design and development and gave a glimpse of alternatives to better integrate usability methods and frameworks in commercial projects. Appendix B Figures 85 86 Appendix B Figures Figure B.1: Initial Portal Structure Design Appendix B Figures Figure B.2: 1st Phase of Changes in Portal Structure Design 87 88 Appendix B Figures Figure B.3: 2nd Phase of Changes in Portal Structure Design Appendix B Figures Figure B.4: 3rd Phase of Changes in Portal Structure Design 89 90 Appendix B Figures Figure B.5: Final Portal Structure Design 91 Appendix B Figures Figure B.6: Initial Portal Skeleton Design 92 Appendix B Figures Figure B.7: 1st Phase of Changes in Portal Skeleton Design 93 Appendix B Figures Figure B.8: Final Portal Skeleton Design 94 Appendix B Figures Figure B.9: Sub Pages Skeleton Design 95 Appendix B Figures Figure B.10: Initial Portal Surface Design 96 Appendix B Figures Figure B.11: 1st Phase of Changes in Portal Surface Appendix B Figures Figure B.12: Initial Online Services Structure Design 97 98 Appendix B Figures Figure B.13: Final Online Services Structure Design Chapter 4 Stakeholders in Web Design: A Theoretical Framework Abstract: This paper evaluates current and available HCI frameworks used in Web design and theorises about the effects that adding different stakeholders at different stages of the design would have on the whole project in terms of effort/time and bottom-line return on investment. It compares three possible alternatives: generic/nirvana; current practice; and a proposed alternative approach in which the user is included from the beginning of the design process without greatly affecting each stage of the overall project progress. The paper concludes by defining areas in research that, through researchable propositions, classify stakeholders in Web design, validate team creation and team member addition theory, and define the amount of involvement each of the stakeholders has. 4.1 Introduction Involving the users in the design and development of Web sites is beneficial for the success of these Web sites and in achieving higher user satisfaction (Kujala et al., 2005; Karvonen et al., 2010). The lack of practical understanding of how a Web design project progresses, however, hinders the implementation of rigorous usability methods. This ideology has been present in HCI research since as early as 1992 (Carroll and Rosson, 1992). Web design is regulated by frameworks that organise what activity is 99 100 Chapter 4 Stakeholders in Web Design done, and when. When discussing frameworks of HCI in the design realm, however, two paradigms emerge according to Carroll and Rosson (1992). First, there are frameworks that concentrate solely on users making them the centre of the design process. These are usually frameworks that are theoretically sound but lack validity from being tested in commercial projects. Second, there Figure 4.1: Human Centered Design are the frameworks that focus on helping designers incorporate HCI methodology into their design process. These frameworks are more centred on previous design experiences and commercial knowledge where theory plays a relatively small role in their design projects. The aim of these frameworks is to simplify the design process that emphasises usability by providing designers with a check-list (Kärkkäinen and Laarni, 2002; Nielsen and Molich, 1990) or a list of design activities (Häkkilä and Mäntyjärvi, 2006; Wharton et al., 1994) among other techniques. These frameworks also widely vary in their involvement of different stakeholders. Historically, owners and designers are the main stakeholders in the design process with minimum, if any, input from other stakeholders in later stages, including users. This is evident by the poorly defined requirements and poor communications between stakeholders; the top two reasons out of 12 as to why software fails, another being stakeholder politics (Charette, 2005). Additionally, there is a lack of usability requirement definitions in call-for-tenders from companies requiring Web design or redesign (Lehtonen et al., 2010). This forces the usability practice to be pushed to the end of the development; critically too late to do anything more than artificial changes (Carroll, 2003). Finally, HCI frameworks vary in the scope they cover; some frameworks encapsulate the total development effort (Bevan, 2003; Maguire, 2001; Mayhew, 1999) whilst others focus on the usability aspect of the design (Garrett, 2002; Yen et al., 2005). Design, in most of these frameworks, is listed as a single entity (Maguire, 2001). Some (e.g., Garrett, 2002), however, break down design into several stages each with distinct requirements and outcomes and include the earlier stages 101 Chapter 4 Stakeholders in Web Design such as planning and requirement gathering within each stage. Human Centered Design (HCD: Figure 4.1), for example, defines the boundaries of the field as the designers’ and users’ contribution to the design effort (Keinonen, 2009). Other stakeholders, according to Keinonen (2009) are included as either designers or users. For example, managers do not title themselves as designers but yet participate in the design process and so are implicated as designers. Another example is where users of a product brought in to help develop a new model of a product are considered both users and designers. In a commercial Web design project, both users and designers are vital for the success of a website design effort and their input should always be considered. However, more stakeholders exist and should be considered separately and individually because of their individual requirements, type of knowledge, motivations, and the influence they have on the design project. They should, therefore, not be just grouped together as either users or designers. These stakeholders include owners, managers, developers, designers, and users (Figure 4.2). Stakeholders, according to Freeman (1984), are “any group or individual who can affect or is affected by the achievement of the organization’s objectives” (p. 46). Within a design project, however, not all stakeholders are considered because of the limited amount of resources available (Gulliksen et al., 2006; Vredenburg et al., 2002). None of the frameworks currently available, however, focus on the Figure 4.2: Stakeholders in Web Design involvement and value of different stakeholders in a design project. Focusing on specific stakeholders without taking into consideration the views of others in the design process makes it difficult to integrate the requirements that are relevant to the overall plan of design. Also, each of the frameworks with narrower scope has separate planning and gathering requirements that might be, and usually are, beneficial to the overall design/development effort. Considering the significant benefits 102 Chapter 4 Stakeholders in Web Design that could be gained from including all stakeholders’ contribution in Web design and the current lack of evidence in both academic arenas and professional practice, this paper aims to conceptualise the value of incorporating different stakeholders in the design process by adapting Garrett’s (2002) design-only framework. The choice of a design-only framework rather than a design/development framework emanated from the sense that development should not commence until the design is completed to ensure that it fits the requirements. This idea, presented exceptionally early on in the HCI discipline (Zahniser, 1983), was found to be beneficial by increasing both product quality and user satisfaction (Charette, 2005). Garrett’s framework was chosen specifically because it breaks down the design into five different planes. This allowed for inclusion of different layers of details through the process of moving from more abstract to more concrete levels of abstraction. These details may have otherwise been omitted due to a effort of trying concentrating on everything at once. Moving through the levels of abstraction also allowed those involved in the design to focus on specific requirements of that plane rather than concentrating on all the requirements of all the planes. For example, in the strategy plane, the focus is on more broad necessities such as setting the objectives of the website and success metrics. On the surface plane, however, the focus is on more specific details such as fonts and colours. The scope of this paper will focus on the three most influential stakeholders who are relevant to the design and analyse the ways in which these stakeholders interact with, complement, or conflict with each other whilst designing the website. These stakeholders are owners, designers, and users (Figure 4.3). Each group brings a unique set of requirements relating to, and contributions to, the design of the website. Additional discussion on what constitutes a stakeholder is done later in the section ”Stakeholders in Web Design”. In general though, and based on Freeman (1984)’s definition, stakeholders can be anyone related directly or indirectly to the company. It is argued, however, that not all of the stakeholders should be represented in the design team, as discussed by Preece et al. (2002). Furthermore, it is out of the scope of this paper to concentrate on all stakeholders. However, if a stakeholder is considered important and is to be included in the design team, the inclusion process is similar to that of the aforementioned three; analysis through stakeholder theory (Mitchell et al., 1997) and evaluation of the benefits of involvement at different phases of Garrett’s framework. These three stakeholders were chosen because they are significantly Chapter 4 Stakeholders in Web Design 103 different from each other in regards to the different attributes explained by Mitchell et al. (1997). Figure 4.3: The Three Stakeholders’ Scope Definitions of terms are given below in order to aid understanding of stakeholders and narrow the use of words being used in this paper. ❼ Owners: The entity or person who owns the website and intellectual property of the produced website. ❼ Managers: The people who act on behalf of the owners in the day-to-day management of the design effort. ❼ Developers: The entity or group of people responsible for creating the website, especially the coding of the website. ❼ Designers: The entity, group of people, or person responsible for the creative aspects of Web development including graphic design. ❼ Users: The end user that the site is designed for. Depending on the website being developed, they could be customers, employees, administration, or management. ❼ Design: A part of the development process concentrating on the creation of the website: articulating what the website’s requirements are, how are they going to be implemented, and the blueprints for the structure and individual pages. 104 Chapter 4 Stakeholders in Web Design ❼ Development: The process of building the website based on the design semantics produced in the design phase. It is the full circle from the beginning of the design process to the launch of the website. Development includes the notion of design as part of it. 4.2 Current HCI Frameworks There are several HCI frameworks currently available that attempt to organise and include users and user requirements in a design project. Most of these frameworks are targeted towards designers with the purpose of including users, or at least to think about users, when designing. Two distinctive eras emerge: one prior to the Web and one after. The pre-Web era of research focused on users and, to some extent, stakeholders, through the notion of user-centred design which originated from Norman and Draper (1986)’s work. These include CD notations (Green and Petre, 1996) and task-artefact frameworks (Carroll and Rosson, 1992). There are two distinctive lines of research in this regard: one that focused on user-centred design and the role of the user in the design and development process (Norman and Draper, 1986; Leffingwell and Widrig, 1999; Rozanski and Woods, 2005), and a second which focused on involving stakeholders in software and systems design/development but without a concentration on a usability and user-centred approach (Girgensohn et al., 1994; Ben-Chaim et al., 2010; McManus, 2005; Watson et al., 2009; Jordan and Henderson, 1995). Prior to the notion of user-centred design, the design activity took on a more functional aspect. This did not require any user input into the design process. After the publication of Norman and Draper (1986)’s book, however, the idea that users are what the product is being designed for became more prominent. Many researchers afterwards investigated how to gain users’ involvement in design through designing a language of discussion when talking about design (Green and Petre, 1996), and the requirements and artefacts that result from these designs (Carroll and Rosson, 1992). In much of the literature mentioning stakeholders in design, stakeholders are defined as only developers and users (Girgensohn et al., 1994), as users (Boutelle, 2004; Watson et al., 2009), or a larger group of stakeholders broken down into primary, secondary, external, and extended groups (McManus, 2005). These papers however focus largely Chapter 4 Stakeholders in Web Design 105 on the managerial and design aspects of Web design rather than usability. Examples include the frequent mention of managing requirements (Leffingwell and Widrig, 1999), supporting communications (Girgensohn et al., 1994), and analysing and evaluating stakeholder performance (McManus, 2005). Even though the above mentioned research was carried out prior to the existence of the Web, the ideas prevalent in it are relevant to interaction with the Web and form the basis for the ideas represented in post-Web research; research that has been done after the invention of the Web which focus solely on Web issues, that have been referenced in this paper. The aim in this paper, therefore, is to merge these two lines of thought by incorporating stakeholders in the design process through the focus on usability and user-centred design. Web design is the main research area because, even though the general software design and Web design literature is largely similar, some differences are known to exist (Murugesan et al., 2001). The cognitive dimensions of notations framework (Green and Petre, 1996) focussed on the language of discussion when talking about interfaces, rather than on the interface itself. The notations are created to help designers evaluate the designs at hand while talking in the same language; the CD notations. The CD notations are not constrained by the content of the design but rather design decisions (Green, 1989). The CD notations also allow designers the ability to “evaluate their designs with respect to the impact that they will have on the users of those designs” (Blackwell, 2006, pg. 325). The CD notations, however, do not provide designers with a process of how to go about designing the interface. The focus here is only on the language used. Furthermore, the CD notations do not include other stakeholders other than designers, and the intentions of the user, without enforcing the involvement of users, due to time and budget constraints or the unavailability of users. The task-artefact framework, on the other hand, was created as a bridge between the design decisions and the reasoning behind those decisions (Carroll and Rosson, 1992) and aimed to “support better utilization of behavioural, cognitive, and social science in Human-Computer Interaction (HCI) design” (Hao et al., 2010, p. 22). The Task-Artefact cycle consists of task-setting requirements and then artefacts creating new and unexpected possibilities or creating new constraints on the tasks at hand. The task-artefact framework is a precious tool when examining a current design with the aim of planning to redesign; it allows designers to evaluate not only the design but also the reasons behind it. 106 Chapter 4 Stakeholders in Web Design However, even though the task-artefact framework examines the design from an HCI perspective, stakeholders other than designers are rarely involved. In an approach to guide the user-centred design, the ISO 13407 (Bevan, 1999) has a more general approach to design. Rather than focusing on a single aspect, the ISO 13407 incorporates a five-step process: planning the human-centred process, understanding and specifying context of use, specifying user and organisational requirements, producing design solutions, and evaluating design against requirements (Figure 4.4). The process then has a feedback loop from the fifth to the second step. This issue, Figure 4.4: ISO 13407 namely requiring a designer to go through the whole process before getting any feedback on the design, is a general problem with frameworks that follow the system development life-cycle or waterfall model. A related issue with the ISO 13407 is the rigidity of the process; the next step cannot start until the previous step is done. The ISO 13407 also concentrates solely on the HCI aspect of the website design; it does not lend itself well to the overall project development effort. Many illustrations show the framework as being similar to the generic system development life-cycle (SDLC) as shown in (Figure 4.5), which is considered slow and unreliable (Swenson, 2007; Carroll, 2003), less accommodating to changes as the design progresses (Folmer and Bosch, 2004), and less accommodating to changes in general (Ragunath et al., 2010). Research has also found that the ISO 13407 does not align well with the definitions of usability as stated in the ISO 9241-11 (Jokela et al., 2003). Finally, the ISO 13407, even though it provides relevant and valuable knowledge and is targeted towards designers, is not designer-friendly (Bevan, 2009; Carter, 1999; de Souza and Bevan, 1990). The TRUMP (TRail Usability Maturity Process) (Jacko and Stephanidis, 2003) tried to overcome the shortcomings of the ISO 13407 by mapping the ISO 13407 processes onto the Chapter 4 Stakeholders in Web Design 107 Figure 4.5: ISO 13407 as SDLC system life-cycle (feasibility, requirements, design, implement, and release). It does this by grouping ‘Plan Process’ and ‘Specify Context of Use’ into ‘Feasibility’. ‘Specify Requirements’ is put under ‘Requirements’. ‘Design Solutions’ are included in ‘Design’. Finally, ‘Evaluate against Requirements’ are separated into ‘Implement’ and ‘Release’. The TRUMP also tries to incorporate usability methods within the framework to facilitate the use of the most suitable method in any given stage of development. Although the TRUMP provides a structure to ISO 13407 through a general design/development framework, and because of choosing the system development life-cycle structure, it has the same issues mentioned in the discussion of the ISO 13407: slow and unreliable (Swenson, 2007; Carroll, 2003), less accommodating to changes as the design progress (Folmer and Bosch, 2004), and less accommodating to changes in general (Ragunath et al., 2010). UsabilityNet (Bevan, 2003) took a different approach by targeting usability professionals. In an effort to provide these professionals with recommended methods to be used in commercial projects, the methods were grouped into six stages: planning and feasibility, requirements, design, implementation, test and measure, and post release. The framework helps designers and usability professionals to choose the methods best suited for their situation. The methods are filtered according to three criteria: limited time/resources, no direct access to users, and limited skills/expertise. Choosing one (or more) of the filters highlights the methods suited for the design activity. The UsabilityNet does not present a way of categorising methods but rather 108 Chapter 4 Stakeholders in Web Design optimises the way the methods are used. The framework, however, other than an indication of the “stakeholder meeting” under planning and feasibility, does not show where stakeholders are involved in the different stages. Also, as with the other frameworks, the emphasis is only on users rather than all the stakeholders. Further, the UsabilityNet framework gives an overall view of where to use methods; however, because it is covering the whole development process, the scope is increased to the extent that it loses some of the necessary details required, especially in design. Garrett’s (2002) framework moved towards a different direction, focusing only on the design aspect whilst including planning and requirement gathering. The author discussed a five-level development process: Strategy, Scope, Structure, Skeleton, and Surface. This is a step away from the generic planning, requirements, design, development, and evaluation process usually followed by other frameworks (e.g., Bevan, 2003; Jacko and Stephanidis, 2003; Maguire, 2001; Mayhew, 1999). Additionally, each stage in Garrett’s framework moves from an abstract to a more concrete level. These stages, which the author calls “planes”, are articulated to ensure that all aspects of the user experience are covered when designing. The first plane, strategy, identifies what the owners and users want from the website. The scope plane converts the strategy objectives into specific requirements in respect to both content and function. The structure plane develops a conceptual structure of the website. The skeleton plane defines how the pages will look like in an illustrative format or “wireframes” as the author describes it. Layouts of page elements become clear in this plane: navigational elements, labelling elements, and content are all given an area in this plane. The final plane, surface, is what a user first sees when they visit the website: fonts, colours, and other visual elements are detailed in this plane. The level of abstraction, used by Garrett (2002), is a concept that has been employed in many fields to reduce the complexities of the problem at hand. This is done by moving from an abstract to a concrete level of abstraction, and to reduce complexities. The idea is to be able to have a wider more holistic view at the higher levels of abstractions and progress to the increasingly specific details in the concrete levels. This removes unnecessary details from the overall view of the conceptual design and allows details to be handled at the appropriate time, thus ensuring all necessary details are dealt with and not left out of the design process (Zahniser, 1983). For example, Rasmussen’s (1986), as cited in Stanton et al. (2006), Levels of Abstraction Hierarchy Chapter 4 Stakeholders in Web Design 109 defined five levels of cognitive abstraction when analysing how process control operators in Human Supervisory domains perceived the system interfaces. The five abstraction levels are functional purpose, abstract function, generalized function, physical function, and physical form. At the top of the hierarchy is the functional purpose that defines the overall meaning of the system through a global lens and the strategy it carries. The physical level, on the other hand, gives an accurate representation of the system including all the components and connections. Another example (Zahniser, 1983) is from software engineering. The system development life cycle (SDLC) is mapped to an abstraction framework. The author defined seven levels of abstraction: “Nirvana”, Functions, Data, Boxes, Control, Builds and. finally, Problem Solution. As with Rasmussen’s hierarchy, Zahniser’s life-cycle start with the most abstract level called “Nirvana” a concept that only exists in a user’s mind. The most concrete level is the Problem Solution where the produced system is installed in the user’s environment that alleviates the problem the user had conceptualised in the “Nirvana” stage. This allows the development process to continue from the end of the framework, easily utilising the output of the process; the design is implementable with most, if not all, of the questions answered. Abstraction, through Garrett’s (2002) framework, is beneficial to all the stakeholders involved in the design of the website as it allows discussion and input from different stakeholders without using overly technical terms in either HCI or development. Choosing not to cover the later development process is also beneficial in limiting the complication of the discussions. The end design details what the stakeholders require from the website and leave the underlying hardware and coding to the developers. This also simplifies development because developers are presented with clear details of what the stakeholders require from the website. Evaluation on the nearly-completed website is also done from a design perspective rather than an overall evaluation of the system; as long as the hardware and coding is able to provide the necessary outcome. For example, if the loading speed and compatibility with different Web browsers is correct, then the designer is less concerned about the underlying coding than how it is presented to the user. Garrett’s framework provides technical and non-technical team members with a common ground, namely the five planes, on which to facilitate discussions using a common language and concentrate on usability issues that the website is currently experiencing, or is likely to in the future through abstraction. This, also, is a step away from the overly technical discussions that involve technical terms that are imposed by the stages they fall within 110 Chapter 4 Stakeholders in Web Design that usually alienate other stakeholders that are not accustomed to such technical terms. It also gives the wider perspective of Web design as all stakeholders are able to give attention to the five planes in a more general sense rather than be concerned with specific technical or managerial details. These five planes also direct the effort towards the users’ requirements, rather than them having to become embroiled in technical aspects of design and development. This means that this framework can be easily embedded into a business context and is, therefore, more likely to be accepted by potential users. Garrett, however, other than focusing on the choices of design, does not explicitly involve other stakeholders in the design process. Also, the planes are depicted as equal which this author considers to be an oversimplification of the process. To summarise, the literature shows that there are several frameworks in the HCI field that focus on design. These reviewed frameworks generally suffer from one or more of three concerns; scope, lack of stakeholders’ perspective, and redundant activities. The current frameworks either concentrate solely on the HCI aspect of the design and are treated as a separate module from the overall process of design, or try and encapsulate the entire development of the website by trying to control the input of users throughout. When the scope is narrow and focuses only on the HCI aspect, it affects the design effort because this is considered a separate process undertaken outside the realm of the design. When the scope is broad, it overwhelms the project with an array of requirements and processes that should not be considered until later in the project. The other problem with current frameworks is the lack of stakeholders’ perspective. This is particularly evident in the wider-scope frameworks. There is usually a stakeholders’ meeting defined at the beginning of the process (e.g. at the ‘plan’ phase), but then there is no indication of the input of stakeholders throughout the continuing process. Incorporating stakeholders’ perspectives early on in the design, and throughout, ultimately has to be beneficial to the project. The final issue is the creation of redundant activities. When trying to incorporate narrower-scope frameworks into the design effort, several redundant steps are created. For instance, the ISO 13407 requires a planning phase and a requirement-gathering stage. These steps, and by implication the information gathered, are usually undertaken in the overarching framework that tries to utilise the ISO 13407. The next section will focus on three fundamental issues in relation to adapting Garrett’s framework into a comprehensive and appropriate method of Web design. Chapter 4 Stakeholders in Web Design 111 First I will discuss stakeholders in more detail, reflecting their input and importance in Web design. Second, a brief analysis of the current literature on teams and information sharing will provide some context relative to considering stakeholders as team members. Finally, the link between stakeholders and Garrett’s framework will be explicated, as a precursor to analysing how involving additional stakeholders in Web design theoretically affects each of Garrett’s planes and the overlap between them. 4.3 Stakeholders in Web Design Figure 4.6: Stakeholder Typology (Mitchell et al., 1997) Several definitions of the concept of stakeholders are available. These definitions vary from a broad to a narrow view. Broad definitions of stakeholders include “individuals or organizations who stand to gain or lose from the success or failure of a system” (Nuseibeh and Easterbrook, 2000, pg. 39), “any group or individual who can affect or is affected by the achievement of the organization’s objectives” (Freeman, 1984, pg. 46), and “groups to whom the corporation is responsible” (Alkhafaji, 1989, pg. 36). Narrow definitions of stakeholders include “those who have placed something at risk in relationship with the firm” (Clarkson, 1995). Mitchell et al. (1997) defined stakeholders according to three attributes. This enabled a distinction to be made between the 112 Chapter 4 Stakeholders in Web Design importance of stakeholders to an organisation and who are given priority by managers. These three attributes are power, legitimacy, and urgency. The authors then follow with an analysis of the eight types of stakeholders that are created from the interaction of these three attributes (Figure 4.6). Dormant, Discretionary, and Demanding stakeholders are those who have only one attribute (power, legitimacy, and urgency, respectively). Dominant, Dangerous, and Dependent stakeholders are those who have two attributes. Dominant, Dangerous, and Dependent stakeholders have power and legitimacy, power and urgency, and legitimacy and urgency, respectively. Finally, Definitive stakeholders have all three attributes. The position of stakeholders in the topology has a significant impact on Web design. Even if several stakeholders are present within a stage of Web design, their relative position in the stakeholders’ topology influences how their input is treated. This is particularly true when one of the stakeholders is what can be considered as dangerous whilst the other is what can be considered as discretionary. The assumed outcome here is that the dangerous stakeholder will have opinions that could be enforced whilst those of the discretionary stakeholder will be ignored. Additionally, the angle from which these stakeholders are viewed should be defined. They can be considered as stakeholders in the company, who own the website or stakeholders in the website itself. This minor difference effects stakeholders’ classification according to the Mitchell et al.’s stakeholders’ typology. For example, users from the company’s perspective might be classified as discretionary because of their legitimate claims as customers. When, however, users are viewed in respect to the website itself, they can be either considered dependent because they have legitimacy and urgency but not the power, or as definitive if that power has been bestowed upon them by owners requiring their input and interaction in the design process. The same applies to owners; from a company perspective, they could be considered dormant because they have the power to act upon the matter but choose not to have a say. From a website perspective, however, they could be considered dominant because they have the legitimacy, because the website is believed to be built on both owner and user requirements, and the power to see their input is implemented. Owners could also be considered as dangerous because they experience an urgency to finish the design and development process in an attempt to save resources and the power to implement their plans without being a representative of the strategic plan of the website and, therefore, lack the legitimacy attribute. The classification of stakeholders, therefore, needs to be embedded in scenarios to be able to better clarify Chapter 4 Stakeholders in Web Design 113 the reasoning behind the classification process. This is done later in the “Changes in the Stakeholders’ Involvement” section. 4.3.1 Stakeholders as Team Members It is essential that stakeholders work as a team in Web design because the information required transcends any individual and aids in the creation of shared understanding amongst all the stakeholders which can lead to new insights, ideas, and artefacts (Arias et al., 2000). The choice of describing stakeholders as a team rather than a group is multifaceted. According to Katzenbach and Smith (2009) 1. Teams tend to share the leadership role rather than have one strong leader. 2. Team meetings are observed to encourage open-minded discussions and active problem solving as opposed to focussing solely on efficiency as observed in group meetings. 3. Because members are interdependent and usually have a shared goal, teams involve genuine collaborative work rather than the delegation of tasks which is often the method/outcome of work group meetings. Working as a team can also increase effectiveness given that all members have a shared understanding of the task or situation at hand (Duncan et al., 1996; Kozlowski and Ilgen, 2006; Marks et al., 2002). Furthermore, stakeholders working as a team eventually help to reduce the individual cognitive workload of each of the team members (Beith, 1987; Hollingshead, 1998; Prichard et al., 2010). Finally, it has been shown that robust dialogue between team members is crucial for successful group problem solving and learning (Hausmann, 2006). However, this seems to be true only when information is shared early on in the decision process with information discussed later in the decision process having limited impact on the team decisions made (Larson et al., 1994). Research has shown that the most successful teams begin as diversified in their thinking but then develop more consensus as they progress (Kilduff et al., 2000). Furthermore, smaller teams are more likely to add new members in order to increase the amount of human capital at the disposal of the team (Ucbasaran et al., 2003). 114 Chapter 4 Stakeholders in Web Design It is expected that when teams start working together differences arise at the commencement of their life cycle as more effort is invested in the team not only for the task at hand but also for communication between the team members. However, these differences are soon alleviated as the team takes on its own identity and develops; then it can start to perform more efficiently (Tuckman, 1965; Tuckman and Jensen, 1977). Effective performance is dependent on several factors, which include the specific environment in which the team is working in, the level of interdependence among members, the nature of the task, and where the team is in their decision-making process (Forbes et al., 2006; Mohammed and Dumville, 2001). Another concern of working in teams is the effect that adding additional team members later in the design stage will have on team dynamics. Adding members at a later stage can be viewed in two ways: a rational process guided by economic and instrumental considerations or a social network interpersonal attraction (Forbes et al., 2006). The first way, according to Forbes et al. (2006) is usually a response to fill a certain resource requirement. The second is determined by the trust and similarity of current members (Ruef et al., 2003). Not all additions to teams fall within the two aforementioned approaches. Additions can sometimes happen against the wishes of the current team, by being imposed by top-management or a power stakeholder (Bruton and Ahlstrom, 2003). The timing of the addition of new members to the team is critical. Some researchers have argued that potential members are considered when a problem arises that the team cannot solve (Cyert and March, 2005). Therefore, team members have to identify a problem and perceive that a solution cannot be found within the current team which usually triggers a search for additional team members. Even when this happens, however, the process through which a new member is selected is not clear (Forbes et al., 2006). 4.3.2 Stakeholders and Garrett’s Framework In the original Garrett illustration of design planes (2002, pg. 27), there is no indication of the stakeholder involvement other than a focus on the user. As discussed earlier, however, there are several more stakeholders in website design and, therefore, a generic depiction of where all the observed stakeholders have the same level of interaction in the design process (Figure 4.7). These stakeholders’ involvement or non-involvement is depicted in each stage of design from the commencement of the design project from the strategy plane through to the end of the design project at the Chapter 4 Stakeholders in Web Design 115 Garrett Framework in Commercial Project: A Mini-Case Study A governmental agency in the United Arab Emirates (UAE) required a redesign for its current website. The redesign required creating two identifiable sections: the portal and the online services, something that was previously not available. The governmental agency is one of the major governmental agencies in the country and plays a vital role in the rapidly growing and dynamically changing economy of the United Arab Emirates (UAE). The UAE is a relatively small country with a total area of 83,600 square kilometres. The country has seen a population boom: from 557,887 in 1975 to 4,229,000 in 2006 to an estimated 8,132,491 in 2010. The GDP rose from 321.8 billion in 2003 to 599.2 billion in 2006 (UAE Dirhams). The UAE has one of the highest GDP per capita in the world. Problem The redesign effort had little to go on. Several previous redesigns were less than optimal in offering different stakeholders the information and services they required. Additionally, there was a extensive amount of information targeted towards different types of users of the website that was grouped under a single group: users. Finally, several individuals were involved in the redesign all coming from different backgrounds and required to communicate efficiently. The project had already started and one of the two sections, the online services, was nearly completed. Solution Garrett’s framework was chosen because it is beneficial to all the stakeholders involved in the design of the website as it allows discussion and input from different stakeholders without using overly technical terms in either HCI or design. It defined five planes of design: Strategy, Scope, Structure, Skeleton, and Surface which was used to manage the design process of the new website. Garrett’s framework offered a non-technical means of communicating requirements of the design from a user-centred perspective allowing different stakeholders to involved in the design process. This was done through levels of abstraction: moving from the more abstract to the more concrete as the design moves along. A list of requirements and generic content that is commonly available in other similar Web sites was constructed and used as part of the scope definition.The content was then prioritised. After defining the content, a structure was provided that showed how the content will fit together in the overall structure. A simple hierarchical structure was used. Several alternative skeletons of the homepage were created and the best suited alternative was chosen. A graphic designer was then brought in from the development company to design the majority of the surface plane in accordance to recommendations of the usability company. Results The design document helped different stakeholders agree on design issues in different planes especially when different alternatives were available. Different team members were engaging more throughout the design process with ideas, suggestions, and comments. Being able to lay down a design path and follow it through helped keep the project on time and within budget and, together with the help of an excellent project management company, the finished product was usable. Garrett Framework in Commercial Project: A Mini-Case Study 116 Chapter 4 Stakeholders in Web Design surface plane. These stakeholders, however, have a different amount of involvement in each of the planes and have differences between them according to the stakeholders’ typology. For example, one of the stakeholder types (i.e. owners), according to the stakeholders’ typology, will have a larger impact of the design of the website than other types of stakeholders (i.e. users), with other types of stakeholders trying to ’sell’ their ideas to owners. The notion of the changing involvement in each plane will be discussed in more detail later in the paper. The stakeholders’ depictions in the figures of this section, however, does not take into consideration the varying amount of involvement a stakeholder; rather, the depictions only show involvement or non-involvement of each of the stakeholders in each of the planes. Figure 4.7: Stakeholders in Nirvana In what will be called a “nirvana” scenario (Figure 4.7), a supreme but usually unattainable state, it is assumed that owners are user-centred design (UCD) champions (Gulliksen et al., 2006, 2003) and, therefore, provide the power aspect to allow users and designers to be definitive stakeholders. User-centred design champions can also be considered the person(s) on the design team dedicated to implementing user requirements (e.g. usability consultant or usability designer). The earlier definition is what is considered here as usability champion. This will result in each group of stakeholders having the same amount of involvement in each of the design processes. In the practice of Web design, there is a different interaction. As shown in Figure 4.8, owners are solely responsible for the strategy and scope stages, something that is evident from poorly defined requirements and poor communications between stakeholders (Charette, 2005). If users’ input is solicited, it is usually done in the Chapter 4 Stakeholders in Web Design 117 Figure 4.8: Stakeholders in Practice surface stage, generally too late for any major changes to be made (Lehtonen et al., 2010). The user input is sometimes not even solicited through the design phase but rather closer towards the end of the development phase, near to the launch of the website. This has less impact on the actual design or development of the website because of the amount of time and effort required for any changes. Figure 4.9: Stakeholders in Proposed Scenario An alternative way, which is proposed here, of incorporating stakeholders into Web design is shown in Figure 4.9. Users are involved in the first two stages, strategy and scope, with the owners of the website. This captures the requirements from the two main stakeholders; people building the website and people who are going to use it. The strategy sets the stage for the later planes and, therefore, if user interaction and input is required from the beginning, there will be a fair chance of these requirements being implemented throughout the different planes and until the end of the process. 118 Chapter 4 Stakeholders in Web Design Afterwards, designers are involved to interact with the owners and users of the website and facilitate the actual design of the website by bringing their experience and knowledge to the structure, skeleton, and surface of the website. To summarise, the previous figures relate stakeholders to Garrett’s (2002) framework only in the notion of if they (the stakeholders) are involved or not involved in any of the five planes. The next section discusses Garrett’s framework from the time*effort paradigm he created. This is done through a scenario-based analysis of the paradigm and how different scenarios might impact on the overall design project. Also, presented in this section is the description of the time*effort paradigm and how it changes when interacting with stakeholders. This provides a more in-depth analysis of stakeholders and Garrett’s framework in lieu of the team- and information-sharing concepts. 4.3.3 Garrett’s Framework Time*Effort Paradigm In this section, the original Garrett’s time*effort figure will be discussed, with a focus on the overlapping areas between the planes, followed by a discussion on how this original figure could change according to project constraints. Later, discussion on how the original Garrett figure could change depending on team creation, addition of members at different planes, and information sharing is undertaken. 4.3.3.1 Garrett’s Original Time*Effort Figure Figure 4.10: Garrett’s Original Time*Effort Figure In the Garrett’s (2002) original illustration (p. 27) of overlapping planes (Figure 4.10), he argued that the work on any particular plane cannot be completed until the work on the previous plane has been completed. This allows for feedback between the planes Chapter 4 Stakeholders in Web Design 119 in that overlapping area. Another way of articulating the overlapping areas is that each plane informs, and is informed by, the planes it overlaps with. For instance, the strategy plane informs the scope plane and directs the creation of the scope of the website which constitutes a strategy → scope flow. However, new ideas and requirements might arise whilst working through the scope plane because of the more concrete nature of the scope plane in comparison to the strategy scope, and therefore changes to the strategy plane need to happen to incorporate the new ideas. This is actually a backward flow constituting a strategy ← scope. The combination of these two types of flow of information is what is considered to be feedback; the entire loop instead of the backward arrow. This is a process that can, theoretically, exist throughout the design process and between different planes. Additionally, the size of the overlap constitutes the amount of effort and time depreciated on each of the overlapping planes. Sometimes the work is done on one plane towards the end and another towards the start. However, the emphasis differs on each of the overlapping planes. So, for example, it would not be beneficial to spend the majority of time on the strategy plane when work has already started on the skeleton plane; the work on the strategy plane should be nearing completion whilst the work on the skeleton plane should start to escalate. Additionally, accommodating major changes in more than one prior or one later plane is assumed to be more taxing in both time and effort (Folmer and Bosch, 2004). Although there is no indication of the actual amount of work to be done on each plane, the assumption here is that the amount of work, roughly, is equal in each of the planes. This is done to be able to focus on the interaction between stakeholders and their involvement in the different planes without complicating the discussion with the amount of work to be done in each of the planes. 4.3.3.2 The Overlapping Planes When imposing fixed time on a design project, which is an accurate expectation in any design project, the dynamics between the planes change accordingly. Several factors play a role in such a scenario: the amount of relative time, the amount of relative effort, the total time available, the total effort available, the number of team members, the amount of time the team members have worked together, and the number of added team members after the start of the design process; when these additional team members are added all play a role in the dynamics of the design process. The notion of 120 Chapter 4 Stakeholders in Web Design relative time and relative effort means the time and effort available to each particular plane. Each plane cannot take up the total time allocated to the project, because then the distinction between the planes would be of no benefit. Therefore the time limit on each plane should be researched in accordance with the overall time limit imposed. Each plane also has a limited amount of effort at its disposal and any one plane cannot monopolise all the available effort disregarding the effort requirement of other planes that it overlaps with. The total time used will be controlled; the project has to finish within an allocated timeframe. The changes in interaction between the planes are then observed. 4.3.3.3 The Changing Planes Changing the different planes has an effect on the structure of the design project and the overlapping areas between these planes. For the sake of simplicity, the discussion will begin with a generic, if less accurate, analysis of the planes and how changes in relative time and relative effort within each plane affect the general view of these planes and the overlapping areas between them. This will be done by varying the amount of time and effort invested in each of the planes but keeping the overall time constant. Later, in the section called stakeholders’ scenarios, additional attributes are taken into consideration whilst illustrating the relationship between the planes and the overlapping areas. These attributes involve team formation theories and the effect of adding team members at the beginning and then later in the design project. Furthermore, the impact on effort and time of adding additional team members is taken into consideration. All of this is encapsulated by the stakeholder analysis carried out earlier with regards to Garrett’s (2002) framework. The stakeholders involved are put into perspective and the impact of their involvement is considered and illustrated. 4.3.3.3.1 Generic Scenarios There are a total of six variations, or generic scenarios, which are described here. The first four scenarios vary one of the two attributes whilst keeping the other constant. These four are increasing relative effort, increasing relative time, decreasing relative effort, and decreasing relative time. The next two scenarios keep the relationship between the two attributes constant in the sense that increasing Chapter 4 Stakeholders in Web Design 121 relative effort decreases relative time and decreasing relative effort increases relative time. This keeps the relationship between the two attributes equal at all times. 4.3.3.3.1.1 Increasing Relative Effort Figure 4.11: Increasing Relative Effort The planes in Figure 4.11 have equal relevant time but increasing relative effort. The scenario starts with a generic strategy plane and then the effort is gradually increased in the next planes. This, theoretically, might be due to the type of work involved, designing several Web pages, graphic design, and other time-consuming tasks in later stages. Since the relative time allocated to that plane is constrained, the amount of effort has to increase in order to be able to complete the work on time. This causes the overlapping areas to gradually increase between each of the planes but without any changes to the dynamics of it; the planes still have the same overlapping areas between them as in the generic diagram, although these areas are larger. 4.3.3.3.1.2 Increasing Relative Time Another scenario could be that relative effort on each of the planes cannot be increased and therefore the amount of relative time has to be increased. In Figure 4.12 the time allocated to each of the planes is increased gradually but the relative effort is held constant. This causes the planes to start earlier and therefore the overlapping areas are expanded. Additionally, the dynamics of the overlapping areas change, making more planes overlap; for instance, strategy and surface are now overlapping, something that did not exist with either the generic diagram nor whilst only increasing effort. 122 Chapter 4 Stakeholders in Web Design Figure 4.12: Increasing Relative Effort 4.3.3.3.1.3 Decreasing Relative Effort Figure 4.13: Decreasing Relative Effort The relative effort could be decreased while keeping the relative time constant (Figure 4.13). This is a plausible alternative if the overall effort is misused, leaving less for the later planes; or where that effort is allocated to more important projects later in the design. The dynamics of the overlapping areas do not change here but the amount of it does change due to the decrease in relative effort. 4.3.3.3.1.4 Decreasing Relative Time Relative effort could be constant but relative time decreased (Figure 4.14). This might be due to weak project management for the five planes, which causes the planes to keep taking less and less time. As shown in the figure, the planes decrease the amount by which they overlap greatly, leaving the surface plane overlapping with just the skeleton plane. Chapter 4 Stakeholders in Web Design 123 Figure 4.14: Decreasing Relative Effort 4.3.3.3.1.5 Increasing Relative Effort while Decreasing Relative Time Figure 4.15: Increasing Relative Effort while Decreasing Relative Time This variation is similar to a combination of increasing relative effort and decreasing relative time (Figure 4.15). This might be due to several situations. First, managers might be running out of time while the project progresses and therefore try to rectify that by increasing the amount of effort put into the planes. As for the overlapping areas, these change considerably creating a mostly secluded surface plane. 4.3.3.3.1.6 Decreasing Relative Effort while Increasing Relative Time As with the previous scenario, the variation here is a combination of the previous stand-alone scenarios: decreasing relative effort and increasing relative time (Figure 4.16). The resulting scenario though is more notable. The planes mostly merge together creating virtually one big overlapping area with planes starting much earlier than expected. This is caused by the decreasing investment in effort of each plane. 124 Chapter 4 Stakeholders in Web Design Figure 4.16: Decreasing Relative Effort while Increasing Relative Time 4.3.3.3.2 Stakeholders’ Scenarios The generic Garrett (2002) illustration (Figure 4.10) showed how planes overlap. It, however, did not take into consideration other factors such as the issues of creating team members, and adding team members later in the design project. In this section, we attempt to simulate the effect of these issues on each plane and on the overall design project. In the stakeholders’ scenarios, the stakeholders’ analysis and the planes’ analysis are merged to illustrate the impact of varying stakeholders in addition to varying the time*effort attributes. The time*effort attributes are varied according to team theories and not generically varied. Three scenarios for each of the figures described in the stakeholders’ section will be presented: one that varies the relative effort and relative time, one that varies the relative effort but keeps the relative time fixed, and finally one that fixes the relative effort but varies the relative time, resulting in a total of nine scenarios. All of the illustrations are super-imposed on Garrett’s generic time*effort illustration to show the effect of these variations on the outcome. 4.3.3.3.2.1 “Nirvana” There are three nirvana variations that will be considered here: varying effort and time, varying effort and keeping time constant, and varying time and keeping effort constant. All of the three stakeholder types are involved from the beginning to the end in the nirvana alternative. Chapter 4 Stakeholders in Web Design 125 Figure 4.17: Varying Relative Effort and Relative Time Varying Relative Effort and Relative Time In the first variation (Figure 4.17) and according to (Kilduff et al., 2000), teams commence their life-cycle as a number of diversified individuals which in effect increases the relative effort and relative time that goes into the strategy plane. This effect soon diminishes and the team starts to become more effective (Duncan et al., 1996) which decreases the amount of effort and time going into each of the later planes. The surface plane, as a result, consumes significantly less relative effort and relative time than the strategy plane and the generic surface plane. Also, because of the cumulative effective use of effort and time and the set total time available, later planes have more time than the generic plane and therefore can start later than the generic planes whilst still finishing at the same time overall. If there was a requirement to finish the project as soon as possible however, then the planes could have started at the same time as the generic planes and ending sooner. There is no impact from adding additional team members in the shape of new stakeholder types; the same three stakeholder types exist from the beginning to the end of the design. Varying Only Relative Effort The second variation (Figure 4.18) only varies relative effort, keeping relative time constant. This is applicable when specific relative time is allocated to each of the planes but the relative effort is not. As with the previous variation, the effort put into 126 Chapter 4 Stakeholders in Web Design Figure 4.18: Varying Only Relative Effort the strategy plane is higher than it is in the generic because of team creation impact on effort. This effect starts to diminish in later stages. The relative time taken by each of the planes is the same. The dynamics of the overlapping areas do not change here either, and the same areas exist throughout the design. The overlapping areas, as with the planes themselves, decrease in size as the project moves forward. Varying Only Relative Time Figure 4.19: Varying Only Relative Time The third variation (Figure 4.19) varies only time but keeps the effort constant. There is a significant shift in the planes towards the end because of the decreasing use of Chapter 4 Stakeholders in Web Design 127 time. It is clear that the first two planes:, strategy and scope, take a while longer than the generic planes but the structure plane is similar to that of the generic plane; the later planes, skeleton and surface, take less than the generic planes. There is a clear indication that the planes, overall, can take less time without affecting the dynamics of the overlapping areas. 4.3.3.3.2.2 Current Practice The current practice has a different type of involvement of stakeholders at different planes of the design project. Their involvement is analysed here and the addition of the stakeholders in later stages is discussed. The owner is involved in each of the planes; however, there are no other stakeholders involved until the structure plane, where the designer is involved to start designing the actual website. The designer is then involved until the end of the design. The user, however, if involved, is usually included towards the end of the design process; the surface plane. Varying Relative Effort and Relative Time Figure 4.20: Varying Relative Effort and Relative Time The first variation (Figure 4.20) varies the relative effort and relative time and shows the effect of the addition of specific stakeholders at the different design planes. The planes are greatly influenced by the varying amount of stakeholders involved and the timing of the involvement. The first two planes, strategy and scope, take the same 128 Chapter 4 Stakeholders in Web Design relative effort and relative time because of the involvement of only one stakeholder, the owner. However, the scope plane starts a bit earlier than the generic scope plane to accommodate the increased requirement for effort and time later in the design. When the designer is involved in the structure plane, there is an impact on relative effort and relative time on that plane; this is similar to what is seen in the nirvana effect but on a smaller scale because of the addition of only one other stakeholder rather than two. The relative effort and relative time invested in the later plane skeleton is decreased because of the improved effectiveness. This, however, does not last because of the addition of the third stakeholder, the user. The surface plane is significantly affected by the addition because of the timing of the addition of this new stakeholder. The addition of the user at the end of the design project with impending deadlines requires the investment of greater relative effort and relative time. The input of the user is probably ignored as well (Larson et al., 1994), so more relative effort and relative time is invested in the plane without any real difference. It is also observed that, overall, in general none of the planes in this variation uses less relative effort and relative effort than the generic. Varying Only Relative Effort Figure 4.21: Varying Only Relative Effort The lack of benefit from the practice alternative is clearly visible from varying only the relative effort and keeping the relative time constant (Figure 4.21). Here, is it clear that none of the planes has a decreased effort usage, compared to the generic planes. If Chapter 4 Stakeholders in Web Design 129 anything, the planes are either similar or use more relative effort than that of their generic counterparts. Varying Only Relative Time Figure 4.22: Varying Only Relative Time When varying only relative time and keeping relative effort constant (Figure 4.22), the same effect is visible on the planes as with the previous variation; planes start earlier than their generic counterparts. The overlapping areas are similar to when relative effort and relative time is varied. The strategy plane, for instance, overlaps with the surface plane creating greater overlapping areas overall. 4.3.3.3.2.3 Proposed Practice In the proposed practice, an alternative is proposed to the involvement of different stakeholders. This is based on team concepts as well as considering when different stakeholders should be involved depending on the value of their involvement. The stakeholders’ involvement in this scenario is similar to that of the nirvana except in the first two planes, strategy and scope, where the designer is missing. This is based on the requirement of what the stakeholders’ input is in each of the planes; and whether they have valuable input at different planes. 130 Chapter 4 Stakeholders in Web Design Varying Relative Effort and Relative Time Figure 4.23: Varying Relative Effort and Relative Time When varying relative effort and relative time (Figure 4.23), and considering the involvement of stakeholders in the design, the strategy plane realises an increase in both relative effort and relative time. In the scope plane, however, the work becomes more effective and therefore decreases the amount of relative effort and relative time used to that as compared to that of the generic scope plane. The structure plane sees the introduction of the designer as an additional stakeholder who increases the relative effort and time used. This also diminishes in the next plane: skeleton. The surface plane sees a gain in relative effort and relative time compared to the previous planes and the generic surface plane. As with the nirvana scenario, there is an overall impact on effort and time with the design project being able to complete earlier if the planes started with their generic counterparts. Varying Only Relative Effort When varying only relative effort (Figure 4.24), the planes in the proposed scenario look close to the generic planes which two planes: strategy and structure, slightly having an increased relative effort and the surface plane having a slightly decreased relative effort. The other two planes: scope and skeleton are identical to their generic counterparts. From this view, there seems to be no advantage in the use of the proposed as opposed to current practice except that the user is involved from the Chapter 4 Stakeholders in Web Design 131 Figure 4.24: Varying Only Relative Effort beginning of the design process without greatly affecting the various planes in a negative manner.When varying only relative effort, the planes in the proposed scenario look similar to the generic planes. For instance, the two planes, strategy and structure, slightly have an increased relative effort and the surface plane has a slightly decreased relative effort. The other two planes, scope and skeleton, are identical to their generic counterparts. From this view, there seems to be no advantage in the use of the proposed scenario as opposed to current practice; what is gained in some planes are lost in others. There is, however, one crucial difference: the user is involved from the beginning of the design process without greatly affecting the various planes in a negative manner. Varying Only Relative Time When varying only relative time (Figure 4.25), this variation becomes similar to that of the nirvana variation. The planes keep starting later than their generic counterparts because of the increased effectiveness. The change is not as significant as that of the nirvana variation but is better than the current practice. 132 Chapter 4 Stakeholders in Web Design Figure 4.25: Varying Only Relative Time 4.3.4 Changes in the Stakeholders’ Involvement The stakeholders’ involvement in each of the depictions in the previous section “Stakeholders and Garrett’s Framework” was shown as equal for the sake of simplicity and lack of evidence to support any different depictions. However, according to Mitchell et al. (1997) typology of stakeholders, and the plane where the design process is at in a particular point of time, it is expected that different stakeholders will have a different amount of involvement. In this section, three different depictions based on Figures 4.7, 4.8, and 4.9 will be described in an effort to theorise about the relative involvement of stakeholders. Additionally, it is observed that the value each stakeholder adds to the design process in the different planes will differ according to the plane the design is currently in. For example, the value given to owners in middle planes such as structure and skeleton might be less than that in the early planes such as strategy and scope. The reason for this is that owners are more likely to have a higher impact on the strategy and scope because of their position as strategy creators for the organisation, and by extension, the website. This is also true in the surface stage where the owners are usually particularly keen to have the end product portray the image of the organisation as they see it. Another example is the value of designers in stages starting from the structure where they are valued for the knowledge they bring regarding design standards and processes that not only can expedite the design progression but also facilitate the maintenance of the website after it has been developed. Additionally, following through from previous sections on this paper, the Chapter 4 Stakeholders in Web Design 133 concentration will be solely on the three stakeholders discussed at the beginning of the paper: owners, users, and designers. If, however, more stakeholders are identified, they could be described according to Mitchell et al.’s 1997 topology and their involvement inserted in the depictions depending on the outcome of such analysis. To be able to depict the different involvement of stakeholders in different planes, however, requires defining each of the stakeholders: owners, users, and designers in according to Mitchell et al. (1997) topology of stakeholders. The definitions used are based on the three scenarios: nirvana, practice, and proposed. Therefore, instead of only showing the existence or non-existence of each scenario, the relative amount of involvement will be shown in the next sections. 4.3.4.1 Nirvana Figure 4.26: Varying Nirvana Stakeholders’ Involvement In nirvana (Figure 4.26), all stakeholders are treated as definitive stakeholders in Mitchell’s stakeholder topology. Therefore, they have, in essence, an equal amount of power, legitimacy, and urgency. Overall, all stakeholders have an equal amount of participation in the project. However, the relative participation differs according to the value they can add by being involved in each of the planes. The owners, for instance, are involved more heavily in the strategy and scope planes but less at later planes until the surface plane. This can be due to, as mentioned earlier, the owners excelling in setting strategy, and to ensure it aligns to the organisation’s strategy. The involvement increases at the surface plane because of the requirement of the website to reflect the organisation’s image. 134 4.3.4.2 Chapter 4 Stakeholders in Web Design Practice Figure 4.27: Varying Practice Stakeholders’ Involvement In practice (Figure 4.27), owners are categorised as dangerous stakeholders, users as discretionary stakeholders, and designers as dependent stakeholders. Owners dominate the project overall, being the only stakeholder in the strategy and scope, and having the majority of involvement in the surface plane. This is because of their need to finish the project as soon as possible. The ideology underpinning this is that going through the project quickly will save time and budget without focusing on what is important here; namely the usability of the website. This is shown in a website’s call-for-tender; it is released without any usability requirements, and the designer is left to create the website before the owner’s feedback so as to enforce the website image in the surface plane. Designers are dependent on owners for design approval, time limits, and budgeting. Finally, if users are involved, it happens at the end of the design process; however, the involvement of users is more commonly delayed until near the end of the development (Carroll, 2003). 4.3.4.3 Proposed In the proposed scenario (Figure 4.28), the main focus of the approach is the involvement of users throughout the design of the website, much like in nirvana. The difference, however, is that the extent to which the users participate is different to the nirvana example. Owners and users are considered definitive stakeholders. Designers, on the hand, are considered dependent stakeholders. In strategy and scope, and because the stakeholders here are related to the website rather than the organisation, Chapter 4 Stakeholders in Web Design 135 Figure 4.28: Varying Proposed Stakeholders’ Involvement owners and users have equal involvement. This is because the website is built to fulfil the requirements of both owners and users. The designers are then brought in to take charge in creating the website. This shows a decrease in involvement of owners until the surface plane. The value of the designers emerges later in the design phase where the knowledge they bring in is not available with either of the other stakeholders; the actual best-practices in design and related knowledge. Including designers in earlier planes might be a disadvantage because of their orientation towards current solutions and hardware/software limitations. When they are excluded from these planes, discussions can focus on requirements of owners and users without worrying about the implementation of these requirements prematurely. 4.4 Stakeholders in Web Design: Research Agenda The previous sections showed the effect of adding more stakeholders earlier into the design process; however, there are some questions that need to be answered in order to realise any related benefits. This section discusses the researchable propositions that relate to the unanswered questions and revolves around three main interdependent themes: identification of stakeholders, researching the effect of adding stakeholders to different planes of design, and the amount of involvement of stakeholders. As mentioned in the introduction, many stakeholders play a role in the design of the Web design. However, we focused on the three more prominent stakeholders: owners, users, and designers. These stakeholders have to be identified according to Mitchell 136 Chapter 4 Stakeholders in Web Design et al. (1997) stakeholders’ typology so as to understand these stakeholders according to three attributes: power, legitimacy, and urgency. Being able to identify the stakeholders through these attributes will enable more in-depth analysis of the interaction between the stakeholders, and thus facilitate attempts to enhance the design project within the various planes and as a whole. Through various sections of the paper, scenarios have described how stakeholders could be classified. Future research, however, has to be directed towards establishing how stakeholders should be classified. The effect of adding stakeholders is dependent on identifying stakeholders. The different scenarios in this paper were based on team creation and team member addition research; however, these members, in such research, are considered to be equal. In Web design, on the other hand, there needs to be an evaluation of the validity of such research in the context of stakeholders’ typology. The scenarios here provide a platform upon which to identify different ways of classifying stakeholders, and a theory of how the design project will be affected. Early scenarios in this paper assumed that stakeholders are either involved or not involved in the design process without discussing the different amount of involvement they might have within each plane and throughout the design project. Later, in the section entitled “Changes in the Stakeholders’ Involvement”, however, the author presented scenarios where the involvement of different stakeholders, in various planes, might not be equal even in nirvana. This issue could be investigated further through observation of Web design projects. 4.5 Stakeholders in Web Design: Practical Agenda There are some practical implications assuming that the aforementioned discussed themes - identification of stakeholders, researching the effect of adding stakeholders to different planes of design, and the amount of involvement of stakeholders - can be validated. In terms of identifying stakeholders of a design, the entity responsible for the project management in terms of payments and schedule, usually the owners, and by extension managers, should be actively trying to identify stakeholders early on in the design process; preferably even before the beginning of the design process, to allow for the Chapter 4 Stakeholders in Web Design 137 recruitment of representatives in each of the stakeholder groups. If the website is being designed for a specific group of people, then a representative should be present, allowing opportunity for the integration of the different requirements from that group. This could mean a variety of things; for example, in some instances a website will be designed for the general population, whereas in other contexts the website will need to be bespoke and designed for internal use, and, therefore employees could be identified as users. The identification of stakeholders does not necessarily mean that all the stakeholders will be involved from the beginning of the design process, Even though some stakeholders may have been identified as being key to the success of the website design process, they may be thought of as more important in different stages of the design process rather than being important to the process as a whole. This could indicate that, for the sake of managing the budget and time constraints more efficiently, some stakeholders may be deliberately left out of certain stages of the design, or their involvement may be decreased over the whole project without affecting the quality of design. Building on the previous two points; even if a stakeholder is identified as important and is involved in all of the stages it does not necessarily follow that this stakeholder will have an equal amount of involvement. The involvement in each of the stages is relevant to the amount of benefit the stakeholder is bringing to the design which might vary at different stages. This paper has illustrated that owners might have a greater input in the strategy, scope, and surface planes and lesser input in the structure and skeleton planes. The important issue in this regard is to ensure that the stakeholder identified as important has sufficient involvement in the stage to be beneficial to that stage, and by extension, the entire design. 4.6 Conclusion This paper conceptualised the effects of adding additional stakeholders into the design process through Garrett’s (2002) framework of design. The conceptualisation investigated adding new stakeholders at different stages of design and how their involvement would change the interaction between Garrett’s five planes of design. The 138 Chapter 4 Stakeholders in Web Design conceptualisation also examined the varying degrees of involvement of different stakeholders in different stages of design. The analysis showed that involving stakeholders from the beginning of the project does not have to adversely impact on the time and effort of each of the planes, or the design project overall. The proposed scenario also illustrated how improvement could be made over current practice where users are critically involved too late in the planes of the design project to allow for any significant change in the design. The paper concluded by defining areas in research that, through testable propositions, would enable practitioners to classify stakeholders in Web design, validate prior team creation and team member addition research, and define the amount of involvement each of the stakeholders has. Chapter 5 Conclusion This thesis aimed to explore the gap between theory and practice within the context of HCI, specifically relating to effective implementation of HCI methods and frameworks within practice. The thesis is structured as follows; three connected but stand-alone papers are presented preceded by this introduction, and followed by a conclusion relating to recommendations for practice and future research. This introduction defines HCI, discussing its history and evolution and how it has been influenced by different disciplines. The first paper covers the usability of a Web technology, namely personalisation of Web sites, and consists of four studies, evaluating the standard definition of usability and issues relating to it. The first three studies are quantitative concentrating on the indicators of usability: efficiency, effectiveness, and satisfaction. The fourth study is a qualitative study exploring the results of the three previous quantitative studies. Second, we present a case study; a practical evaluation of how usability is implemented in commercial website design projects and compare and contrast the difference between targeting usability issues early in the design or later in the development. The case study showed some of the benefits of involving usability analysis early on in the design project and how the usability recommendations are implemented easier than when the usability consultancy is involved later in the design project. Finally, the third paper conceptualises involving users early on in design projects and how this affects design projects in the context of Garrett’s (2002) framework and how it compares with current and optimal approaches. Analysis showed that involving users early on in the design project does not to adversely impact on the time and effort involved in the design project’s different stages. 139 140 Chapter 5 Conclusion The thesis presented three separate but integrated papers offering three overlapping themes reflecting current thinking about HCI within the context of Web design. This conclusion summarises the findings, limitations, conclusions, and potential development of each paper and discusses the contribution to research and how this fits with the overall research aims. Finally, the implications for practice are presented. 5.1 Personalization and Usability: How Google’s Personalization Scheme Affects Efficiency, Effectiveness and Satisfaction 5.1.1 Summary This paper attempted to research how personalisation affects usability, specifically efficiency, effectiveness, and satisfaction. Results have shown that in normal circumstances the amount of information on a Web page affects the time it takes users to find information on that Web page, which is also supported by previous research (e.g., Lu et al., 2010). When users are given the opportunity to skim through the Web page after they have personalised it, however, that effect diminishes. Furthermore, taking longer to find information on the page did not have an effect on the subjective satisfaction of the users with that Web page. Finally, the measurement of efficiency, effectiveness, and satisfaction is considered to be inappropriate. 5.1.2 Limitations and Future Research From the findings of the three quantitative studies, the author considers that measures of effectiveness may not have been appropriate in this context because of the simplicity of the task given to participants. The pilot, main, and supporting studies asked participants to find information on the page rather than click through the links to find answers, which is a representative scenario for personalisation. Simplicity of the experiment is believed to be the reason why participants achieved the task easily. This, however, is due to the fact that in the experiments, the effectiveness measure was described as if the users were able to complete the task successfully. A better measure might have been breaking down the effectiveness measure into sub-categories of 25%, Chapter 5 Conclusion 141 50%, 75%, and 100% effectiveness according to the time taken to find answers. This would have allowed the visual search time to influence the effectiveness measure as well as the efficiency measure. Future research might also concentrate on creating a more complex experiment where effectiveness, as well as efficiency and satisfaction, could be measured more appropriately. Such research should also consider a more complex website design and test whether finding information deeper within the structure of a website rather than on the same Web page had any effect on the dependent measures. Future studies could also focus on Web sites that incorporate personalisation services other than portals and search engines. More precisely, sites that require time critical decisions to be made, for example, taking longer to find information in a brokering interface, might have a negative effect on performance of the broker and therefore affect satisfaction differently. There are instances on the Web where time is a critical factor; for example when checking in online for a flight. In such contexts some people would want to check-in as soon as the online check-in opens and go through it as fast as possible to ensure getting the best possible seats. 5.2 Adopting Usability Methods and Frameworks in Commercial Web Projects. A Case Study Exploring the Benefits and Pitfalls 5.2.1 Summary The current case study utilised Garrett’s (2002) framework whilst developing a large-scale website. It focused on the difficulties experienced in implementing usability methods into a ’real-world’ situation that had limited resources and tight deadlines. It also discussed the barriers of entry regarding usability analysis. Furthermore, this case study examined the usability analysis as part of the whole development of a portal and online services for a government agency as well as examining Garrett’s framework, which identified many of its strengths and weaknesses within a commercial Web design project. Finally, the case study allowed a greater insight into the workings of the website design and its development and provided a contextual snapshot of alternatives to better integrate usability methods and frameworks into commercial projects. 142 5.2.2 Chapter 5 Conclusion Limitations and Future Research It is clear that often, in the real-world, usability is an after-thought; therefore it is necessary for researchers to develop methods of analysis and evaluation that are both rigorous and valid as well as being easily utilised and implemented. This is a complicated task and requires concerted effort to immerse oneself into a project that is commercial in nature so that observations can be made in the actual context of use rather than a laboratory setting. Data collection within the project should be done continuously and not at the end of the project. People move from one project to another and tend to forget and have less time for past projects. Discussing usability at the beginning of the project might of also helped set a guideline on the level usability knowledge and acceptance that other people on the team had prior to the start of the project. The author is intending to extend Garrett’s (2002) framework to make it more adaptable to real-world situations. The problem at hand is twofold. First, Garrett’s framework should be extended to include usability methods if it is to be of any real benefit. This is due to the fact that usability methods are not integrated within Garrett’s framework (Hodgetts et al., 2005). There is a plethora of methods available and categorising these methods within Garrett’s framework will provide a way to successfully select the most appropriate methods for each of the planes. A first step towards achieving this goal was undertaken through the selection of methods used in this case study as shown in Table 3.9. However, future research needs to establish that these particular methods are valid, provide the appropriate fit, and match the stage of design to which the method is being applied. To date, researchers have adopted frameworks and categorised methods depending on the stage of design/development recommending a specific method for several different stages of design without critically evaluating how such methods are enhancing usability. Therefore, more in-depth analysis is required to validate the feasibility of use of each of the methods detailed in Table 3.9 in the different stages. Furthermore, future research should consider an evaluation of these methods within Garrett’s framework to ensure the feasibility of these methods, something which is currently lacking in the literature. Next, Garrett’s framework should be evaluated against complementary frameworks currently available that manage the development aspect of Web design/development to test whether they Chapter 5 Conclusion 143 could be coupled together to create a holistic approach to design/development. These development frameworks, which are commercially available, differ according to the technology used to build the website which includes HTML5, Ajax, and PHP. These two strategies, if integrated, would allow the development of more user-friendly Web sites without negatively affecting budget and/or time constraints of the project because the integration of usability methods could be adopted from the beginning of the project. This would be an improvement to the current status of usability being an afterthought, as all requirements would be seamlessly integrated and become part of the design/development efforts. 5.3 Stakeholders in Web Design: A Theoretical Framework 5.3.1 Summary This paper conceptualised t the effects of adding additional stakeholders into the design process through Garrett’s (2002) framework of design. The conceptualisation investigated adding new stakeholders at different stages of design and how their involvement would change the interaction between Garrett’s five planes of design. The conceptualisation also examined the varying degrees of involvement of different stakeholders in different stages of design. The analysis showed that involving stakeholders from the beginning of the project does not have to adversely impact on the time and effort spent in each of the planes, or on the design project overall. The proposed scenario also illustrated how improvement could be made to current practice where users are critically involved too late in the design project to allow for any significant change in the design. The paper concluded by defining areas in research that, through testable propositions: to be able to classify stakeholders in Web design, validate team creation and team member addition theory, and define the amount of involvement each of the stakeholders has. 144 5.3.2 Chapter 5 Conclusion Future Research The previous sections showed the effect of adding more stakeholders earlier into the design process; however, there are some questions that need to be answered in order to realise the benefits. This section discusses the researchable propositions that relate to the unanswered questions, and revolves around three main interdependent themes: identification of stakeholders, researching the effect of adding stakeholders to different planes of design, and the extent of involvement of stakeholders. As mentioned in the introduction, many stakeholders play a role in the design of the Web design. We, however, focused on the three more prominent stakeholders: owners, users, and designers. These stakeholders have to be identified according to Mitchell et al.’s (1997) stakeholders’ typology so as to understand these stakeholders based on their three attributes: power, legitimacy, and urgency. Being able to identify the stakeholders through these attributes will enable for more in-depth analysis of the interaction between the stakeholders, and thus lead to attempts to enhance the design project within the various planes and as a whole. Through various sections of the paper, scenarios have described of how stakeholders could be classified. Future research, however, has to be directed towards establishing how stakeholders should be classified. The effect of adding stakeholders is dependent on identifying stakeholders. The different scenarios in this paper were based on team creation and team member addition research; however, these members, in such research, are considered to be equal. In Web design, on the other hand, there needs to be an evaluation of the validity of such research in the context of stakeholders’ typology. The scenarios here provide a platform upon which to identify different ways of classifying stakeholders and a theory of how the design project will be affected. Early scenarios in this paper assumed that stakeholders are either involved or not involved in the design process without discussing the different amount of involvement they might have within each plane and throughout the design project. Later, in the section entitled “Changes in the Stakeholders’ Involvement”, however, the author presented scenarios where the involvement of different stakeholders, in various planes, might not be equal even in nirvana. This issue could be investigated further through observation of Web design projects. Chapter 5 Conclusion 5.4 145 Contribution to Research The aim in this thesis was to investigate the gap between theory and practice in HCI. To that aim, the first paper tried to assess how a website’s usability can be evaluated. This was done through commonly used indicators of usability; the ISO 9241-11 definition of usability and through a common method to operationalise these indicators. However, Web sites that incorporate personalisation as a technology are not easily measured well with these indicators. Earlier research has measured one aspect of usability (Hornof, 2001; Hornof and Halverson, 2003; Lu et al., 2010), concentrating on the efficiency measure of usability. This, however, illuminates the point regarding the theory-practice split discussed in the introduction chapter. Measuring a small aspect of the system or website might be sufficient for empirical research or theory building. It is, on the other hand, not enough to assess the overall system or website (Dray, 2009; Parush, 2006). This placed the research on a dichotomous path, either rigorously researching a smaller aspect of the website, or opting for relevance and evaluating the whole website. In the personalisation paper, the results show this discrepancy. In the second paper, the case study, research took a different approach in analysing the gap through examining how usability is actually applied in a typical Web design project. An interpretive approach was adopted in a single-case study to analyse the process of website design and development and what role members of the design project played in the project. The case offered a fundamental distinction between undertaking usability consultation at the beginning of the project and near the end of it through the structure of the project which was separated into two sections; the portal and online services. Being involved early allowed more recommendations to be implemented, while being involved late resulted in a minority of recommendations being implemented. This is caused by the pressure of impending deadlines and the perceived additional cost of implementing any additional recommendations. The case study also discussed the initial resistance to any recommendations due to usability work not being a common practice, which is something the author also observed through his practice of usability consultation. This resistive stance soon changed as reported by individual feedback. The third paper tried to extent the findings of the other two papers,, and especially the case study, by conceptualising the addition of different stakeholders in the Web design 146 Chapter 5 Conclusion project. From a user-centred design, it is undeniably vital to focus on users as a key measure of success of a website. Users, however, are not the only stakeholders involved in the design and, therefore, it is important to involve other stakeholders to ensure as much as possible the success of the design project and the website. Failing to involve users in the design process may move the project forward faster; however, as the success of the project is related to the success of the website, both project and website may fail Charette (2005). This presents an intriguing dichotomy as although the website requirement (through the call-for-tenders) does not require usability standards to be implemented (Lehtonen et al., 2010), it might then be considered a success if it stayed within budget and time allocation, in one sense; but if it does not satisfy intended users, a discrepancy is created between developers and owners of the website. Developers were successful in producing the required website within the allocated time and budget and satisfying all the requirements; a pure engineering approach to the project (Wixon, 2003). The users, however, are reluctant to use a badly designed website because the website does not meet their requirements, in which case the owners consider the website to failure in delivering what it should. This strengthens the argument to include the requirements of users much earlier on than is currently the case. Stakeholders in Web design are considered in this paper through the stakeholders’ typology by (Mitchell et al., 1997). The paper considered three of the prominent stakeholders as an approach to stakeholder analysis in Web design: owners, users, and designers. Through Garrett’s (2002) framework, two things are achieved. First, the amount of involvement of stakeholders is discussed evaluating the involvement and non-involvement of different stakeholders through the different planes of design. Second, the effect of stakeholders’ involvement on the different planes and changes in the overlapping areas is considered and discussed. This allowed the author to show the effect of involving different stakeholders in the different planes and on the overall project. 5.5 Implication for Practice It is imperative for HCI practitioners to be involved in the design much earlier than they usually are in order to be able to input user requirements early on in the design project. If they are involved after the call-for-tender is published, then they are being Chapter 5 Conclusion 147 involved, technically, after the strategy and scope planes have been completed which means their proposed recommendations for the design have less of an impact than if they are involved from the strategy plane. Also, the later the usability practitioners are involved in the Web design project, the less likely any recommendations made will be implemented. It is complicated, however, to become involved in a project before the call-for-tenders are sent out. There are cases where getting involved early is applicable. In one of the projects the author was involved in, for example, the usability analysis and recommendations for the design were the basis for the call-for-tenders being sent out by the company. It ensured, minimally, although only to a limited extent, that the requirements of the users were included in the call-for-tenders. If the practitioner is involved after the call-for-tenders is sent out, they should discuss the possibility of changing some of the requirements - mostly adding user requirements to the requirements of the design - for the sake of building a more usable website. Another relevant issue is the identification of the practitioner within the stakeholders’ typology. The author, as discussed in the fourth chapter, considers HCI practitioners the same as users; however, HCI practitioners are given many different names that might dilute this understanding. Such names might be something like ’usability designer’ or ’usability engineer’. From the author’s experience, owners usually want an overall solution to design with the minimum number of people. The distinction is made here is that asking an HCI practitioner to design a website is similar to asking an architect to build a house. Bibliography Accot, J. and Zhai, S. (2003). Refining fitts’ law models for bivariate pointing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 193–200, Ft. Lauderdale, Florida, USA. ACM. Ajzen, I. and Fishbein, M. (1973). Attitudinal and normative variables as predictors of specific behavior. Journal of Personality and Social Psychology, 27(1):41–57. Alkhafaji, A. F. (1989). A stakeholder approach to corporate governance: Managing in a dynamic environment. Quorum Books. Andreasen, M. S., Nielsen, H. V., Schrøder, S. O., and Stage, J. (2007). What happened to remote usability testing?: an empirical study of three methods. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1405–1414, San Jose, California, USA. ACM. Arias, E., Eden, H., Fischer, G., Gorman, A., and Scharff, E. (2000). Transcending the individual human mind—creating shared understanding through collaborative design. ACM Transactions on Computer-Human Interaction (TOCHI), 7(1):84–113. Augustine, S. and Greene, C. (2002). Discovering how students search a library web site: A usability case study. College and Research Libraries, 63(4). Baecker, R. and Buxton, W. (1987). Readings in human-computer interaction: A multidisciplinary approach. Morgan Kaufman Publishers, Inc.,Los Altos, CA. Baecker, R. M. (1995). Readings in human-computer interaction: toward the year 2000. Morgan Kaufmann. Baecker, R. M. (2008). TIMELINES: themes in the early history of HCI—some unanswered questions. Interactions, 15(2):22–27. 149 150 BIBLIOGRAPHY Battleson, B., Booth, A., and Weintrop, J. (2001). Usability testing of an academic library web site: a case study. The Journal of Academic Librarianship, 27(3):188–198. Beith, B. H. (1987). Subjective workload under individual and team performance conditions. In Human Factors and Ergonomics Society Annual Meeting Proceedings, volume 31, page 67–71. Bellas, F. (2004). Standards for second-generation portals. IEEE Internet Computing, page 54–60. Bellotti, V., Dalal, B., Good, N., Flynn, P., Bobrow, D. G., and Ducheneaut, N. (2004). What a to-do: studies of task management towards the design of a personal task list manager. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, page 742. Bellotti, V., Ducheneaut, N., Howard, M., and Smith, I. (2003). Taking email to task: the design and evaluation of a task management centered email tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, page 345–352. Bellotti, V., Fukuzumi, S., Asahi, T., and Suzuki, S. (2009). User-Centered design and Evaluation–The big picture. Human-Computer Interaction. New Trends, page 214–223. Bellotti, V. and Smith, I. (2000). Informing the design of an information management system with iterative fieldwork. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, page 237. Ben-Chaim, Y., Levy, M., Hadar, I., Farchi, E., and Bronshtein, A. (2010). Engaging stakeholders in globally distributed software development processes. MCIS 2010 Proceedings. Benbasat, I. (2010). HCI research: Future challenges and directions. AIS Transactions on Human-Computer Interaction, 2(2):1. Benbasat, I., Goldstein, D. K., and Mead, M. (1987). The case research strategy in studies of information systems. MIS Quarterly, 11(3):369–386. ArticleType: research-article / Full publication date: Sep., 1987 / Copyright ➞ 1987 Management Information Systems Research Center, University of Minnesota. BIBLIOGRAPHY 151 Bevan, N. (1999). Design for usability. In Proc. HCI International, volume 99. Bevan, N. (2003). UsabilityNet methods for user centred design. Human-Computer Interaction: Theory and Practice, 1. Bevan, N. (2009). International standards for usability should be more widely used. Journal of Usability Studies, 4(3). Bhide, A. M., Heung, Y. J., and Kee, C. M. (2007). Research library: A new look of academic digital libraries. In Internet and Web Applications and Services, International Conference on, volume 0, page 57, Los Alamitos, CA, USA. IEEE Computer Society. Blackwell, A. F. (2006). The reification of metaphor as a design tool. ACM Trans. Computer-Human Interaction, 13(4):490–530. Blandford, A., Keith, S., and Fields, B. (2006). Claims analysis ”In the wild:” a case study on digital library development. International Journal of Human-Computer Interaction, 21(2):197. Blandford, A., Stelmaszewska, H., and Bryan-Kinns, N. (2001). Use of multiple digital libraries: A case study. In Proceedings of the 1st ACM/IEEE-CS Joint Conference on Digital Libraries, pages 179–188, Roanoke, Virginia, United States. ACM. Bonoma, T. V. (1985). Case research in marketing: opportunities, problems, and a process. Journal of Marketing Research, 22(2):199–208. Booth, P. (1989). An Introduction To Human-Computer Interaction. Psychology Press, 1 edition. Boutelle, J. (2004). Understanding organizational stakeholders for design success. In Proceedings of the 2004 conference on Designing interactive systems: processes, practices, methods, and techniques, ACM Press New York, NY, USA. Bruton, G. D. and Ahlstrom, D. (2003). An institutional view of china’s venture capital industry:: Explaining the differences between china and the west. Journal of Business Venturing, 18(2):233–259. Buie, E. A., Dray, S. M., Instone, K. E., Jain, J., Lindgaard, G., and Lund, A. M. (2010). Researcher-practitioner interaction. In Proceedings of the 28th of the 152 BIBLIOGRAPHY International Conference Extended Abstracts on Human Factors in Computing Systems, page 4469–4472. Buscher, G., Cutrell, E., and Morris, M. R. (2009). What do you see when you’re surfing?: Using eye tracking to predict salient regions of web pages. In Proceedings of the 27th International Conference on Human Factors in Computing Systems, page 21–30. Card, S. K., Moran, T. P., and Newell, A. (1983). The psychology of human-computer interaction. Lawrence Erlbaum Associates. Carroll, J. M. (1997). Human-Computer interaction: Psychology as a science of design. Annual Review of Psychology, 48. Carroll, J. M., editor (2003). HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science. Morgan Kaufmann, 1 edition. Carroll, J. M. and Rosson, M. B. (1992). Getting around the task-artifact cycle: how to make claims and design by scenario. ACM Transactions on Information Systems (TOIS), 10(2):181–212. Carter, J. (1999). Incorporating standards and guidelines in an approach that balances usability concerns for developers and end users. Interacting with Computers, 12(2):179–206. Chang, H. H. and Chen, S. W. (2008). The impact of customer interface quality, satisfaction and switching costs on e-loyalty: Internet experience as a moderator. Computers in Human Behavior, 24(6):2927–2944. Charette, R. (2005). Why software fails. Spectrum, IEEE, 42(9):42–49. Chattratichart, J. and Lindgaard, G. (2008). A comparative evaluation of heuristic-based usability inspection methods. In CHI’08 extended abstracts on Human factors in computing systems, page 2213–2220. Chellappa, R. K. and Sin, R. G. (2005). Personalization versus privacy: An empirical examination of the online consumer’s dilemma. Information Technology and Management, 6(2):181–202. Chisman, J., Walbridge, S., and Diller, K. (1999). Usability testing: A case study. College & Research Libraries, 60(6):552–69. 153 BIBLIOGRAPHY Clarkson, M. B. (1995). A stakeholder framework for analyzing and evaluating corporate social performance. Academy of Management Review, page 92–117. Cockton, G. (2005). A development framework for value-centred design. In Conference on Human Factors in Computing Systems, pages 1292–1295. ACM New York, NY, USA. Corley, K. G. and Gioia, D. A. (2011). Building theory about theory building: What constitutes a theoretical contribution? The Academy of Management Review (AMR), 36(1):12–32. Corry, M., Frick, T., and Hansen, H. (1997). User-centered design and usability testing of a web site: An illustrative case study. Educational Technology Research and Development, 45(4):65–76. Cyert, R. M. and March, J. G. (2005). A behavioral theory of the firm. Blackwell. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3):319–340. ArticleType: primary article / Full publication date: Sep., 1989 / Copyright ➞ 1989 Management Information Systems Research Center, University of Minnesota. de Souza, F. and Bevan, N. (1990). The use of guidelines in menu interface design: Evaluation of a draft standard. In Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction, pages 435–440. North-Holland Publishing Co. Diaper, D. and Stanton, N. (2003). The Handbook of Task Analysis for Human-Computer Interaction. CRC Press, 1 edition. Dray, S. M. (2009). Engaged scholars, thoughtful practitioners: The interdependence of academics and practitioners in User-Centered design and usability. Journal of Usability Studies, 5(1):1–7. Duncan, P. C., Rouse, W. B., Johnston, J. H., Cannon-Bowers, J. A., Salas, E., and Burns, J. J. (1996). Training teams working in complex systems: A mental model-based approach. Human Technology Interaction in Complex Systems, 8:173–232. 154 BIBLIOGRAPHY Engelbart, D. C. (1962). Augmenting human intellect: A conceptual framework. SRI Summary Report AFOSR-3223, Air Force Office of Scientific Research, Washington, DC. Fang, X. and Holsapple, C. W. (2007). An empirical study of web site navigation structures’ impacts on web site usability. Decision Support Systems, 43(2):476–491. Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6):381. Fitts, P. M. and Peterson, J. R. (1964). Information capacity of discrete motor responses. Journal of Experimental Psychology, 67(2):103. Folmer, E. and Bosch, J. (2004). Architecting for usability: a survey. Journal of Systems and Software, 70(1-2):61–78. Forbes, D. P., Borchert, P. S., Zellmer-Bruhn, M. E., and Sapienza, H. J. (2006). Entrepreneurial team formation: An exploration of new member addition. Entrepreneurship Theory and Practice, 30(2):225–248. Freeman, R. E. (1984). Strategic Management: A Stakeholder Approach. Pitman Publishing. Frøkjær, E., Hertzum, M., and Hornæk, K. (2000). Measuring usability: are effectiveness, efficiency, and satisfaction really correlated. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 345–352, The Hague, The Netherlands. ACM. Furniss, D., Blandford, A., and Curzon, P. (2008). Usability work in professional website design: Insights from practitioners’ perspectives. Maturing Usability, page 144–167. Garrett, J. J. (2002). The Elements of User Experience: User-Centered Design for the Web. New Riders Press, third & 2002. edition. Girgensohn, A., Redmiles, D. F., and Shipman Iii, F. M. (1994). Agent-based support for communication between developers and users in software design. Proceedings of the 9th KnowledgeBased Software Engineering Conference, pages 22–29. Gould, J. D. and Lewis, C. (1985). Designing for usability: key principles and what designers think. Communications of the ACM, 28(3):300–311. BIBLIOGRAPHY 155 Gray, W. D. and Salzman, M. C. (1998). Damaged merchandise? a review of experiments that compare usability evaluation methods. Human-Computer Interaction, 13(3):203–261. Green, T. R. and Petre, M. (1996). Usability analysis of visual programming environments: A ’Cognitive dimensions’ framework. Journal of Visual Languages and Computing, 7(2):131–174. Green, T. R. G. (1989). Cognitive dimensions of notations. In People and computers V: Proceedings of the Fifth Conference of the British Computer Society Human-Computer Interaction Specialist Group, University of Nottingham, 5-8 September 1989, page 443. Grudin, J. (2005). Three faces of human-computer interaction. Annals of the History of Computing, IEEE, 27(4):46–62. Guiard, Y. (1987). Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior, 19:486. Gulliksen, J., Boivie, I., and Göransson, B. (2006). Usability professionals–current practices and future development. Interacting with Computers, 18(4):568–600. Gulliksen, J., Göransson, B., Boivie, I., Blomkvist, S., Persson, J., and Åsa Cajander (2003). Key principles for user-centred systems design. Behaviour & Information Technology, 22(6):397. Halverson, T. and Hornof, A. J. (2004). Local density guides visual search: Sparse groups are first and faster. In Human Factors and Ergonomics Society Annual Meeting Proceedings, volume 48, page 1860–1864. Hao, J., John, M., et al. (2010). Extending the task–artifact framework with organizational learning. Knowledge and Process Management, 17(1):22–35. Harris, A. and Lessick, S. (2007). Libraries get personal: Facebook applications, google gadgets, and MySpace profiles. Library Hi Tech News, 24(8):30–32. Hartson, H. R., Andre, T. S., and Williges, R. C. (2001). Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction, 13(4):373–410. 156 BIBLIOGRAPHY Hausmann, R. G. (2006). Why do elaborative dialogs lead to effective problem solving and deep learning. In 28th Annual Meeting of the Cognitive Science Society, page 1465–1469. Hewett, T., Baecker, R. M., Staurt, C., Carey, T., Gasen, J., Mantei, M., Perlman, G., Strong, G., and Verplank, W. (1992). ACM SIGCHI Curicula for Human Computer Interaction. Association for Computing Machinery. Hick, W. E. (1952). On the rate of gain of information. The Quarterly Journal of Experimental Psychology, 4(1):11–26. Hodgetts, P., Inc, A. L., and Fullerton, C. A. (2005). Experiences integrating sophisticated user experience design practices into agile processes. In Agile Conference, 2005. Proceedings, pages 235–242. Hollingshead, A. B. (1998). Communication, learning, and retrieval in transactive memory systems. Journal of Experimental Social Psychology, 34(5):423–442. Holsapple, C. W., Pakath, R., and Sasidharan, S. (2005). A website interface design framework for the cognitively impaired: A study in the context of alzheimer’s disease. Journal of Electronic Commerce Research, 6(4):291–303. Holzinger, A. (2005). Usability engineering methods for software developers. Communications of the ACM, 48(1):71–74. Hopkins, W. G. (2000). Quantitative research design. Sportscience, 4(1). Hornof, A. J. (2001). Visual search and mouse-pointing in labeled versus unlabeled two-dimensional visual hierarchies. ACM Transactions on Computer-Human Interaction (TOCHI), 8(3):171–197. Hornof, A. J. and Halverson, T. (2003). Cognitive strategies and eye movements for searching hierarchical computer displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, page 249–256. Hyman, R. (1953). Stimulus information as a determinant of reaction time. Journal of Experimental Psychology, 45(3):188–196. Häkkilä, J. and Mäntyjärvi, J. (2006). Developing design guidelines for context-aware mobile applications. In Proceedings of the 3rd International Conference on Mobile Technology, Applications & Systems, page 24. BIBLIOGRAPHY 157 Hübner, R. (2001). A formal version of the guided search (GS2) model. Perception & Psychophysics, 63(6):945–951. ISO9241-11 (1998). Iso 9241-11:1998. http://www.iso.org/iso/catalogue detail.htm?csnumber=16883. Ivory, M. Y. and Hearst, M. A. (2002). Improving web site design. Internet Computing, IEEE, 6(2):56–63. Jacko, J. A. and Stephanidis, C. (2003). Human-computer interaction: theory and practice. Routledge. Jacob, R. J. and Karn, K. S. (2003). Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. Mind, 2(3):4. Jagacinski, R. J. and Monk, D. L. (1985). Fitts’ law in two dimensions with hand and head movements. Journal of Motor Behavior, 17(1):77–95. PMID: 15140699. John, B. E. and Kieras, D. E. (1996a). The GOMS family of user interface analysis techniques: comparison and contrast. ACM Transactions on Computer-Human Interaction (TOCHI), 3(4):320–351. John, B. E. and Kieras, D. E. (1996b). Using GOMS for user interface design and evaluation: which technique? ACM Transactions on Computer-Human Interaction (TOCHI), 3(4):287–319. Jokela, T., Iivari, N., Matero, J., and Karukka, M. (2003). The standard of user-centered design and the standard definition of usability: analyzing ISO 13407 against ISO 9241-11. In Proceedings of the Latin American Conference on Human-Computer Interaction, pages 53–60, Rio de Janeiro, Brazil. ACM. Jordan, B. and Henderson, A. (1995). Interaction analysis: Foundations and practice. The Journal of the learning sciences, 4(1):39–103. Karvonen, H., Saariluoma, P., and Kujala, T. (2010). A preliminary framework for differentiating the paradigms of Human-Technology interaction research. In 2010 Third International Conference on Advances in Computer-Human Interactions, page 7–12. Katzenbach, J. R. and Smith, D. K. (2009). The Discipline of Teams. Harvard Business School Press. 158 BIBLIOGRAPHY Kay, A. and Goldberg, A. (1977). Personal dynamic media. Computer, 10(3):31–41. Kay, A. C. (1969). The reactive engine. PhD thesis, The University of Utah. Keinonen, T. (2009). Design contribution square. Advanced Engineering Informatics, 23(2):142–148. Kelso, J. A., Southard, D. L., and Goodman, D. (1979). On the coordination of two-handed movements. Journal of Experimental Psychology. Human Perception and Performance, 5(2):229–238. PMID: 528935. Kilduff, M., Angelmar, R., and Mehra, A. (2000). Top management-team diversity and firm performance: Examining the role of cognitions. Organization Science, 11(1):21–34. Koren, Y., Bell, R., and Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8):30–37. Kozlowski, S. W. and Ilgen, D. R. (2006). Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest, 7(3):77 –124. Kuan, H. H., Bock, G., and Vathanophas, V. (2005). Comparing the effects of usability on customer conversion and retention at E-Commerce websites. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS’05), page 174a. Kujala, S., Kauppinen, M., Lehtola, L., and Kojo, T. (2005). The role of user involvement in requirements quality and project success. In Proceedings of the 13th IEEE International Conference on Requirements Engineering, page 75–84. Kules, B., Kang, H., Plaisant, C., Rose, A., and Shneiderman, B. (2004). Immediate usability: a case study of public access design for a community photo library. Interacting with Computers, 16(6):1171–1193. Kwon, K., Cho, J., and Park, Y. (2010). How to best characterize the personalization construct for e-services. Expert Systems with Applications, 37(3):2232–2240. Kärkkäinen, L. and Laarni, J. (2002). Designing for small display screens. In Proceedings of the Second Nordic Conference on Human-Computer Interaction, page 227–230. BIBLIOGRAPHY 159 Lang, M. and Fitzgerald, B. (2007). Web-based systems design: a study of contemporary practices and an explanatory framework based on “method-in-action”. Requirements Engineering, 12(4):203–220. Larson, J. R., Foster-Fishman, P. G., and Keys, C. B. (1994). Discussion of shared and unshared information in decision-making groups. Journal of Personality and Social Psychology, 67(3):446. Lee, S., Koh, S., Yen, D., and Tang, H. L. (2002). Perception gaps between IS academics and IS practitioners: an exploratory study. Information & Management, 40(1):51–61. Leffingwell, D. and Widrig, D. (1999). Managing Software Requirements: A Unified Approach. Addison-Wesley Professional. Lehtonen, T., Kumpulainen, J., Liukkonen, T. N., and Jokela, T. (2010). To what extent usability truly matters?: a study on usability requirements in call-for-tenders of software systems issued by public authorities. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, page 719–722. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1):57–78. Li, M., Song, Y., Lu, S., and Zhong, N. (2009a). The layout of web pages: A study on the relation between information forms and locations using Eye-Tracking. Active Media Technology, page 207–216. Li, M., Yin, J., Lu, S., and Zhong, N. (2009b). The effect of information forms and floating advertisements for visual search on web pages: an eye-tracking study. Brain Informatics, page 96–105. Licklider, J. C. R. (1960). Man-computer symbiosis. Human Factors in Electronics, IRE Transactions on, page 4–11. Ling, J. and van Schaik, P. (2002). The effect of text and background colour on visual search of web pages. Displays, 23(5):223–230. 160 BIBLIOGRAPHY Ling, J. and van Schaik, P. (2006). The influence of font type and line length on visual search and information retrieval in web pages. International Journal of Human-Computer Studies, 64(5):395–404. Ling, J. and van Schaik, P. (2007). The influence of line spacing and text alignment on visual search of web pages. Displays, 28(2):60–67. Lu, W., Li, M., Lu, S., Song, Y., Yin, J., and Zhong, N. (2010). Impact of information overload for visual search on web pages: An eye-tracking study. In Complex Medical Engineering (CME), 2010 IEEE/ICME International Conference on, page 260–264. MacKenzie, I. S. and Buxton, W. (1992). Extending fitts’ law to two-dimensional tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 219–226, Monterey, California, United States. ACM. MacLeod, M., Bowden, R., Bevan, N., and Curson, I. (1997). The MUSiC performance measurement method. Behaviour & information technology, 16(4-5):279–293. Maguire, M. (2001). Methods to support human-centred design. International Journal of Human-Computer Studies, 55(4):587–634. Manber, U., Patel, A., and Robison, J. (2000). Experience with personalization of yahoo! Communications of the ACM, 43(8):35–39. Marks, M. A., Sabella, M. J., Burke, C. S., and Zaccaro, S. J. (2002). The impact of cross-training on team effectiveness. Journal of Applied Psychology, 87(1):3–13. Martin, J. (1973). Design of Man-Computer Dialogues. Prentice Hall. Mayhew, D. J. (1999). The Usability Engineering Lifecycle: A Practitioner’s Handbook for User Interface Design. Morgan Kaufmann, 1st edition. McManus, J. (2005). Managing stakeholders in software development projects. Butterworth-Heinemann. Mitchell, R. K., Agle, B. R., and Wood, D. J. (1997). Toward a theory of stakeholder identification and salience: Defining the principle of who and what really counts. Academy of Management Review, 22(4):853–886. BIBLIOGRAPHY 161 Mohammed, S. and Dumville, B. C. (2001). Team mental models in a team knowledge framework: Expanding theory and measurement across disciplinary boundaries. Journal of Organizational Behavior, 22(2):89–106. Murugesan, S., Deshpande, Y., Hansen, S., and Ginige, A. (2001). Web engineering: a new discipline for development of Web-Based systems. In Murugesan, S. and Deshpande, Y., editors, Web Engineering, volume 2016, pages 3–13. Springer Berlin Heidelberg, Berlin, Heidelberg. Myers, B. A. (1995). User interface software tools. ACM Transactions on Computer-Human Interaction (TOCHI), 2(1):64–103. Myers, B. A. (1998). A brief history of human-computer interaction technology. Interactions, 5(2):44–54. Naumann, A., Wechsung, I., and Schleicher, R. (2009). Measurements and concepts of usability and user experience: Differences between industry and academia. Human Centered Design, page 618–626. Nelson, T. H. (1965). Complex information processing: a file structure for the complex, the changing and the indeterminate. In Proceedings of the 1965 20thNational Conference, ACM ’65, page 84–100, New York, NY, USA. ACM. ACM ID: 806036. Newell, A. (1994). Unified theories of cognition. Harvard University Press. Nielsen, J. (1994). Usability inspection methods. In Conference Companion on Human Factors in Computing Systems, pages 413–414, Boston, Massachusetts, United States. ACM. Nielsen, J. (2003). Usability 101: Definition and fundamentals - what, why, how. http://www.useit.com/alertbox/20030825.html. Nielsen, J. and Molich, R. (1990). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing systems: Empowering People, page 249–256. Nielsen, J. and Phillips, V. L. (1993). Estimating the relative usability of two interfaces: Heuristic, formal, and empirical methods compared. In Proceedings of the INTERACT’93 and CHI’93 Conference on Human Factors in Computing Systems, page 221. 162 BIBLIOGRAPHY Norman, D. A. and Draper, S. W. (1986). User Centered System Design: New Perspectives on Human-computer Interaction. CRC Press, 1 edition. Nuseibeh, B. and Easterbrook, S. (2000). Requirements engineering: a roadmap. In Proceedings of the Conference on the Future of Software Engineering, page 35–46. Nørgaard, M. and Hornæk, K. (2006). What do usability evaluators do in practice?: an explorative study of think-aloud testing. In Proceedings of the 6th Conference on Designing Interactive Systems, page 209–218. Ojanpaa, H., Nasanen, R., and Kojo, I. (2002). Eye movements in the visual search of word lists. Vision Research, 42(12):1499–1512. Palanque, P., Basnyat, S., and Navarre, D. (2007). Improving interactive systems usability using formal description techniques: application to healthcare. HCI and Usability for Medicine and Health Care, page 21–40. Parush, A. (2006). Toward a common ground: practice and research in HCI. Interactions, 13(6):61–62. Pearson, R. and van Schaik, P. (2003). The effect of spatial layout of and link colour in web pages on performance in a visual search task and an interactive search task. International Journal of Human-Computer Studies, 59(3):327–353. Pierrakos, D., Paliouras, G., Papatheodorou, C., and Spyropoulos, C. D. (2003). Web usage mining as a tool for personalization: A survey. User Modeling and User-Adapted Interaction, 13(4):311–372. Poltrock, S. and Grudin, J. (1994). Organizational obstacles to interface and development: Two participant – observer studies. ACM Transactions on Computer-Human Interaction, 1(1):52 – 80. Poole, A. and Ball, L. J. (2005). Eye tracking in human-computer interaction and usability research: current status and future prospects. Encyclopedia of human computer interaction, page 211–219. Porac, C. and Coren, S. (1981). Lateral Preferences and Human Behavior. Springer, 1 edition. Preece, J., Rogers, Y., and Sharp, H. (2002). Interaction design: beyond human-computer interaction. John Wiley. BIBLIOGRAPHY 163 Prichard, J. S., Bizo, L. A., and Stratford, R. J. (2010). Evaluating the effects of team-skills training on subjective workload. Learning and Instruction. Proctor, R. W. and Vu, K. P. (2006). The cognitive revolution at age 50: Has the promise of the human Information-Processing approach been fulfilled? International Journal of Human-Computer Interaction, 21(3):253–284. Ragunath, P. K., Velmourougan, S., Davachelvan, P., Kayalvizhi, S., and Ravimohan, R. (2010). Evolving a new model (SDLC model-2010) for software development life cycle (SDLC). IJCSNS, 10(1):112. Rasmussen, J. (1986). Information processing and human-machine interaction: An approach to cognitive engineering. Elsevier Science Inc. New York, NY, USA. Robson, C. (2002). Real World Research: A Resource for Social Scientists and Practitioner-Researchers. Wiley, 2 edition. Rogers, Y. (2005). New theoretical approaches for human-computer interaction. Annual Review of Information Science and Technology, 38(1):87–143. Rozanski, N. and Woods, E. (2005). Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives. Addison Wesley, 1 edition. Ruef, M., Aldrich, H. E., and Carter, N. M. (2003). The structure of founding teams: Homophily, strong ties, and isolation among US entrepreneurs. American Sociological Review, 68(2):195–222. Sackman, H. (1970). Man-computer problem solving: experimental evaluation of time-sharing and batch processing. Auerbach. Sackmann, S., Struker, J., and Accorsi, R. (2006). Personalization in privacy-aware highly dynamic systems. Communications of the ACM, 49(9):32–38. Saffer, D. (2009). Designing for Interaction: Creating Innovative Applications and Devices (2nd Edition). New Riders Press, 2 edition. Sauro, J. and Kindlund, E. (2005). A method to standardize usability metrics into a single score. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, page 401–409. 164 BIBLIOGRAPHY Seow, S. C. (2005). Information theoretic models of HCI: a comparison of the Hick-Hyman law and fitts’ law. Human-Computer Interaction, 20(3):315–352. Shackel, B. (1959). Ergonomics for a computer. Design, 120:36–39. Shackel, B. (2009). Human-computer interaction - whence and whither? Interacting with Computers, 21(5-6):353–366. Shackel, B. and Richardson, S. J. (1991). Human Factors for Informatics Usability. Cambridge University Press. Shahabi and Chen (2003). Web information personalization: Challenges and approaches. http://www.springerlink.com/content/cvj698w7ar3j5t3q. Shneiderman, B. (1983). Direct manipulation: A step beyond programming languages. Computer, 16(8):57–69. Spencer, R. (2000). The streamlined cognitive walkthrough method, working around social constraints encountered in a software development company. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 353–359. ACM New York, NY, USA. Stanton, N. and Baber, C. (1996). A systems approach to human error identification. Safety Science, 22(1-3):215–228. Stanton, N. A., Ashleigh, M. J., Roberts, A. D., and Xu, F. (2006). Levels of abstraction in human supervisory control teams. Journal of Enterprise Information Management, 19(6):679–694. Stewart, T. F. (1976). Displays and the software interface. Applied Ergonomics, 7(3):137–146. PMID: 15677208. Suchman, L. A. (1987). Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge University Press, 2 edition. Sutherland, I. (1963). Sketchpad: A Man-Machine Graphical Communication System. PhD thesis, Massachusetts Institute of Technology. Swenson, K. M. (2007). The design and development of quality elementary library media center websites. PhD thesis, St. Cloud State University. BIBLIOGRAPHY 165 Tam, K. Y. and Ho, S. Y. (2005). Web personalization as a persuasion strategy: An elaboration likelihood model perspective. Information Systems Research, 16(3):271. Treiblmaier, H., Madlberger, M., Knotzer, N., and Pollach, I. (2004). Evaluating personalization and customization from an ethical point of view: An empirical study. In Proceedings of the Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS’04) - Track 7 - Volume 7, page 70181.2. IEEE Computer Society. Tuckman, B. (1965). Forming storming norming performing team-development model. Online at: http://www. businessballs. com/tuckmanformingstormingnormingperforming. htm (accessed 22 March 2009). Tuckman, B. W. and Jensen, M. A. (1977). Stages of small group development revisited. Group and Organizational Studies, 2(4):419–427. Tufte, E. R. (1983). The Visual Display of Quantitative Information. Graphics Press. Tullis, T. S. (1983). The formatting of alphanumeric displays: a review and analysis. Human Factors, 25(6):657. Ucbasaran, D., Lockett, A., Wright, M., and Westhead, P. (2003). Entrepreneurial founder teams: Factors associated with member entry and exit. Entrepreneurship Theory and Practice, 28(2):107–128. Urbach, N., Smolnik, S., and Riempp, G. (2009). A conceptual model for measuring the effectiveness of employee portals. In Proceedings of 15th Americas Conference on Information Systems (AMCIS 2009), San Francisco, volume 6, page 2009. Usability (2011). Usability home. http://www.usability.gov/. Vredenburg, K., Mao, J. Y., Smith, P. W., and Carey, T. (2002). A survey of user-centered design practice. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Changing Our World, Changing Ourselves, page 471–478. Ward, J. A. (2007). RSS feeds: sweating the really simple stuff. Toxicologic Pathology, 35(6):846. 166 BIBLIOGRAPHY Watson, P. G., Duquenoy, P., Brennan, M., Jones, M., and Walkerdine, J. (2009). Towards an ethical interaction design: the issue of including stakeholders in law-enforcement software development. In Proceedings of the 21st Annual Conference of the Australian Computer-Human Interaction Special Interest Group: Design: Open 24/7, OZCHI ’09, page 313–316, New York, NY, USA. ACM. ACM ID: 1738884. Weinberg, G. M. (1972). Psychology of Computer Programming. Van Nost.Reinhold,U.S. Wharton, C., Rieman, J., Lewis, C., and Polson, P. (1994). The cognitive walkthrough method: A practitioner’s guide. Usability Inspection Methods, page 105–140. Whyte, W. F. (1979). On making the most of participant observation. The American Sociologist, page 56–66. Wing, A. M. (1982). Timing and co-ordination of repetitive bimanual movements. The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology, 34(3):339. Wixon, D. (2003). Evaluating usability methods: why the current literature fails the practitioner. Interactions, 10(4):28–34. Wordpress (2011). WordPress blog tool and publishing platform. http://wordpress.org/. Wu, D., Im, I., Tremaine, M., Instone, K., and Turoff, M. (2003). A framework for classifying personalization scheme used on e-Commerce websites. In Hawaii International Conference on System Sciences, volume 7, page 222b, Los Alamitos, CA, USA. IEEE Computer Society. Yen, B., Hu, P., and Wang, M. (2005). Towards effective web site designs: A framework for modeling, design evaluation and enhancement. In Proceedings of the 2005 IEEE International Conference on e-Technology, e-Commerce and e-Service (EEE’05) on e-Technology, e-Commerce and e-Service, pages 716–721. IEEE Computer Society. Yeung, T. A. and Law, R. (2004). Extending the modified heuristic usability evaluation technique to chain and independent hotel websites. International Journal of Hospitality Management, 23(3):307–313. BIBLIOGRAPHY 167 Zahniser, R. A. (1983). Levels of abstraction in the system life cycle. ACM SIGSOFT Software Engineering Notes, 8(1):6–12.