Keywords

1 Background

Australia’s ageing population is associated with a growth in demand for health services and with an increase in residential care costs, creating a public health challenge. It is projected that by 2056, 22% of Australians will be 65 or over [1]. A similar trend is observed globally [2]. As this demographic continues to grow, discussions will need to focus on how elderly people can safely live at home longer, while improving their Quality of Life (QoL) and maintaining their self-independence. This challenge can be met with technologies that not only meet the needs and expectations of elderly people, but also are reliable, usable, and are suitable for everyday use. Over the past decade, there is an increasing number of technological developments and research projects (i.e. the main ICT-Agenda of the European Union: Horizon 2020) dedicated to the healthcare sector. In this regard, we argue that although the developments are promising, most are mostly technology-driven with little focus on the intended users.

2 Introduction

In general, Internet of Things (IoT) is a paradigm which consists of internet technologies where everyday objects are inherently connected allowing them to communicate and interact often bi-directionally with other networked objects [3]. Recently, various IoT solutions are being explored to sense, measure, and control indoor and outdoor smart connected objects [4], which aim to analyse and monitor the wellbeing of elderly people and to support their independent living. Although there is a growing interest for these technologies, older people are unfamiliar or they face some difficulties interacting with the technology [5,6,7]. In addition, older people are faced with perceived challenges and barriers (e.g. privacy and ethical concerns) [8, 9]. This is partly due to the fact that recent developments in these areas are mostly technology-driven [10,11,12], with only a few guidelines and principles on how to design and develop these technologies [13,14,15,16]. However, it is imperative that assistive technologies are relevant, easy to use, usable, appealing, and beneficial to the everyday lives of elderly people. We use a participatory design approach to address the social and ethical concerns of developing assistive technologies and ensure an optimised user experience meeting the needs and expectations of elderly people.

Participatory design plays an important role in influencing the technology adoption rates. This method actively involves different stakeholders – elderly people, family members, closest friends, and healthcare providers – in conceptualising design ideas and evaluating new technologies. With proper design and careful consideration, the quality of user interaction and user experience might improve adoption rates, especially of elderly people [17]. The long-term aim of this research is to present interaction design principles for combining Mixed Reality (MR) and Internet of Things (IoT) technologies as an Assistive Technology (AT). We hope that it can lead to more successful technical and social outcomes. In this research, MR is envisaged as a tool for elderly people to improve interactions with IoT technology, to enhance remote communication with their family, their closest friends, and their healthcare networks, to properly aid and assist them in their daily activities, and to address loneliness and social isolation which are often key components of depression among elderly people.

This paper is organized as follows: Sect. 3 discusses the applied research design. Section 4 outlines interaction design principles for developing ATs for elderly people. Finally, Sect. 5 presents a discussion of the implications of our study and suggests directions for future work.

3 Methodology

In this section, the applied research design is discussed. The first phase is conducted as a participatory design workshop that brought elderly people aged 65 or over, their family, their closest friends, and their healthcare network together for contextual inquiry. The second phase consists of an iterative development cycle that draws feedbacks from the previous phase to inform the development. The final phase is carried out as an evaluation workshop through quantitative and qualitative analysis. We outline a set of interaction design principles for developing Assistive Technologies (AT) for elderly people. In addition, we provide a series of future direction and recommendations for similar works in this research area. Finally, we constructively discuss the benefits and drawbacks of each phase to inform current practices which are still mostly technology driven.

3.1 Participatory Design Workshop

To provide the elderly people with a platform to have their voices heard and generate collaborative discussions with the other stakeholders, a participatory design workshop was conducted. The aim of the participatory design workshop was to explore the opportunities and challenges inherent to the development of an Assistive Technology (AT) to improve the Quality of Life (QoL) and to maintain the self-independence of elderly people. Overall, the participatory design approach is premised on the assumption that different stakeholders, including the elderly, families, closest friends, and healthcare providers, play an important role in the system development [18]. Besides technological developments, focusing on user experience and acceptance is important for a successful adoption. The ideal coordination and communication with the target group enables us to imagine how the AT can be adjusted based on the needs and wishes of the intended users. The system development should be driven completely and without exception by the needs of the target group at stake. However, there should be an optimal balance between users’ desires and technical feasibility.

Participant Profiles.

To ensure high utility and usability, different stakeholders were included throughout the design process. Participants were recruited from a local healthcare facility (refer to Table 1). They were selected for their expression of interest in participating in the study. All of them signed a consent form and agreed to voluntarily participate without compensation in the study. Afterwards, they answered a questionnaire which addressed demographic aspects, such as age, gender, and sex.

Table 1. Participant profiles
Full size table

Elderly People.

Most developed world countries have accepted the chronological age of 65 years as a definition of ‘elderly’ or older person [19]. Although a number is not a good predictor of functional status in biological, psychological and social processes [20], it should be sufficient to identify people who are starting to experience the ageing process. As such, we considered people aged 65 or above as potential participants in this study. We did not exclude those with physical disabilities or cognitive decline.

Family Members.

Elderly people usually live or lived with family members who know their usual habits. Involving family members in the participatory design workshop brings valuable insights and different perspectives in the design of innovative technologies for elderly people [21]. We considered people who live with a family member aged 65 or above as potential participants to this study.

Healthcare Provider.

Using healthcare providers as proxies for elderly in the development of assistive technologies showed promising and positive results [22]. Since they are to a great extent familiar with the needs and demands of elderly people, their inclusion is immensely important as they support and complete the understanding of how and where assistive technologies could be beneficial to the intended users. Thus, their contribution to the development is perceived as more meaningful, both to elderly and healthcare providers. In this regard, we recruited elderly healthcare providers to participate in this study.

Workshop Setting.

The participants were recruited through a local community centre (‘Holdsworth Community’) in Sydney, who had expressed an interest after being introduced to the research project. Holdsworth Community also provided the setting for the workshop. The community centre describes itself as “a community that supports older people who may be frail, ill, lacking mobility, experiencing social isolation or living with dementia” throughout Sydney, Australia. They also aim to “build a community where all people have the support and services that they need in order to build their personal capacity; have a voice and make choices; and make the meaningful connections they need to live an active and happy life in the community.” The Holdsworth Community carries out discussion group sessions which run weekly on Mondays and Tuesdays, where older people are invited to attend to socialise.

Ethics Approval.

Ethics approval to conduct this research was granted by the University of New South Wales Human Research Ethics Committee Executive (HREC) with HC No: HC180805. This ensures that the research meets the requirements of the National Statement of Ethical Conduct in Human Research (2007), updated 2018 [23]. Particular attention was paid to configuring informed consent for elderly people within this study. A different version of consent forms to support the provision of consent by an elderly person’s family member or healthcare provider was also prepared. Participation in any of the activities was voluntary. Mixed Reality (MR) activities were ensured to be safe and easy to use for the study participants. Participant information sheet and consent forms were handed over to the participants prior to any of the activities. These documents informed the participants about the possibility of feeling nauseous after using the system and the risk of falls and injury since they will be moving around while wearing the headset. To mitigate the risks, potential participants, who have previously experienced motion sickness, were asked to confirm if they still want to proceed with the activities. In addition, proper precaution was taken to ensure that the environment was free of loose objects to minimize the risk of falls and injury. As an introduction to the activities, details about what participants could expect by wearing the headset were provided. If a participant expressed doubt or hesitation to take part in any of the activities, they will not be forced to continue. Furthermore, they have the option to withdraw their participation anytime.

Mixed Reality Activities.

As it is expected that the participants will have little to no knowledge about the current technologies, a background introduction was administered at the start of the workshops. This included the process of introducing the Internet of Things (IoT) technologies, showing participants photos and videos of people trying out Mixed Reality (MR) devices, asking them to try out MR devices, and then discussing current approaches on how technologies are being used to improve the Quality of Life (QoL) of elderly people.

During the activities, participants frequently interacted with a Microsoft HoloLens device. At the beginning of each activity, each participant did a calibration setup using an application that is pre-installed on the Microsoft HoloLens. The application calibrates the HoloLens display according to the users’ individual inter-pupillary distance. By going through the calibration setup, all participants were introduced to the basic controls of the HoloLens. Afterwards, each participant finished a tutorial session which made them familiar and comfortable with the device.

User Study – Object Selection.

The first activity is an object selection task (shown in Fig. 1). During this activity, elderly people are surrounded by coloured boxes and asked to select boxes with a colour that was randomly chosen by the system. The goal is to select the correct coloured boxes with the least amount of time possible. It consisted of different session with each session only allowing them to select a box using only one of the following selection techniques:

Fig. 1.
figure 1

User study – object selection

Full size image

  • By pressing a button on a remote control

  • By performing a specific hand gesture

  • By looking at the coloured box for at least 3 s

  • By saying the word, “Select”

User Study – Navigation.

The second activity is a navigation task (refer to Fig. 2). The goal of this activity is to determine the most effective way to provide directions in a mixed reality environment. In each session, audio, visual, and haptic cues were used to provide directions and to help participants locate virtual objects. Arrows were used as a visual cue to show direction and location of the target objects. The goal is to find and select virtual objects with the least amount of time possible.

Fig. 2.
figure 2

User study – navigation

Full size image

User Study – Assistance.

The third activity is an assistance task. The goal of this activity is to determine how mixed reality (MR) can enhance the communication between elderly people, their family members, and their closest friends, as well as how it can provide remote consultation between elderly people and healthcare providers.

Each activity, as well as the measurement of completion times and error, will begin as soon as the participants perform any of the selection technique discussed above.

Three activities were constructed with an overall aim of gathering enough information about the most suitable and preferred interaction techniques of AT for elderly people. This will be done through quantitative analysis of the participants’ performance and qualitative analysis of participants’ feedbacks.

Quantitative Evaluation - Questionnaire Design.

After completing each activity, participants will be asked to complete a questionnaire. The questionnaire consisted of several parts which aims to determine the efficiency and effectiveness, along with user acceptability and user satisfaction. The perceived ease of use of the developed system was measured using the System Usability Scale (SUS). SUS aims to cover different aspects of the system:

  • Effectiveness (how well users successfully achieve their objectives)

  • Efficiency (how much effort and resources are necessary to achieve their objectives)

  • Satisfaction (how much the users’ expectations are met when they achieve their objectives)

Qualitative Evaluation - Interview Design.

Semi-structured interviews were conducted during the initial workshop. The focus of questions to participants was to learn what the participants think of emerging technologies, how they see these technologies to be useful to their daily lives, what is the impact such technologies might bring, whether they can see themselves using such technologies, what are the barriers for technology adoption, and what they would want such technologies to have. In addition, the interview was complemented with observations about their current life situation and experiences, the social relationship they currently have, specific issues they encounter daily, and the types and uses of technology in their daily lives. This was done to capture how current technologies address the needs of elderly people, and how emerging technologies can improve aspects that are not currently being focused. Gathered data were analysed by creating affinity diagrams.

A scenario was designed in order to ensure that all participants refer to the same vision – a smart personal home environment where everyday objects are ‘smart’ with diverse functions: full automation, remote control functionality, indoor monitoring of daily activity, etc. The other stakeholders (e.g. family members, closest friends, and healthcare providers) were asked to put themselves in an elderly person’s position, to identify everyday objects that might need to be smart, and to imagine how he will interact with these smart objects. Subsequent to the scenario, the participants were asked to assess the perceived benefits, barriers, and the acceptance of as well as intention to use the described smart personal home environment.

3.2 Iterative Development Cycle

During this stage, an initial prototype using Mixed Reality (MR) and Internet of Things (IoT) technologies is developed. The system requirements were obtained through affinity diagrams constructed based from participants’ performance and feedback during the participatory design workshop.

3.3 Evaluation Workshop

In this workshop, the prototype is presented to the participants and evaluated through quantitative and qualitative measures.

4 Results

An initial workshop with 12 elderly people was conducted to introduce the emerging technologies (such as Internet of Things, Smart Home, Mixed Reality Technology) and the different perspectives of the elderly people are presented below. In this section, we summarise the lessons that we have learned and draw observations from the initial workshop. In addition, we constructively discuss the benefits and drawbacks of each phase to inform current practices which are still mostly technology driven.

4.1 Participatory Design Workshop

The initial workshop was carried out as part of Holdsworth Community’s afternoon discussion group sessions on Tuesdays. It lasted for lasted approximately two hours. Our primary aim was to explore the initial reactions to Mixed Reality (MR) environments for elderly people. In the initial workshop, where participants were seated around a table, an introduction about the research was conducted. This included the process of introducing the Internet of Things (IoT) technologies, explaining how IoT devices are being incorporated in a house to make it smart, showing participants photos and videos of people trying out Mixed Reality (MR) devices, asking them to try out MR devices, and then discussing current approaches on how technologies are being used to improve the Quality of Life (QoL) of elderly people.

Data Collection and Analysis.

Data from the initial workshop was collected through video and audio recording of the session. The workshop questions were semi-structured. The interviews were transcribed and interpreted through thematic analysis. This allowed us to investigate recurring themes and requirements. Affinity diagrams were constructed to organize ideas, issues, insights into a collection showing common themes and structures [24]. The affinity diagram revealed the scope of the user problem (i.e. issues, worries, etc.) and defined the key system requirements (i.e. reliability, performance, etc.) to generate design ideas. The themes emerged from the interviews through a process of induction [25]. The analysis focused on answering the questions, “what are the opportunities and challenges present when designing and developing an assistive technology for elderly people?”

Establishing Effective Communication.

Effective communication between researchers and participants is one of the most important elements during the initial workshop. Age-related changes in hearing, vision, cognitive, and physical abilities can make face-to-face communication even more demanding [26]. Before the interview, it is useful to gather preliminary data on user background which can help assess the level of sensory impairments of the older person. Following the advice of the Holdsworth Community, proper care was taken to make sure that the researchers conducted the interview in a room with a low background noise, the workshop instructor had a moderate speech rate, audible voice, and clear articulation, and the workshop slides contain texts which are readable from a distance.

Mixed Reality Device Introduction.

Older participants are generally unfamiliar with the Microsoft HoloLens device due to its relative newness and less accessibility of the consumer market owing to its high cost. When introducing the device to the participants, it was necessary to play videos of people trying the device and to show a live demonstration where they learned the basic interactions with the device. In addition, the live demonstration was supplemented with a live-stream first view of the scene. This allowed the workshop instructor to share the experience with the participants by allowing them to see the holograms as he sees them. During the workshop, the HoloLens was connected to the community centre’s wireless internet connection (over WiFi) and the Mixed Reality Capture feature was accessed through the Windows Device Portal.

Genuine Interest.

During the initial workshop, the participants expressed great interest in integrating emerging technologies into their daily lives. After listening to the workshop introduction, the participants immediately wished to try the HoloLens. The initial workshop showed that elderly people are interested and able to take part in participatory design workshop on combining Internet of Things (IoT) and Mixed Reality (MR) technologies as an Assistive Technology (AT).

Basic HoloLens Interactions.

Because of the time constraints during the initial workshop, only half of the participants were able to try the HoloLens. They were amazed on how the workshop instructor can interact with virtual objects around him. One participant even said: “You can click the air”. Although the participants worried that they might not be able to perform the gestures correctly, they showed particular interest in learning how to use it. After trying the different gestures, one participant said, “Once you are used to doing that [referring to the gestures], you can do anything!”

Initial Reactions.

After learning the basic HoloLens interactions, the participants were asked to try the 3D Viewer which is a pre-installed and standalone application on HoloLens. It allowed them to easy place, scale, and rotate 3D models and view animations in real-time. The participants were immediately impressed - “This is amazing!” “Absolutely incredible.”

Entertainment.

The participants thought that Mixed Reality devices can be used to revisit memorable places but could not visit anymore or explore unfamiliar places that are hard for them to visit. Someone argued that instead of spending money to travel overseas, one can just buy the device, put it on, and stay in the comfort of your own home. She said, “You could go on holidays but sit on your own armchair.” and added, “You could also go underwater and see, you could go swimming.” Most participants affirmed by saying, “I can imagine.” Although it is convenient to visit places without physically being there, the older participants think that would make them feel isolated. One argued, “Don’t you think that’s really sad?” In line with this, discussions about how the HoloLens can be used to improve social communication between elderly people, their family, their closest friends, and their healthcare providers were talked about. The older participants were likely to perceive the usefulness of this technology for staying in touch with people over long distances. It is important to consider how elderly people can share entertaining experiences and meaningful experiences with their loved ones when using Mixed Reality devices.

Assistive Technology.

Interesting suggestions on the feasibility of the HoloLens device as an assistive technology for older people were drawn from the initial workshop. Participants were convinced that the device can be used as a navigational aid to help in wayfinding in indoor and outdoor environments, especially when they get lost. In addition, they suggested that the device can be useful in locating important objects around their houses. Finally, they argued that the device can be used to connect with smart objects around the house and to talk to them if they forgot to turn off something.

Perceived Usefulness Outweighs Device Cost.

One participant expressed curiosity about how expensive it is to use the technology. She asked, “Is it too expensive to use each time you get on it?” When told about the unit price of one Microsoft HoloLens device, some participants expressed their shock: “So, we’re not talking about little bits and pieces. It is a serious money.” Although the device itself is expensive, participants discussed about the perceived usefulness of the device. They think that the potential benefits outweigh the device cost. “Well, you have to need it. If you need it, then it’s cheap. It’s fabulous!” A participant asked the group and said “Would you like to use it and what for? Forget about the cost.” The group noted that they are currently unaware on how it could be useful for them. However, they think that if developers are able to make applications that are useful to them, they definitely would buy and use the device. We also believe that based on early prototypes, we can provide scalable and cheap solutions by utilizing commonly available mobile devices or tablets such as iPad.

Social Acceptance.

Older participants expressed great concern on how social interactions will be like in the future. While the workshop instructor was performing gestures on the HoloLens, a participant said, “So, if you think that kids are unresponsive now, imagine how will the world interact with each other. We will be different people then.” Older participants also express their concerns about the potential of stigma from others who think that wearing HoloLens makes them look foolish. One participant said, “Can you imagine yourself going to Woolies? [a supermarket]” Everyone laughed with her. When queried about this, participants indicated that they found the large form factor of the HoloLens a bit intimidating. For this reason, participants talked about the need to have an aesthetically-pleasing device. It is clear that developing an assistive technology for elderly people should be done in such a way that they are evaluated as socially acceptable and even desirable.

Perceived Drawbacks.

The participants argue that the automation brought by the Internet of Things (IoT) technologies may promote sedentary living conditions making elderly people more immobile. When asked if the participants are willing to use the device to operate smart objects in the house, one argued, “We were told that we got to be moving all the time and exercise. Why would we be doing that? Say, turn the lights on. [then proceeds with an air tap gesture]” However, one participant suggests that elderly people who are physically infirmed will greatly benefit from this kind of technology because of the convenience it offers. When designing for an assistive technology for elderly people, it is important to offer convenience without sacrificing the need for them to physically move.

Device Comfort/Ease of Use.

Most of the participants found the HoloLens device problematic due to its weight. While putting on the device, one participant was shocked: “Oh my God! It is so heavy!” During the initial workshop, older participants found it hard to carry the device for an extended period of time. A smaller and lighter device form factor might deal with this issue. In addition, one participant noticed that using the device can be quite disorienting. When asked to explain further, she added “You’d have to have a really good balance. Because it is so disorienting. It’s quite freaky that you are looking at something that is not reality.” Another participant affirmed by saying, “You lose association with reality.” When asked about the experience, she said, “It is like as if I was sitting in a cinema.” Another one explained: “It’d be like trying new glasses” However, they pointed out that it just needs getting used to – “There’s a lot of learning to go with it.”

Benefits and Drawbacks.

While it can be difficult for participants to talk about how emerging technologies can help elderly people in their daily lives, showing videos of people trying the devices and explaining how these devices are being used in different application areas elicit views and provide opportunities for specific design ideas to be critiqued.

5 Discussion

In this study, we considered and compared diverse stakeholder perspectives (elderly people, family members, closest friends, and healthcare networks) in order to explore the opportunities and challenges inherent to the development of an Assistive Technology (AT) to improve the Quality of Life (QoL) and to maintain the self-independence of elderly people. The results provide valuable interaction design principles in developing ATs for elderly people.

5.1 Interaction Design Principles

Understanding the increasing needs of elderly people is critical and crucial to the design and development of ATs. There is a common perception that there is a digital divide where the current old generation is comparatively inexperienced with technology than the younger generation [27]. However, this is a huge misconception since the current share of the population that is over 65 has been through a number of technological innovations during their lifetime [28]. The real and pressing problem when designing and developing ATs for elderly people lies on how to overcome the gradual changes in perceptual and motor skill capabilities that often accompany the aging process in elderly people. In this section, interaction design principles for developing Assistive Technology (AT) for elderly people are outlined. Given the relative newness of this area of research, this can be used as a foundation principle to ensure that ATs are designed in a sensitive way and optimized for elderly people, to provide high-quality positive experiences, and to maximise user satisfaction.

Effective communication between researchers and participants is one of the most important elements during the initial workshop. During the initial workshop, it was evident that the older participants have a limited knowledge about the emerging technologies, such as Mixed Reality (MR) and Internet of Things (IoT). While it can be difficult for participants to talk about how emerging technologies can help elderly people in their daily lives, showing videos of people trying the devices and explaining how these devices are being used in different application areas elicit views and provide opportunities for specific design ideas to be critiqued. A positive interest towards how these technologies can be integrated in elderly people’s daily lives were observed during the initial workshop. The initial workshop captured the elderly participants’ initial reactions on what they think of emerging technologies, how they see these technologies to be useful in their daily lives, what is the impact such technologies might bring, whether they can see themselves using such technologies, what are the barriers for technology adoption, and what they would want such technologies to have.

5.2 Limitations and Further Research

When interpreting the results, the reader should be aware of certain limitations. First, the low number of participants during the initial workshop limited the statistical analyses that could be completed by the study. Instead, we present observations from the initial workshop with 12 elderly people. Arrangements with the Holdsworth Community were already made to recruit more participants and to perform statistical analyses and make generalisations in the future. Second, an ideal scenario where elderly people wearing a Mixed Reality (MR) headset and interacting with smart objects in a smart home was presented to the participants. A working prototype is currently being developed to validate the presented observations. Thirdly, the time constraints during the first visit to the community centre limited the user study that the researchers were able to perform. Although the three activities (selection, navigation, and assistance) for the user study were already fully implemented in a standalone application, only an initial workshop was conducted during the first visit. This is considered as the first phase of the project which informs the succeeding workshops and refines the final Assistive Technology (AT) for elderly people.

6 Conclusion

In this paper, we present an interaction design research which aims to explore opportunities and challenges inherent to the development of an Assistive Technology (AT) for elderly people. The proposed AT, which is a combination of Mixed Reality (MR) and Internet of Things (IoT) technologies, aims to improve the Quality of Life (QOL) and to maintain the self-independence of people aged 65 or above. The intended users are elderly people, their family, their closest friends, and their healthcare network. The main contribution of this project is to provide a set of interaction design principles for combining MR and IoT as an AT. This is achieved by a carefully planned participatory design approach. The benefits and drawbacks of each phase are discussed transparently to inform current practices which are still mostly technology driven. In addition, interesting observations captured during the initial workshop with elderly people, family members, closest friends, and healthcare networks were presented. This included their initial reactions, perceived benefits, impact of technology on their daily lives, barriers to technology adoption, and their suggestions on how these technologies can be integrated in their daily lives.