Fitmirror: A Smart Mirror For Positive Affect in Everyday User Morning Routines

Author pre-print version. Not for redistribution. Please check the original publication for the final version.
FitMirror: A Smart Mirror For Positive Affect in

Everyday User Morning Routines
Daniel Besserer, Johannes Bäurle, Alexander Nikic

Frank Honold, Felix Schüssel, Michael Weber
Ulm University, Germany
{daniel.besserer, johannes.baeurle, alexander.nikic,
frank.honold, felix.schuessel, michael.weber}@uni-ulm.de
ABSTRACT person. So far, no smart mirror that offers added value has
This paper will discuss the concept of a smart mirror for been developed. Moreover, none of the already mentioned
healthier living, the FitMirror. Many people have serious mirrors is able to recognize fitness exercises and offer inter-
problems to get up after sleeping, to get motivated for the action. This paper presents a smart mirror that provides a
day, or are tired and in a bad mood in the morning. The combination of several functions and input modalities (cf.
goal of FitMirror is to positively affect the user’s feelings by Fig. 1). It includes displaying information, such as weight
increasing his/her motivation, mood and feeling of fitness. and other health data, as well as recognition of sports exer-
While concepts for these isolated problems exist, none of cises for affecting the users’ feelings positively and motivate
these combine them into one system. FitMirror is imple- them. This raises the question how the feeling of a user
mented to combine them and evaluate them in a study. It can be manipulated. The implemented smart-mirror, the
consists of a monitor with spy-foil, a Microsoft Kinect v2 and so-called FitMirror, provides the user with normal and fun
a Wii Balance Board and can recognize users and their ges- exercises. These exercises are intended to increase the moti-
tures with these elements. Several hypotheses about the sys- vation, happiness and fitness of the user after getting up in
tem regarding motivation, fun, difficulty and getting awake the morning. The concept of FitMirror should be regarded
were investigated. Participants were grouped by the factors as part of a smart home. Before it is described in detail,
sportspersons and morning persons to investigate the effect related work that is relevant for the concept of FitMirror
based on these aspects. Results show that FitMirror can and the underlying technology is presented. These projects
help users get awake in the morning, raise their motivation are the basis for the concept of FitMirror and show what
to do sports and motivate them for the day. kind of technologies are available. Afterward, the concept
is presented and evaluation results are reported. The paper
closes with ideas for future projects.
CCS Concepts
•Human-centered computing → User studies; Em-
pirical studies in HCI; Graphical user interfaces; Sound-
based input / output; Touch screens; HCI theory, concepts
and models; Gestural input;
Keywords
smart mirror; human-computer interaction; emotion
1. INTRODUCTION
In recent years, companies1 and scientists [3, 17] have de-
veloped smart mirrors, whether in a smart home context,
only to display information, or for consumers in boutiques
or shopping malls. These mirrors were only developed to
either show general information, or 3D visualizations of a Figure 1: A user performing the exercise “Boxing”
1
in front of FitMirror.
e. g., http://memorymirror.com/
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the 2. RELATED WORK
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or In the following, some similar projects in the area of smart
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org. mirrors or magic mirrors, which already exist or have been
MA3HMI’16 , November 16 2016, Tokyo, Japan described in theory, will be briefly described and discussed.
c 2016 Copyright held by the owner/author(s). Publication rights licensed to ACM.
In addition, the topics “Activity Tracking”, “Emotions”, “In-
ISBN 978-1-4503-4562-0/16/11. . . $15.00 teractive Games” and further important points for the con-
DOI: http://dx.doi.org/10.1145/3011263.3011265 cept of FitMirror and its study are discussed.
Author pre-print version. Not for redistribution. Please check the original version for the final version.
ICMI ’16, MA3 HMI Workshop, November 12–16, 2016, Tokyo, Japan
2.1 Smart Mirrors physical activity monitoring in order to test their accuracy.
Several “smart mirrors” or “magic mirrors” have already After giving an overview about the development of wear-
been invented2 , evaluated in studies [15, 10] or written as a able accelerometer-based motion detectors they review their
theory [4]. They display information, or show the user itself selection. According to the review all seven products are
on a standard screen or a reflective screen with a spy-foil. relatively reliable when it comes to measuring the steps of
In this section, some of these mirror concepts are presented the user and calculating the energy expenditure.
and described based on a category of two different types:
augmentation and information. 2.3 Emotions
There is a great deal of research about smiling and fa-
2.1.1 Augmentation cial expressions [16, 2, 9]. The study of Strack et al. [16]
Augmentation mirrors are smart mirrors which render ad- reveals that activating the same facial muscles involved in
ditional information, e.g. makeup or dresses, on 3d visual- smiling manipulates the decision making of the participants.
izations of the user. Cartoons were assessed funnier by the participants who had
Iwabuchi et al. [10] developed a makeup mirror, which activated their smiling muscles. However, not every smile is
helps women when putting on make up. This mirror uses equal. Ekman [5] suggests there are about 50 different kinds
a camera with a resolution of 1624 x 1224 pixels and 30 of smiles. These smiles are differentiated in “genuine” and
fps to take the pictures of the women. The system’s algo- “standard” according to the muscle activated near the eye
rithm is only trained for women. The recorded pictures are and mouth. Zygomaticus major muscles around the mouth
shown on a monitor. When a woman is close to the monitor were stimulated by the “standard” smile and the “genuine”
and camera, the displayed picture becomes blurred, because smile stimulated the orbicularis oculi muscle near the eye
the camera has no integrated auto focus. To compensate as well the zygomaticus muscles [7]. In addition, brain re-
that, an automatic zoom was implemented that reacts when gions liable for positive effects are activated while smiling in
the woman is using a specific control marker. The marker the “genuine” type of smiling [6]. Kraft and Pressman [12]
is recognized by another camera. Different lighting modes checked the different stress reduction between three partic-
provide the women with different views of the makeup they ipant groups (“standard” smile, “genuine” smile, no smile)
used (bright light, dark light and so on). Another function after executing a stressful task. The result implied that the
of this makeup mirror is displaying pictures of the woman participants, especially those with “genuine” smiling, had a
before and after she used makeup. more relaxed heartbeat than the participants who did not
smile.
2.1.2 Information
Another type of smart mirrors are the so called “magic 2.4 Interactive Games
mirrors”, which allow displaying information. The following project describes the benefits of interactive
In their project, Blum et al. [3] demonstrate a magic mir- games compared with conventional exercises. The study of
ror for teaching anatomy. They used a display device, color Monedero et al. [14] compared the benefits between normal
camera, depth camera and the Microsoft Kinect v1 from the cycling and interactive video cycling. They came to the con-
Xbox 360 gaming console. When the user stands in front of clusion that the participants had a higher enjoyment during
the display, the mirror loads computed tomographic visual- interactive video cycling than during conventional cycling.
izations or 3D models of organs to give the user “the illusion In addition, the participants worked on a higher intensity in
to look into his body”. With simple hand-gestures the user the interactive video game than in the conventional way of
can change the slices or zoom in and out. cycling.
Further magic mirrors use single board computer tech- In their paper Aarhus et al. [1] report the fitness effects
nologies, e.g. the Raspberry Pi, and monitors with spy-foil. of senior citizens while they are using Nintendo Wii Fit for
In most instances, those mirrors simply show nothing but a period of six months. To observe their progress, the par-
information. An example for such a mirror is presented by ticipants had to execute a few fitness exercises, for example
Stückler [17]. biceps curls. The results show that there is a positive trend
to physical improvement. However, the motivation of the
2.2 Activity Tracking participants increased, too. But that does not mean that
In a self-monitoring context of daily activities, the track- the Nintendo Wii Fit games replace normal exercise, but
ing of physical activities is indispensable. Lester et al. [13] they can be a supplement. They can also be a supplement
developed a system for recognizing physical activities. They to physiotherapy or be used “as an instrument for rehabili-
put multiple sensors on a multi-sensor board which enabled tative activity”. The study of Theng et al. [18] also comes
them to recognize physical activities such as standing, walk- to the conclusion that the use of Nintendo Wii can improve
ing, climbing stairs, brushing teeth, and similar. Today, the motor skills and hand-eye coordination of seniors.
nearly ten years later, it is not necessary to develop a new
system for this aspect. In recent years the number of com-
mercial fitness trackers such as the Jawbone, Fitbit or the 3. CONCEPT AND USE CASES
Runtastic Orbit has increased. They all advertise their abil- The FitMirror system integrates a smart mirror in the
ity to track the users’ activities and calculate their energy users’ daily morning routine without bothering them. Addi-
expenditure in terms of burned calories. A review by Yang tionally it should increase the users’ motivation, happiness
and Hsu [19] compared seven wearable motion detectors for and fitness for the day by preforming fun and fitness exer-
2
http://www.areamobile.de/news/36473-magic- cises in front of the mirror (cf. FitMirror Video3 ).
mirrorgoogle-now-auf-dem-badezimmer-spiegel-zum-
3
selberbaue FitMirror Video – https://youtu.be/gRLoKyeOzSw
After getting up, the user enters the bathroom. When 3.2 Use Cases
he/she steps in front of FitMirror, it detects the user and There are several use cases based around FitMirror, which
displays the interface. The user logs in and FitMirror loads are described in this section.
step and calorie data from the user’s smartphone. After
loading the data, the user begins to interact with the system. Registering a new user.
He/she has the possibility to check his/her performance, the An unregistered user steps in front of the mirror which
performances of other users, doing some exercises and chal- turns on and the user connects his/her smartphone with the
lenge other users. When the user is not in front of the mirror mirror. The user’s smartphone is detected but not recog-
for at least two minutes it turns off. nized by the system, which causes the mirror to open the
registration dialog for the new user. He/She can either de-
3.1 Features cline registration which closes the dialogue or accept regis-
FitMirror recognizes the presence of a user with the Bal- tration. If the user chooses to accept, he/she is asked to
ance Board and the Kinect v2. When the Balance Board enter his/her first and last name.
detects a user stepping on it, it sends a message to the main
application to turn on the display and activates the Kinect Unregistered user uses FitMirror.
camera. At this point the Kinect checks if the user is still An unregistered user, either without a smartphone or one
standing in front of the mirror. When the user is not stand- that has previously declined registration or connection, steps
ing in front of the mirror for a while, the Kinect recognizes in front of the mirror. The mirror turns on, but only shows
that and sends a message to the main application to turn off the current time, since all other features are unavailable for
FitMirror. To distinguish which user stands in front of the this user.
camera, the mac address of the user’s smartphone is loaded.
In order to avoid extra stress for the user, the interaction Recognition of a user.
with the mirror should be as simple and natural as possible. If a user steps on the Balance Board in front of FitMirror,
For this reason the user has the opportunity to either use the Mirror turns on and the user pairs his/her smartphone,
touch or speech commands to interact with the system. As so FitMirror can load his/her specific data. During syn-
output modality for visual and audio of the mirror system, chronization, the FitMirror app loads step and calorie data
the mirror’s display and loudspeakers are used. from the Google-Fit database, sends it to the main applica-
During synchronization to FitMirror with a smartphone, tion and the Balance Board measures the user’s weight and
the FitMirror App loads the steps and burnt calories data sends the measured value to the main application. Once
from the GoogleFit database and sends them to the main synchronization is finished, FitMirror displays its interface.
application where this data is stored in a database. In ad-
dition, the Balance Board measures the user’s weight and Overview of past day or week statistic.
sends the measured values to the main application where it To check the current status, the user can look at his/her
will be stored in the same database as the other data. data of steps, burnt calories and weight measurements of
The stored data can be shown as diagrams. For this fea- the current day, last day or week. By explicit interaction
ture, the main application of the FitMirror loads the data with the mirror, the user calls up the statistic menu to view
from the last seven days and displays it. The data that is his/her past day or week statistics (steps, calories, weight).
displayed are the measured weight, steps and burnt calories This is important because the users can check their per-
of the user. formances of the last days and at the same time it should
FitMirror offers ten different exercises that the users can motivate them to walk more steps or burn more calories by
execute in front of the mirror. These ten exercise are divided doing more sports.
into two types: fun exercises and normal exercises. The
fun exercises are intended to increase the user’s happiness
Challenges.
and the normal exercises are intended to increase the user’s
To manipulate the users’ emotional state, they perform
motivation and fitness.
challenges in front of the mirror. The mirror challenges the
With the aid of the Kinect camera and the Balance Board,
user from time to time. This happens either randomized
FitMirror can recognize the different exercises. The Kinect
or if a friend challenges them. The user can now either ac-
camera scans the user’s skeleton and can differ between var-
cept or decline the challenge via touch or speech. When
ious implemented gestures. The Balance Board measures
the user accepts a challenge, he/she gets a textual instruc-
changes in the user’s weight distribution to detect particu-
tion and a video how to execute the exercise correctly. If a
lar patterns of movement to determine the exercises.
challenge is started, the necessary sensors have to be ac-
The performance of each exercise of the user is saved in a
tivated and the challenge’s patterns must be recognized.
highscore list. The users can look at all of their performances
This means analyzing weight distribution and position of the
of the exercises in that list. Moreover, the user can look at
user’s limbs. While performing the challenge, repetitions are
the performances of the other registered users in FitMirror
counted. Once the user has finished the challenge, the re-
to compare them with his/her own performances.
sult is displayed on the screen and written to the database.
Another feature of the FitMirror is that the users can
After performing the activity the system adjusts the diffi-
challenge other users with an exercise. After a user has
culty level of future challenges in form of more repetitions
completed an exercise, he/she can challenge another user
according to the user’s achieved repetitions.
with the achieved score by choosing a user from the user
list. When the challenged user uses FitMirror, he/she gets
a notification about a challenge request. They can accept or 3.3 Multi-User
decline the request. In modern times, people want to connect with each other,
especially with their friends. They want to exchange and at the bottom of the screen. A Ball bounces off walls and
compete with others. Because of this, the mirror is able the bar. The goal is to keep the ball in the air above the
to show profiles of the user’s friends, similar to social net- bar.
works. On these profiles the user can see their statistics on While Balancing the user stands on the Balance Board
challenges. Playing alone is not as much fun as competing and his/her position is displayed through a cursor on the
with others, so the user is able to challenge either himself or screen. By shifting his/her weight one can move the cursor
one of his friends with an activity. For this action he/she can in order to hit randomly appearing targets.
activate a multiplayer option on a specific activity or dare For eye-hand coordination the exercise Boxing was in-
his/her friends on their profiles. After choosing the challenge cluded. Hand tracing is provided by the Kinect. The hands
and player-mode, the user has to do the activity him/herself, are displayed as cursors and like during balancing, the user
whether or not he/she picked single or multiplayer mode. has to hit a random target with the cursor.
In order to affect the emotion of the user in a positive way
4. IMPLEMENTATION Smiling was implemented. The user has to smile repeatedly.
His/her facial expression is recognized by the Kinect.
This chapter shows the implementation of the FitMirror
The last exercise in this category is Stretching the body.
system. The hardware an software used for FitMirror are
Body recognition is enabled by the Kinect and the user has
described as well as all implemented exercises.
to stretch his/her arms repeatedly above his head.
4.1 Hardware 4.3.2 Normal Exercises
A 42 ” monitor was used as the mirror’s screen. In order to
To move the whole body of the user, the exercise Jogging
get a reflection of a user like a mirror does, spy-foil was used.
was provided. Here the user has to jog on the Balance Board
Because of its electronic conduction the foil cannot be fixed
which also counts the steps.
directly to a capacitive touch monitor. Therefore the foil
In order to train the user’s arms, the exercise Barbell Curls
was fixed to an Actalyst infrared camera touch-panel from
is implemented. The user has to execute a biceps curl with
SMART Technologies Inc. in front of the monitor. Body
a given object, e.g. a bottle. The execution is tracked by
recognition is provided by the Microsoft Kinect v2. It is
the Kinect in form of a learned gesture.
used to detect and recognize persons, their facial expressions
Pushups train the arms and the chest. The user has to
and gestures or exercises. In order to measure weight and
place his hands on the sides of the Balance Board which
sports activity, the Wii Balance Board “E.A. Sports Active
tracks peaks in weight progression. Because of the small
Edition” was used. The four sensors in each corner allow
width of the Balance Board the Pushups are more likely
the detection of movement and shifting of weight, as well as
training the triceps of the user than the biceps.
normal weight measurement. In other studies or projects [8],
Like Pushups, the Balance Board tracks peaks with the
the Balance Board was proven to be very useful.
exercise Squats. The user has to stand on the Board and do
4.2 Software as many squats as possible.
For the purpose of training calves and balance the last
The main application as well as the exercise recognition
normal exercise is Raise Calves. Again the Balance Board
applications were programmed in C#. The Kinect SDK
tracks weight progression, especially weight differences in
was used to aid in easier development of the application,
upper and lower regions of the board. A peak is marked as
since it provides automatic recognition of pre-defined ges-
a repetition.
tures, which speeded up development. The other part for
recognizing exercises is the Balance Board. To get the data
from the Balance Board’s sensors, the “Wii Device Library” 5. HYPOTHESES
was used. The following hypotheses were evaluated with the study:
To track the steps and calories of a user during his/her
daily life, Android smartphones with the Google Fit app H1: Users are feeling motivated to do sports by using Fit-
were used. Android was used because it is open source and Mirror.
the Fit API is a good way to receive already existing fit-
H2: An increase in the level of difficulty motivates the user.
ness data. During development the LG Nexus 5X running
Android 6.0 and a Samsung Galaxy S4 running Android 4.2 H3: People who have problems to wake up experience Fit-
were used. The mobile application was programmed with Mirror as useful.
Android Studio.
Communication between software components was real- H4: Fun exercises are more popular than normal exercises.
ized using a message oriented middleware based on ActiveMQ.
H5: After fun exercises the user feels happier than before.
Data for user identification and exercise history is stored us-
ing an SQLite Database.
6. EXPERIMENTAL DESIGN
4.3 Exercises While performing the study, two kinds of explicit interac-
This section describes the possible exercises which are pro- tion were used: touch and speech. To feel most comfortable,
vided. They are separated in two categories: fun and normal the participants were allowed to choose the mode of interac-
exercises. tion on their own. Even a switch between the modes within
the study was allowed.
4.3.1 Fun Exercises The preparation of the lab, where the study was con-
The first fun exercise is called Ball. The Kinect tracks the ducted, was also an important aspect. Lights were dimmed
position of the user’s feet. The location is displayed as a bar and cool lighting was turned on, which should imitate a
bathroom atmosphere and support the participant in im- time. To avoid different variants between the participants,
mersing them in a bathroom situation. Some toiletries were we did not change time in the second round. After every
also put up next to the mirror to support this feeling. exercise they had to fill out a small form about their cur-
In order to avoid frustration of a participant if the sys- rent feelings (motivation, happiness, fun and feeling about
tem is not running well or doing something different than difficulty level).
it should do during use, the decision was made to imple- During the execution of the exercises, the observers in the
ment an additional Wizard of Oz (WoZ) scenario, where separate operator-room were observing the participants. If
observers in a separate room control the behaviors of the they had done all exercises and both runs, the participants
system while the exercises are executed. When the partic- were asked to fill out the last form about ranking exercises
ipant executed an exercise, the observers counted the rep- subjected to fun and difficulty level. They were also allowed
etitions, so that the participants thought that everything to give some general feedback.
worked fine. The observers paid attention that all exercises At the end, they were informed about the WoZ procedure
were executed in the same way by each participant. If a and asked to keep it as a secret until the end of the study.
participant executed an exercise in the incorrect way, the
observers did not count the repetition until the participant 7. RESULTS
performed it correctly. This should ensure that each partici- From the 43 participants 839 valid sets of exercise data
pant was treated equally. Only exercises that were based on were retrieved. However, due to technical problems the data
repetition (squats, pushups, raising calves, barbell, stretch- from two participants could not be used for all hypotheses.
ing) were realized by the WoZ. All exercises based on live This reduces the amount for most research questions to 41
data were not faked (jogging, hit box, boxing, ball). Smiling participants.
recognition was also real, even though it was a repetition ex- Based on the results of the Sport AMS form, the partic-
ercise. This was due to the fact that manual counting was ipants were grouped into three different kinds of sportsper-
not possible here. son: 26.83% (11 participants) had high, 58.54% middle (24)
The length of the study for each participants was about and 14.63% low degrees (6) of sports motivation.
45 to 60 minutes. They were paid 10 Euros for their partic-
ipation in the study. 7.1 H1: Sports motivation
6.1 Participants The results regarding our first hypothesis show that 62.79%
of participants (27) found that the system motivated them to
43 participants (19 f, 24 m) were recruited on university exercise in the early morning. Only 13.95% (6) reported that
campus, all of them were students. The average age of the FitMirror did not motivate them and the remaining 23.26%
participants was 24.21 (SD=2.2). Students were from vari- (10) were indifferent. All different sportsperson types an-
ous study courses such as computer science, psychology, and swered FitMirror did motivate them in general. Especially
biology. Due to the broad distribution of course of studies, amongst high and low motivated types 80% (9) and 66.67%
the study did not only deal with computer experts but also (4) participants agreed, respectivley. From the middle mo-
with unskilled computer users. Except for one student, all tivated type 55.56% (13) did agree, which is still more than
students were native speakers. the half of them (see Fig. 2).
6.2 Experimental Procedure
motivation in
First of all the participants were greeted by the examiner 80,0% sports
in the lab and were asked to read and sign the declaration of low
consent. Afterward, they had to fill out a demographic form middle
about their properties including age, gender and the single high
question if one is a morning person or not4 . After the demo- 60,0%
graphic form, they had to fill out “Sport AMS” (Achievement
Percentage
Motives Scale-Sport5 ) and “TA-EG” (Technology affinity for

electronic devices [11]) forms. When all questionnaires were
40,0%
answered the examiner explained the functions of FitMirror
and the procedure of the study. In addition the participants
were given a short introduction about the speech command
control. In case they forgot the speech commands, a list 20,0%
of all speech commands was hung under the Mirror. The
procedure started when the participants agreed they had
understood all introductions and stepped on the Balance
0,0%
Board. no I don't know yes
Each participant had to execute 10 exercises twice in ran-
The system raises my motivation to
dom order to avoid learning effects. At the second round, excercise in the morning
the level of difficulty was raised in the form of suggestions
to beat the last score with 10% more repetitions in the same
Figure 2: High and low motivated sportspersons
4 show high motivation to exercise using FitMirror.
A morning person has no problems getting up or is in a
good mood after awakening from sleep in the morning.
5
http://www.bisp-sportpsychologie.de/SpoPsy/DE/
Diagnostikportal/Motivation/Sportlerfrageboegen/ams/ Concerning the question, whether the participants believe
ams.html that the system would raise their motivation for the day, 70%
Motivation in happier, an obvious approval was available in both groups

60,0% sports (morning persons: 86.36% (19), non morning persons: 85%
high (17)). Regarding the question if the system raises their mo-
middle tivation to do sports in the morning, both groups agreed
50,0%
low (morning persons: 60%(12), non morning persons: 62% (14)
see Fig. 4). The participants also agreed on the question, if
40,0% the system would motivate them for the day.
Percentage
30,0% Morning
person
no
20,0%
60,0% yes
10,0%
Percentage
0,0% 40,0%
worse same better
Performance
Figure 3: Comparison of performance in the second 20,0%

round based on sports motivation. Only low moti-
vated participants performed noticeably worse.
0,0%
of participants (30) answered the question with an agree-
no I don't know yes
ment, 4.65% (2) disagreed and the remaining 25% (11) ab-
stained from that question. The system raises my motivation to
excercise in the morning
7.2 H2: Difficulty Level
Answering the questions for the hypothesis about if the Figure 4: Motivation to do sports in the morning
automatic increase of the difficulty level motivated the par- per morning person type.
ticipants, they had to choose between “too difficult”, “diffi-
cult”, “appropriate”, “easy” and “too easy”. No participants 7.4 H4: Fun exercises vs. normal exercises
stated that exercises were too difficult and 1 participant
(2.38%) experienced them as difficult. 57.14% of partici- In order to find out whether FitMirror achieves the hy-
pants (24) regarded the exercises as appropriate and 30.95% pothesis, the participants were asked to rank the exercises
(13) as easy. By comparing the first and the second round, in descending order from fun to no fun. Rank 1 and 2 were
we could check if participants performed worse, equal or bet- mostly fun exercises (70% and 50% of exercises), however,
ter. As Fig. 3 shows, about half of every sports motivation rank 9 and 10 were also rated with fun exercises (55% and
group performed better in the second round. Only low mo- 77% of exercises), which seems paradox (see Fig. 5). On the
tivated sports persons performed noticeably worse. middle ranks there was no clear preference of exercise type.
Motivation of the different sportsperson types between the 7.5 H5: Happiness after fun exercises
two runs shows that agreement among highly motivated par-
ticipants rises to full agreement and participants with low Participants are a bit happier after fun exercises than after
sports motivation got a minor increase of motivation. normal exercises. The result of the less motivated sportsper-
sons had the biggest difference, although this was not statis-
7.3 H3: Effect on (non) morning persons tically significant. At the same time, the value of happiness
is quite high in all of the exercises (see Fig. 6). The values
The amount of morning persons and non morning per-
on the question if the exercise was fun are not significantly
sons in our study is quite equal. There were 22 participants
different. Participants seemed to have a lot of fun in both
(52.38%) who have no problems to wake up in the morning
types of exercises. The differences between the values of
(the morning persons) and 20 (47.62%) who have problems
the less motivated sportsperson are the biggest, but are also
in the morning (the non morning persons).
very small and not significant.
When asked whether FitMirror can give the participants
a feeling of awakening, they could answer with “yes”, “no” or
“I do not know”. According to that, there was a mutual and 8. DISCUSSION
clear consent of agreement from both types of morning per- The results of the use of the FitMirror shows some aspects
sons. Especially the morning persons believed that they got which confirm or partially prove our hypotheses.
a feeling of awakening by using FitMirror. The data reveals Concerning H1 “Users are feeling motivated to do sport by
that both groups believed that the system could support using the FitMirror”, the system tends to motivate the user,
them in their morning process (awaking from sleep, getting as all participants showed high levels of agreement.
into a good mood, becoming motivated). The results relating to the difficulty levels imply that
On the question if the FitMirror makes the participants the difficulty levels for morning activation is appropriate or
Motivation

5 high

medium
low

4

Mean FeelHappy

3

2

1

0
normal excercise fun excercise

Excercise form

Figure 6: Happiness after exercise type per sports

motivation. There is no clear effect noticeable.

routine due to time limitations. So it is understandable that

Figure 5: Rank 1 and 2 show high agreement for fun
the participants had a few problems and were frustrated with
exercises (right bar), as did, unexpectedly, rank 9
the ball exercise. These facts could be explanations for the
and 10.
already mentioned paradox from section 7.4. With these
values and perceptions, we suppose that fun exercises could
be more popular than normal exercises when they are well
somewhat easy for the users. This includes the automatic
implemented and when the users get enough time to learn
increase of the difficulty levels. Another point is that the
or if a training period is not needed as in “Balancing” and
participants with middle and high motivation (over 80% of
“Boxing”.
the participants) achieved better scores after the FitMirror
Because of the little differences in the values regarding H5
proposed a goal to them. The conclusion of this is that we
“After fun exercises the user feels happier than before”, we
can say the FitMirror’s automatic increase of difficulty lev-
cannot definitely say that fun exercises made the partici-
els does motivate the user. This is also supported by the
pants happier in our scenario.
agreement of the participants when asked directly.
So overall, FitMirror is able to motivate the user to ex-
We can also confirm that there is a positive effect on the
ercise, motivate them with proposed goals and is a general
mood of non-morning persons. This results in the outcome
support to the morning process of getting motivated.
of our questions if the system could help in the process of
waking up. However, it does not matter too much, if a user is
a morning-person or not, because the system equally works 9. SUMMARY AND OUTLOOK
for both types. FitMirror was created as an interactive device in mind.
With regards to the paradox about the rankings of the We thought about the use cases of a mirror and came to the
exercises we come to a few assumptions. While the partic- conclusion that a mirror is part of one’s morning routine.
ipants executed the fun exercises “Smiling” and “Ball” they Hence we decided to make a mirror which recognizes small
had many more problems than compared to other exercises. exercises and motivates the user for the day. We achieved
We presume that “Smiling” is one reason for the problems, this by fixing a spy-foil on a monitor and implementing ex-
because the counter did not work correctly all time6 . The ir- ercise recognition with the help of the Kinect v2 and a Wii
regular counting confused and frustrated them as they could Balance Board. The mirror should not only affect one’s fit-
not imagine what they did wrong. The reason for this mis- ness but also the mood of a user. Therefore we built recog-
counting is incorrect differentiation between mouth gestures nition for fun and normal exercises.
of the Kinect. The camera did not always provide good In order to analyze the FitMirror system we ran a study.
smile recognition. One result of this study is that FitMirror is able to help non-
In spite of the given instructions on the exercise “Ball”, morning persons as well as morning persons in the process of
the participants still had many problems during execution. getting up and motivates for the day. It was also noticeable
So they had the impression that the system did not work that participants felt not only motivated for the day, but
correctly, whereas they just did not move in the way they also to do sports during the use of the FitMirror.
should have. We assume that the user should have a longer The most remarkable result was that all types of morning-
training period, which we couldn’t integrate in the study persons believe that the system supports their morning pro-
cess. Furthermore, we revealed that middle and low moti-
6
According to some participants’ post-test statements vated sportsperson achieve better scores when the system
proposed them a target, which means they were more mo- [7] P. Ekman and W. V. Friesen. Felt, false, and
tivated. Judging from this, the users feel motivated to do miserable smiles. Journal of nonverbal behavior,
sports while using FitMirror. 6(4):238–252, 1982.
It is important to get robust recognition systems, as the [8] S. H. Fraile, J. Browne, E. Brox, and G. Evertsen.
faulty smiling exercise clearly induced frustration in some Suitability analysis of commercial open-source driven
of the participants. In addition, it should be considered motion sensor devices applied to exergames for the
whether more intensive exercises like ball should be replaced elderly. In Ambient Assisted Living Forum Eindhoven,
by lighter exercises, if the instructions for the user should be 2012.
performed longer and included with workouts. Furthermore, [9] L. Harker and D. Keltner. Expressions of positive
a long term study should be conducted to check if these emotion in women’s college yearbook pictures and
kinds of systems lead the users to feel fitter the longer they their relationship to personality and life outcomes
are using them. across adulthood. Journal of personality and social
Another use case that could be examined is to use such a psychology, 80(1):112, 2001.
mirror as a supporting and instructive companion for ath- [10] E. Iwabuchi, M. Nakagawa, and I. Siio. Smart makeup
letes in gyms. One could integrate an electronic coach into mirror: Computer-augmented mirror to aid makeup
the mirror that exactly explains how athletes have to exe- application. In Human-Computer Interaction.
cute an exercise correctly, and what they have to consider Interacting in Various Application Domains, pages
to reduce injuries. Also in point of rehabilitation one could 495–503. Springer, 2009.
use such a mirror. A lot of patients need to do exercises at [11] K. Karrer, C. Glaser, C. Clemens, and C. Bruder.
home but without any supervisor. To compensate the su- Technikaffinität erfassen–der Fragebogen TA-EG. Der
pervisor, a fitness mirror could be used that not only checks Mensch im Mittelpunkt technischer Systeme,
whether the patient executes his/her exercises in a correct 8:196–201, 2009.
way, but also sends a report about the results and progress
[12] T. L. Kraft and S. D. Pressman. Grin and bear it the
of the patient to his/her medical doctor.
influence of manipulated facial expression on the stress
Such smart mirrors could also be integrated in a real smart
response. Psychological science, 23(11):1372–1378,
home concept by connecting the mirror with other compo-
2012.
nents. So the mirror could provide users with the opportu-
nity to select a recipe for breakfast and send it to a smart [13] J. Lester, T. Choudhury, and G. Borriello. A practical
kitchen, that, regarding the received recipe, prepares the approach to recognizing physical activities. In
pans and ingredients. Or the user would manage his day Pervasive Computing, pages 1–16. Springer, 2006.
in the bathroom by using the mirror. Thereby the mirror [14] J. Monedero, E. J. Lyons, and D. J. Gorman.
would be connected with the user’s calendar. Interactive video game cycling leads to higher energy
expenditure and is more enjoyable than conventional
exercise in adults. PloS one, 10(3):e0118470, 2015.
10. ACKNOWLEDGMENTS [15] A. Rahman, T. T. Tran, S. Hossain, and A. El Saddik.
This work was supported by the Transregional Collabora- Augmented rendering of makeup features in a smart
tive Research Center SFB/ TRR 62 “Companion-Technology interactive mirror system for decision support in
for Cognitive Technical Systems”, which is funded by the cosmetic products selection. In Distributed Simulation
German Research Foundation (DFG). and Real Time Applications (DS-RT), 2010
IEEE/ACM 14th International Symposium on, pages
203–206. IEEE, 2010.
11. REFERENCES [16] F. Strack, L. L. Martin, and S. Stepper. Inhibiting and
[1] R. Aarhus, E. Grönvall, S. B. Larsen, and S. Wollsen. facilitating conditions of the human smile: a
Turning training into play: Embodied gaming, seniors, nonobtrusive test of the facial feedback hypothesis.
physical training and motivation. Gerontechnology, Journal of personality and social psychology,
10(2):110–120, 2011. 54(5):768, 1988.
[2] E. L. Abel and M. L. Kruger. Smile intensity in [17] M. Stueckler. Magic Mirror: Spiegel zeigt Wetter und
photographs predicts longevity. Psychological Science, Nachrichten beim Zähneputzen dank Raspberry Pi,
21(4):542–544, 2010. 2014 (accessed April 1, 2016).
[3] T. Blum, V. Kleeberger, C. Bichlmeier, and N. Navab. http://http://t3n.de/news/magic-mirror-spiegel-zeigt-
mirracle: An augmented reality magic mirror system nachrichten-560292/.
for anatomy education. In Virtual Reality Short [18] Y.-L. Theng, A. B. Dahlan, M. L. Akmal, and T. Z.
Papers and Posters (VRW), 2012 IEEE, pages Myint. An exploratory study on senior citizens’
115–116. IEEE, 2012. perceptions of the nintendo wii: the case of singapore.
[4] G. Demiris, B. K. Hensel, et al. Technologies for an In Proceedings of the 3rd International Convention on
aging society: a systematic review of “smart home” Rehabilitation Engineering & Assistive Technology,
applications. Yearb Med Inform, 3:33–40, 2008. page 10. ACM, 2009.
[5] P. Ekman. Telling Lies: Clues to Deceit in the [19] C.-C. Yang and Y.-L. Hsu. A review of
Marketplace, Politics, and Marriage (Revised Edition). accelerometry-based wearable motion detectors for
WW Norton & Company, 2009. physical activity monitoring. Sensors,
[6] P. Ekman and R. J. Davidson. Voluntary smiling 10(8):7772–7788, 2010.
changes regional brain activity. Psychological Science,
4(5):342–345, 1993.

Fitmirror: A Smart Mirror For Positive Affect in Everyday User Morning Routines

Uploaded by

Copyright:

Available Formats

Fitmirror: A Smart Mirror For Positive Affect in Everyday User Morning Routines

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Fitmirror: A Smart Mirror For Positive Affect in Everyday User Morning Routines

Uploaded by

Copyright:

Available Formats

Author pre-print version. Not for redistribution. Please check the original publication for the ﬁnal version.