MASARYK UNIVERSITY

FACULTY OF INFORMATICS

Quality Assurance in Game Development

Master’s Thesis

Matej Komár

Brno, 2017
Declaration
I declare that I have worked on this thesis independently using only
the sources listed in the bibliography. All resources, sources, and
literature, which I used in preparing or I drew on them, I quote in the
thesis properly with stating the full reference to the source.

_____________________
Matej Komár

Supervisor: Mgr. Jiří Chmelík, Ph.D.

ii
ACKNOWLEDGEMENTS
I would like to thank to Mgr. Jiří Chmelík, Ph.D. for supervision,
willingness and help while working on my thesis. Also I would like
to thank to Reborn Interactive for opportunity to work and gain
experience in Quality Assurance field. Last and not least I would like
to thank to all my family and friends who supported me for the
whole duration of my studies.

iii
Resume
The aim of this work is to study and describe Quality Assurance.
Mostly specified in the game industy and introduce methods used in
this process. Afterwards use the knowledge of these method to
analyse chosen games with additional personal testing. Describe
stages of analysis and present the found results.

iv
Key words
Quality Assurance, Game Development, game evaluation, testing,
bug

v
Content
1 Introduction ........................................................................................... 3
2 Quality Assurance ................................................................................ 5
2.1. Software Quality Assurance ......................................................... 5
3 Quality Assurance in Game development ....................................... 7
3.1 Game development process........................................................... 7
3.2 Bug and Issues categorization ....................................................... 9
3.3 Testing............................................................................................. 11
3.3.1 External Quality Assurance .................................................. 12
3.3.2 Internal Quality Assurance ................................................... 12
3.3.3 QA Tester ................................................................................. 14
3.4 Skill Requirements for QA ........................................................... 16
3.4.1 Communication ...................................................................... 16
3.4.2 Understanding of game mechanics ..................................... 16
4 Evaluation techniques and principles ............................................ 19
4.1 Evaluation by Heuristics .............................................................. 19
4.1.1 Playability Heuristics ............................................................. 19
4.1.2 Heuristics Evaluation process .............................................. 21
4.2.1 PLEX framework ........................................................................ 21
4.2.2 PLEX framework Evaluation process .................................. 23
4.3.1 User Testing ................................................................................ 24
4.3.1 Selected User Testing process ............................................... 25
5 Game Analysis .................................................................................... 26
5.1 SurvivAnt ....................................................................................... 26
5.1.1 Heuristics Evaluation ............................................................. 26
5.1.2 PLEX ......................................................................................... 27
5.1.3 User Testing ............................................................................. 28
5.2 Blind Man’s Buff ............................................................................ 29
5.2.1 Heuristics Evaluation ............................................................. 29
5.2.2 PLEX ......................................................................................... 30

1
 5.2.3 User Testing ............................................................................. 30
5.3 SwordMaster .................................................................................. 31
5.3.1 Heuristics Evaluation ............................................................. 31
5.3.2 PLEX ......................................................................................... 32
5.3.3 User Testing ............................................................................. 33
6 Conclusion ........................................................................................... 35
Literature ................................................................................................. 37
Appendix ................................................................................................. 40
Glossary ................................................................................................... 45

2
1 Introduction
Field of computer science has been evolving remarkable since the
first computer has been build. In the present there are countless
companies in software development industry. Creating a software is
not an easy way. However making sure that final product is working
according to the requirements of the consumer and respect all
constraints is even more challanging. Control of quality also known
as Quality Assurance has been introduced exactly for this purpose

In my master’s thesis I am describing what Quality Assurance is in

general in Chapter 2. Afterwards in Chapter 3 I narrow it into Quality
Assurance in Game Development including stages of Game
Development itself. Followed by explanation of possible issues
related to the game development and video games, what are the
different approches in testing as well as description of work, QA
tester as a role need to do. Furthermore in Chapter 4 I am introducing
three different techniques (principles) of Game Analysis including
Playability Heuristics, PLEX framework and User Testing. Practical
part of my work covered in Chapter 5 is analysis and evaluation of
three student games of different genre and platform using the
introduced evaluation techniques as well as my personal gameplay
testing. Analysis includes found issues and bugs as well as what are
the good (positive) aspects of each game. In Chapter 6 I suggest
improvements on game mechanics, design and other aspects based
on the found results.

This master’s work is based on my experiences and knowledge I

have got during studying at the University of Tampere as part of
my Erasmus+ study program. I have had many courses from Game
Studies, including Game Development process and also Quality
Assurance in Software Engineering. Further I was working in the
game studio Reborn (former Fragment Production) for almost three
months internship Tester and afterwards Internal Quality
Assurance Tester for additional five months. During this period I
got more familiar with the development process of video game.
What kind of issues can be found and how to reproduce them,
which is the most crucial part of the and later even fixing several
data or design related issues. I realized that processs of Quality
Assurance is not easy to understand for everyone and some parts
sounds way easier than they actually are. It’s not just simply

3
playing and enoying game for the whole day as many people think
that’s the main point.

4
2 Quality Assurance
The definition of Quality Assurance taken from [12]:

“Often used interchangeably with quality control (QC), it is a wider concept

that covers all policies and systematic activities implemented within a qual-
ity system. QA frameworks include:

(1) determination of adequate technical requirement of inputs and out-

puts,
(2) certification and rating of suppliers,
(3) testing of procured material for its conformance to established qual-
ity, performance, safety, and reliability standards,
(4) proper receipt, storage, and issue of material,
(5) audit of the process quality,
(6) evaluation of the process to establish required corrective response,
and
(7) audit of the final output for conformance to (a) technical (b) relia-
bility, (c) maintainability, and (d) performance requirements.”

Definition of Quality Assurance is very obscure as it covers all fields

of interest where we could call process of control as a part of whole
Quality Assurance process. Further we will focus only in Software
Quality Assurance (SQA) and in next chapter even more specifically
to Quality Assurance in Game Development in more detail.

2.1. Software Quality Assurance

In software development where result is a final product or service,
Quality Assurance is any standardized process of control which
makes certain that product or service that is in development is satis-
fying all specified requirements. Companies start to use SQA as sep-
arate unit in company or as part of the development departure to
increase the possibilities to find all defects. Nowadays SQA is more
important than in past as it’s used even throughout the development
and a deployment process as a way of preventing mistakes before
the final product or service has been made. Allowing to detect many
issues during the development process saves huge amounts of finan-
cial, human and time resources. Usually correcting issues in early
stages is much cheaper than when development process of product
or service is close to the end.

5
Companies uses SQA also as a system how to improve their credibil-
ity and work process with better scheduling and efficiency as well as
increasing customer confidence. Additionally increasing potential of
company to compete with others. [13]

6
3 Quality Assurance in Game development
In this chapter, we define process of game development, what are the
main requirements, issues and stages. Also, who the participants and
roles in this process are. To establish good understating and further
use all this information in introducing Quality Assurance integrated
in the process of development in all stages of game development.
Further what are the main problems related to the Quality Assur-
ance, how QA can be divided based on different perspective and ap-
proaches.

3.1 Game development process

Let us introduce all relevant roles in Game development process
which help us understand their responsibilities in further stages of
this process.

General roles in most of the games are Developers, Publisher/Distrib-

utor and Consumer. Besides these basic roles there can be many oth-
ers involved as can be seen in Figure 1 and even more. For the pur-
pose of this work we need to focus mainly on the Developers roles
within the development team.

Figure 1 Game Development Roles. [15]

7
Development team size also varies on game scale however even
small team can be behind bigger such as AA1 games.

Let us further briefly introduce all stages of the game development

process. Not all the companies have to follow this structure or
scheme but commonly known stages are Concept, Pre-production,
Production and Post-production (Post-Launch phase).

Concept as the starting point for the game is the idea what the game
is about. Nowadays many games are sort of duplicates of existing
concepts but with small changes in graphical or visual aspects of the
games or small twin in mechanics. Besides this duplicates many de-
velopers are trying come up with completely new concepts and gen-
res of games. Concept should contain genre, platform of the game,
what will be the core of the game and what mechanics are supported.
Additionally also marketing, business model, monetization plan and
target audience. Follow up are three main phases as seen in Figure 2.

Figure 2 Game Development Process. [16]

Pre-production is phase where all the fun begins. Also, it is very im-
portant phase as mostly it’s about planning. Changes in this phase
are crucial as in later phases performing critical changes in core of
game is very expensive or time consuming. Proper Pre-production
contains of Prototyping which can be performed as paper prototypes
or any toys or demo versions of game. Assuring that all mechanics
and gameplay works as intended even with temporary graphics or
messy code. Also Game Design Document2 and Technical Design
Document (TDD) – identifying technical challenges and plan of so-
lutions. Both documents are written to describe gameplay, visual,
technical, workflow, pipelines and other aspects of game in detail

1 The scale of games according to budget, goals set and final metacritic score. [14]
2 Document containing description of all elements of game. [17]
8
which are afterwards used in next phase as guidelines. On top of
these documents team selection and work scheduling. [15][18]

Production phase can be followed by different development ap-

proaches such as waterfall or agile. As main task in this phase is to
have First-Playable game which assures that all designed mechanics
and gameplay works as intended as a proof of concept. Together
with creating of all visual, graphical and audio assets and compo-
nents. Achieving this first milestone allow us to focus on next one.
Different conferences or events can be followed by the milestone.
Common milestones are Alpha and Beta version of the game contain-
ing all features and content preferably already tested to avoid critical
bugs and issues (further explained in follow section) finalized with
Gold-Master (GM) version as complete game product. GM version is
ready to be sent to press for reviews, to platform holders and distri-
bution. [15][18] For purpose of my thesis this is the crucial phase
where Quality Assurance takes the most important part to detect all
possible issues early enough.

Post-production as a final phase of game development is about mar-

keting, optimization, polishing visual aspects of game such as user-
interface and performance. After the game has been launched wider
information from consumers are usually posted in the game forums
and discussions which gives developers feedback on issues over-
looked during the testing.[15] Many companies are working on cor-
rections and patches/updates with fixes. Current trend is that even
after launch game is further developed and cycling back on previous
phases creating additional content providing replayability. Best ex-
ample are online games or MMORPGs where new content is con-
stantly delivered to players. (World of Warcraft, League of Legends,
etc.) [19]

3.2 Bug and Issues categorization

In this chapter, we describe what a bug is, general categorization of
issues, what types can be found with the certain approach to testing
and how impactful the issues can be.

Definitions of a bug related to the game industry can vary based on

the different sights of view. Let’s define it as “any behaviour of the game
that was not intended by the design or developers of the game.” General

9
categorization of bugs can be described depending on what the cause
of certain bug is or what effect it has on the gameplay.

Ruuska [20] has divided bugs into two main categories: Functional
and Localization issues. Each of these categories is split into subcat-
egories:

Functional – Performance, Graphics, Collision, Artificial Intelligence (AI),

Audio, Text, Camera, Balance and Design

Localization – Implementation, Linguistic, Audio and Other.

Detailed description related to each subcategory with possible causes

and detailed explanation of issues further in [20].

Other approach in categorization of issues is based on what role in

the team is responsible for its correction as followed: Design, Code,
Art, User Interface (UI) Design and Localization. This is the distribu-
tion we used in Reborn. See Figures 3 to 5 for examples of suchs
issues found in the game Urban Empire with the classification of the
issues.

Figure 3 Localisation Issue - Missing character. Image courtesy of Reborn Interactive. Used with
permission.

Figure 4 Code Issue - Missing premade roads on the loading of map. Image courtesy of Reborn In-
teractive. Used with permission.

10
Figure 5 Art Issues - Missing model of building. Image courtesy of Reborn Interactive. Used with
permission.

3.3 Testing
Testing is key activity of the Game Development. Hence testing
needs to be performed early enough and repetitively to prevent ex-
pensive refactoring at the later stages. Despite all its positives on the
final product testing is overlooked by many companies or just done
in small scale. In case of software and game development can be car-
ried out utilizing two main types of testing: automation and manual
testing. [21]

Automated testing is based on customized scripts and frameworks

for quick testing of simulations or Unit Testing – where we run a spe-
cifically made script to test behaviour of some type of AI to detect
problems related to it. Use of appropriate frameworks allow to con-
struct tests following parts of code, creating visual reports to display
outcomes as pass or fail. The potential of automated testing being
more efficient and faster is great. Nevertheless a transition from man-
ual to automated testing is not genuine as it seems on the first sight.
Because many observations, done by human testers, simply can’t be
replaced by automated tests. [22]

11
Our focus is on manual also known as playtesting in which testing
can be performed by simply playing the game however also certain
pattern and scenarios are followed. As the main part of playtesting
is Quality Assurance which can be further branched into Internal and
External Quality Assurance testing. [21]

3.3.1 External Quality Assurance

Not always a company have to have their own Quality Assurance
department – instead, they can use an independent QA company.
This sort of testing always include planning of what tests are going
to be performed with a primary specified goal. Such a testing can be
focus on hardware compatibility, performance evaluation or general
search for bugs and issues. External Quality Assurance usually have
fresh view on a game which is more beneficial allowing to find usa-
bility issues as well. External testing is usually done in sprints or pre-
defined periods of time and final results report is delivered after-
wards, containing list of all found issues with severity assigned to
them. Additionally, not every single version of game can be tested as
the distribution and communication between developing studio and
external company might overcome the budget.

3.3.2 Internal Quality Assurance

Internal QA tester is a member within the development team who
have access to the most recent version of game if not decided other-
wise. Giving QA tester ability to react to the actual state of the game
with changes done on the way by developers. Purpose of QA tester
is not only to find and report found issues but also to make sure that
applied fixes works as intended and doesn’t create even more issues.

Workflow for Internal QA is commonly defined within the company

itself and can be very different from others. Despite this fact some
similarities can be found between them. Tester is responsible for
looking for issues, creating reports eventually assuring that issues
will be assigned to responsible person to making it correct.

12
 Figure 6 Example of bug life cycle.

Above is example how can be stated bug lifecycle - workflow de-

cided in company how to proceed after discovering an issue.

1. QA tester finds a bug and create report

2. QA leader assigns report to leader of department responsible
for fixing bug or directly to responsible person
3. After developer fix issue, QA tester confirms correctness of
fix
4. If issue still persists tester assign issues back to QA leader
5. Otherwise close the issue

Above present workflow could be counted as ideal although in prac-

tice it’s not that simple. Problems related to the actual workflow in
the company depends on many factors. In case QA leader is not ded-
icated person within the team QA tester might struggle with assign-
ing reports to responsible developer especially if tester is new in

13
company or didn’t get proper instructions. On the other hand, prob-
lem can occur on the site of the developer leader who gets over-
whelmed with amount of reports and those will stagnate, gets lost in
tracking system or never reach person responsible for fixing issue.

3.3.3 QA Tester
Mentioned in the previous section one of main tasks of QA tester is
to look for issues and bugs. According to [23] we can determine two
leading manners how to detect bugs is positive and negative testing.
Positive approach looks at the game and confirms the basic function-
ality of various game aspects: doors opens correctly as well as re-
sponsibility of UI elements and their actions. On the other side, neg-
ative testing is being mean and trying to break game mechanics with
inappropriate behaviour: trying to walk through solid objects, spam-
ming keys. Both ways are very important and tester should not only
focus on one of them. Avoiding negative testing may allow consum-
ers to find exploits – glitch allowing player to get advantage over in-
tended gameplay also known as exploiting.

Second step is creating a report containing information of the bug in

most viable detail with attributes such as: Description, Priority/Se-
verity, Reproducibility, Reproduction steps, Game version and pos-
sibly others.

Description – detailed explanation of wrong behaviour of found bug

as well as what the expected behaviour usually is. Additionally at-
tachment with screenshots capturing the area of bug in-game, if pos-
sible.

Priority/ Severity – usually specified in the company guidelines or

TDD as a certain prioritization list with explanation of each level.

Priority Explanation
Blocker Prevents player from progression during gameplay
(crash, hang, blocked elements).
Critical Issue has big impact on gameplay even though progres-
sion is still possible (missing textures, game elements).
Major Has low impact on gameplay and doesn’t prevent
player from progression but still can cause confusion or
annoy player (unimportant NPC glitching in walls or
disappearing, wrong values are displayed in UI).

14
 Minor Small bug with no impact on gameplay, but still can be
spotted and frustrating player (NPC mount not moving,
overlapping objects).
Cos- Commonly connected to level design or graphics (tex-
metic ture errors, misplaced textures).
Table 1 Example of priority table.

Reproduction steps – precise steps how to achieve reported bug, con-

taining all necessary moves or actions to reach exact same state of
game as when the bug was discovered. Allowing developer respon-
sible for fixing to accomplish bug in need to fix it.

Reproducibility – How often reported bug occurs follow the repro-

duction steps. Taking into account that not always bug has to occur
every single time following steps. Usually marked as: Always, Rare,
Not reproducible.

Game Version – using version control and to assure that reported

bug is still relevant to the currently used game version.

Others – may alter according to company needs, using tags for easier
sorting of reported bugs, labels, etc.

As finding bug is usually the easy part as well as writing down the
description and expected behaviour, real challenge of the QA tester
comes to finding proper Reproduction steps and Reproducibility of
bug as without this information developer would have hard time
finding possible fix even in case of smaller game where code size is
still notable. Furthermore, when fix is applied and QA tester need to
verify correctness of fix. Usually first by following the Reproduction
steps. However the tester should try to reproduce the bug couple of
times because as mentioned before the bug doesn’t have to always
occur. Only after QA tester is sure that the found bug is fixed without
any other side effects a report should be closed.

Along with previously mention tasks, one of the most important re-
sponsibilities of QA tester is to assure that fixing one bug doesn’t cre-
ate many others – in this case trying to figure out better way how to
fix issues is preferable if cost for it is less than fixing all newly found
issues.

15
3.4 Skill Requirements for QA
Even though many issues and glitches are very easy to spot, certain
type of them is usually hard discover and even harder to reproduce.
That’s why QA tester requires a particular skill set to be able to detect
even the smallest bugs. Every person has different interests, skills
and all of their life experiences may be useful. In this section, differ-
ent skills essential for QA are explained.

3.4.1 Communication
As one of the most important skills QA tester needs to have in his
skill set to communicate within the team and being the connecting
part between artist, designers and coders. In comparison with de-
signers whose priority skill lays upon listening as written in [24], QA
testers needs to be able to express and talk about issues found. To be
able to communicate and understand what’s going on in game the
connection between designers and QA testers is very tight. Espe-
cially great communication within the team helps to avoid reporting
of issues which may be seen as bugs at the first sight by the QA tester,
but they are actually intended by the designer.

3.4.2 Understanding of game mechanics

To fully understand the meaning of this particular skill, first we need
to have understanding of what game mechanics are.

Game mechanics can be seen as a skeleton underneath the skin of the

game as core to interpret what a game truly is. “Game mechanics are
the core of what a game truly is. They are the interactions and relationships
that remain when all of the aesthetics, technology, and story are stripped
away“[24]. There isn’t one general taxonomy of them as even in very
simple and straightforward games, they tend to be considerably
complex. On one hand they are strictly defined set of rules for game
objects but on the other one they sort of contain something more
mysterious inside. And various researches can choose different tax-
onomies. However, I took categorization from [24] where game me-
chanics are split into six large categories: 1. Space, 2. Objects, Attrib-
utes, and States, 3. Actions, 4. Rules, 5. Skill, 6. Chance

16
 Space – is a determined area where gameplay of game is taking
place. This game mechanic defines individual places of game and re-
lations between them and can be seen as abstract contains of game
without any visuals or aesthetics.

Objects, Attributes, and States – space in the game cannot be

empty, that’s why we need to place various objects into it such as
characters, props, buildings. Each of these objects have set of attrib-
utes as some sort of information of the object and every attribute has
a present state. For example, an object “House” can have attribute
colour and state would be blue. Different attributes have distinct
amount of states related to them.

Actions – are a set of rules what player can do within the game
divided into two branches as operative actions - simple actions, driv-
ing car left or right. And resultant actions – meaning of these actions
is related to the bigger picture of the game and related to how player
wants to achieve a goal.

Rules – creating a connection between all previously mentioned

mechanics and are the most viable one. Set of constrains on the game
space, objects and actions allowing us to set the main part of the game
– the goals. There are many categories of rules based on specific
game, objects or even related to gameplay (Operational rules, Behav-
ioural rules, Laws, etc.)

Skill – This mechanic is taking the player as a target and not the
game itself. Different level of skill is required from the player to al-
low him to play the game. Also, we can categorize skills into physi-
cal, mental and social skills. Skill of the player should match the dif-
ficulty of the game for the best experience. Many games are easy to
learn but hard to master. Player acquires better skill to follow the in-
crease of difficulty.

Chance – considering interactions between all previously men-

tioned game mechanics, chance is giving uncertainty into the game.
Might be understood as the part of fun. Defining what are the prob-
abilities and expected values of other mechanics.

Discovering localization and visual bugs is easier and doesn’t require

understanding of game mechanics too much. On the other hand, this
skill is most crucial to discover and be able to detect most of the func-

17
tional and design issues. A QA tester needs to have general under-
standing of the game mechanics outlined above and how they are
represented in the tested game. Furthermore, the tester needs to have
a knowledge of each game mechanic category and theirs specifica-
tions. For example to detect not working button, tester needs to know
what are the constrains of button and what action is expected for this
button. In short understanding of game mechanics is ability of tester
to determine what the mechanics of the game are and what the de-
sired behaviour of them is.

18
4 Evaluation techniques and principles
In this chapter I introduce expert evaluation techniques such as
Playability Heuristics, PLEX-framework where in both certain patter
of evaluation is followed. Another technique which is not expert is
User Testing. Each of them will be further used in next chapter for
game analysis.

4.1 Evaluation by Heuristics

Is widely used expert technique for evaluation of Usability and
Playability of given software/games. But we focus on those used in
Game Development.

These Heuristics are specifically designed rules of what aspects

should be included in the game design as a simple method for
evaluation. Expert with experience in game compare and evaluate
various parts of game or a whole game with these heuristics. Results
are listed ina table of Heuristic Violations. A violation of any
heuristics can be also intentional when there is be good reason or
design feature for it.

Besides evaluation, heuristics can be used as a great tool while in the

process of designing a game to avoid problems in later stages of
development.

There are several different Heuristics that can be used for expert
evaluation but I have chosen Playability Heuristics [1] even though
these are specified for mobile games, I have experience with their use
and authors says we can use these specific heuristics for other
platforms as well.

4.1.1 Playability Heuristics

Heuristics are divided into three groups Usability, Mobility and
Gameplay. Besides these main categories we also added Multiplayer
category. See Tables 2 to 5. Since my analysis contains different genre
and platform games we need to establish all categories even though
we use specific ones for each game. Detailed explanation of these
heuristics can be found in [1].

No. Game Usability Heuristics

GU1 Audio-visual representation supports the game.
19
 GU2 Screen layout is efficient and visually pleasing.
GU3 Device UI and game UI are used for their own purpose.
GU4 Indicators are visible.
GU5 The player understands the terminology.
GU6 Navigation is consistent, logical, and minimalist.
GU7 Control keys are consistent and follow standard
conventions.
GU8 Game controls are convenient and flexible.
GU9 The game gives feedback on the player’s actions.
GU10 The player cannot make irreversible errors.
GU11 The player does not have to memorize things
unnecessarily.
GU12 The game contains help.
Table 2 The heuristics for evaluating game usability. [1]

No. Mobility Heuristics

MO1 The game and play sessions can be started quickly.
MO2 The game accommodates with the surroundings.
MO3 Interruptions are handled reasonably.
Table 3 The heuristics for evaluating mobility. [1]

No. Gameplay Heuristics

GP1 The game provides clear goals or supports player-created
goals.
GP2 The player sees the progress in the game and can compare
the results.
GP3 The players are rewarded and rewards are meaningful.
GP4 The player is in control.
GP5 Challenge, strategy, and pace are in balance.
GP6 The first-time experience is encouraging.
GP7 The game story supports the gameplay and is meaningful.
GP8 There are no repetitive or boring tasks.
GP9 The players can express themselves.
GP10 The game supports different playing styles.
GP11 The game does not stagnate.
GP12 The game is consistent.
GP13 The game uses orthogonal unit differentiation.
GP14 The player does not lose any hard-won possessions.
Table 4 The heuristics for evaluating gameplay. [1]

20
 No. Multi-player Heuristics
MP1 The game supports communication.
MP2 There are reasons to communicate.
MP3 The game helps the player to find other players and game
instances.
MP4 The game supports groups and communities.
MP5 The design minimizes deviant behaviour.
MP6 The design hides the effects of network.
Table 5 The heuristics for evaluating multi-player aspects.[3]

4.1.2 Heuristics Evaluation process

To use such evaluation technique expert need to have knowledge
about evaluated game. My evaluation process includes short period
of gameplay for each of selected games. Immediately writing down
notes when I saw any violation of any of introduced Playability
Heuristics.

As a final result of this evaluation method is Heuristics Violation

table which contain what heuristic was violated, what is a reason of
violation and severity rating, see Table 6.

Critical Crashes the game or makes it unplayable in some way.

Major Severely affects negatively on the play experience.
Minor Minor nuisance which has a small effect on the play
experience.
Cosmetic Small mishap or similar, no effect on the play
experience but stands out.
Table 6 Problem severity ratings.

4.2.1 PLEX framework

Playful experiences (PLEX) framework is another expert evaluation
technique. This framework is built as a follow up from others
theoretical work on game experiences, emotions, elements of play
and argumentation why people play games. [4]

With „Playful Experience‟ we mean experiences that are mostly non-goal-

oriented and mainly evoked by fun or the pleasurable aspects of using a
product. Playfulness may also include temporary negative experiences as
part of the overall experience. This paper presents an overview of the
Playful Experience (PLEX) framework, which defines 22 experience
categories as the components of Playful Experience.[5]

21
Based on PLEX framework other researches come up with practical
use, such as in design and evaluation related activities not only for
video games but even other fields of development to design for
playfulness.

For easier transmission of understanding 22 PLEX categories to

stakeholders and designers who want to design for playfulness,
Lucero and Arrasvuori had made [6] and evaluated [7] PLEX-cards.

Figure 7 Photo of PLEX cards – Photo author: John Paul Bichard–

http://www.funkydesignspaces.com/plex/

All of the 22 PLEX categories can be seen below in Table 7 with

brief description.

Experience Description
Captivation Forgetting one’s surroundings.
Challenge Testing abilities in a
demanding task.
Competition Contest with oneself or an
opponent.
Completion Finishing a major task, closure.
Control Dominating, commanding,
regulating.

22
 Cruelty Causing mental or physical
pain.
Discovery Finding something new or
unknown
Eroticism A sexually arousing experience.
Exploration Investigating an object or
situation.
Expression Manifesting oneself creatively.
Fantasy An imagined experience.
Fellowship Friendship, communality or
intimacy.
Humour Fun, joy, amusement, jokes,
gags.
Nurture Taking care of oneself or others.
Relaxation Relief from bodily or mental
work.
Sensation Excitement by stimulating
senses.
Simulation An imitation of everyday life.
Submission Being part of a larger structure.
Subversion Breaking social rules and
norms.
Suffering Experience of loss, frustration,
anger.
Sympathy Sharing emotional feelings.
Thrill Excitement derived from risk,
danger.
Table 7 PLEX framework consisting of 22 categories. [5]

4.2.2 PLEX framework Evaluation process

For PLEX-category testing, I played the games for a while again to
confirm or bust some memories about possible PLEX-functions, that
came to my mind while working on Heuristics Evaluation. Base idea
is to determine which PLEX categories are invoked in player while
playing game to get the most of playfulness.

To display results I will construct another table containing all PLEX

categories found in each of the played game. Table also includes
explanation where and why or what aspects of game are responsible
for certain experience.

23
As a human nature and personality results of this evaluation
depends on subjective experiences found by tester. That’s why this
type of evaluation is better as a group of experts rather than done
individually. Still it’s interesting evaluation technique and further
can be used as framework for designing games . That’s why I choose
it.

4.3.1 User Testing

Definition of Usability: “The extent to which a product can be used by
specified users to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use.” [9]

User Testing or also known as Usability Testing is another method

how to evaluate games to find possible usability issues. It’s the most
fundamental usability method where, players (testers) are
representing the target group or the final consumer. Similar to
previously introduced techniques User Testing narrows down the
issues and challenges within game itself, that were not directly
intended by game developers. [8] Additionally User Testing helps in
Game Development to game designers and developers see game
from other perspective as they can’t think of all possible ways player
will play their final product as well as they suffer “Developers
blindness”. That means that no matter how hard developers try there
always will be issues or bugs they can’t see simply because they
know too much about the design of game itself.

General requirements for User Testing are: Test Users, Determined

plan for play session, Session Evaluators, Evaluation of results. Let’s
take a look how to proceed in the setting up User Testing evaluation.
[11]
Important aspect before User Testing can be started is to state what
are the outcomes we expect and think about what we want to test.
Different approaches we can take are for example: what user like or
dislike, what they think after first-time experience, what are critical
points, crucial mechanics and design issues. After we establish
research questions we can proceed to selection of correct
methodology to get proper answers. [10] There are several
methodologies that we could choose from. Questionnaire and
interviews are the most common methods to gather results after play
session. Subsequently we need to determine the plan for Play Session

24
– which can be settled as laboratory session with strict rules for
session evaluators and for tester. Next step is to collect responses
from the Test User with already mentioned methodologies and last
step is to evaluate and consider the difference between each of Test
Users and see results.

As testers are selected individuals each of Test Users should be from

different demographic group as well as with various game
experiences so we are sure we covered all potential end users and
consumers into consideration of results.

4.3.1 Selected User Testing process

My selected methodology for user testing was individual Play
Session 1 on 1 with the Test User with duration around 30 minutes –
throughout the gameplay I was observing reactions and making
notes when different aspects of game very introduced to Test User or
when he faced certain issues. Play session was followed up by
interview where I was asking different questions depending on each
of selected games to get most of the information and responses from
Test User. Selection of Test Users was focus on diversity based on
users experience with games – stated as level of gamer (depending
on how often they play games), males and females as well as test
users from various age groups.

25
5 Game Analysis
As practical part I have used above introduced techniques of
Playability Heuristics, PLEX framework, User Testing and my
personal QA testing. Games has been made by students of Masaryk
University as part of the course PV255 Digital games. I have pick three
very different games to see how analysis depends on genre and
platform of each game. Each game have short introduction and
description of game followed with analysis itself.

Chosen games are:

 SurvivAnt – PC, RTS game

 Blind Man's Buff – PC, Online Multiplayer game
 SwordMaster – Android, RPG game

5.1 SurvivAnt
Authors: Ľuboš Hyžák, Pavel Kouřil, Jakub Medvecký-Heretik

Platform: PC

About: SurvivAnt provides a spin on traditional RTS games, based

on the idea of having indirect control of your units while also
combining different phases of a gameplay - gathering resources and
improving your units during daytime, and defending the anthill
from waves of enemies each night. [2]

5.1.1 Heuristics Evaluation

# Playability Problem Description Violated Problem
Heuristic Severity
1 Screen layout should be in GU2 Cosmetic
displayed with more contrast color.
2 Queen abilities are lacking GU4 Major
indicators when it’s possible to use
them and when they are on
cooldowns.
3 Indicator of ants carrying resources GU4 Minor
is never explained.

26
 4 Indicators – should be explained GU4 Minor
immediately and not require player
to see Help.
5 Lack of information of possibility to GU4 Major
click on anthill to show ant statistics
and description.
6 Placing pheromones above anthill GU4 Minor
is not possible without any
indication. Also indicator of radius
behaves a bit strange around
anthill.
7 Game is too easy at the beginning. GP5 Major
Leaving game just run without any
interation you an easly survive 2
nights.
8 Game is about repeating the same GP8 Minor
action of gathering recourse – but
this was violation on purpose of
design.
9 Player can’t customize or express GP9 Minor
himself other than way how he
selects bonus points.
10 Attacking enemies can get stucks GP12 Major
around anthill and just stay there
without any further actions. Makes
game very easy if this happens.
11 Game supports only win or lose GP14 Minor
end conditions. After lose all
upgrades are lost and new game
needs to be started.
Table 8 Table of violated heuristics for game SurvivAnt.

5.1.2 PLEX
# Category Explanation
1 Challenge Possible end of game is to win when surviving
throughout three nights or lose.
2 Completion Each day trying to gather as many resources as
possible.
3 Exploration As amount of recourcses is limited for each food
found, player need to explore whole map.
4 Nurture Being a queen of anthill and taking are of the rest
of ant as their leader.

27
Table 9 Table of found playful experiences from game SurvivAnt.

5.1.3 User Testing

Interview questions:

1. Was tutorial explanation of main goal and how to achieve it

adequate?
2. What was the first that you got interested in or got your atten-
tion? And why? (confusion of the game at the start, because
the lack of information in comparison of other games)
3. Which boost category did you selected first? Did you balance
the boosts or focus in specific category?
4. Have you discovered something that was wrong or you don’t
like about the game? (bugs, mechanics, anything) (nothing
special)

Observations from interviews:

Players without experience in the game had a problem with under-

standing the basic gameplay mechanics such as controlling ants with
the use of pheromones. Tutorial was text based with more text than
necessary and players intend to just quickly read through it which
was possible cause of confusion. However, explanation of range in-
dicator for pheromones is only in Help section, even though it sup-
posed to be in tutorial as one of main points. After proper explana-
tion or read through the Help, players were mostly focus on up-
grades which allowed them better resource gathering. However
main target were night shouts as every tester picked them to increase
chance to survive the night. Every tester had besides understanding
of pheromones mechanic, problem with different aspects of the
game. Such as responsiveness of the UI elements, UI feedback to
player and indicators explanation should be definitely prior objective
for improvements. Furthermore, players liked the simplified look
and changes in audio to invoke danger throughout the night but
calm background music during day time. Also, the autonomy of ants
moving around on their own and ability to remember where are the
resources even after night. Players’ progression in the game in-
creased significantly with more experience they got. With full under-
standing of game, it seems to be too easy to achieve the main goal.

28
5.2 Blind Man’s Buff
Authors: Jan Krejsa, Matouš Procházka, Adam Tuček

Platform: PC

About: Multi-player game based on traditional similarly named

game. Full of action and funny moments with catchy graphics,
chasing, and giving buff to others and various power ups. Collect
more point than the others to win. [2]

5.2.1 Heuristics Evaluation

# Playability Problem Description Violated Problem
Heuristic Severity
1 Missing indicator for the duration GU4 Major
of power-up effects.
2 Missing indicator when player can GU4 Minor
use clapping to get points.
3 Player can’t clearly understand GU5 Cosmetic
characters’ models’ name while
customizing it.
4 Jumping isn’t allowed when GU7 Minor
standing on the spot.
5 Missing description of power-ups GU11 Major
and explanation what are their
effects.
6 Help isn’t included in the game but GU12 Major
delivered as external document
which is not lacking a lot of
information.
7 Game requires all players in same MO1 Major
network or 3rd party programs for
launching, which requires time to
start the game.
8 No message is displayed when any MO3 Major
of the player is disconnected or
when host disconnects and game is
disbanded.
9 Chat could be available before the MP1 Major
game has started or even during
the game session.
10 Game is connected to the direct IP MP3 Major
adress so all players needs to

29
 connect to specific one while in
same network.
Table 10 Table of violated heuristics for game Blind Man's Buff.

5.2.2 PLEX
# Category Explanation
1 Challenge Give buff to others or trying to avoid getting it.
2 Competition Main principle of the game is to be the winner,
trying to earn more points than others.
3 Humour Graphical aspect of the game is very joyful and
characters’ models are also amusing.
4 Relaxation Game doesn’t require too much of thinking and
short game sessions are good for relief.
5 Simulation Game is based on actual similar game that can be
played outside with friends.
6 Suffering Not being able to give buff or catch other players
can lead to frustration.
7 Thrill Getting closer to the player with buff is
rewarded with more points per second.
Table 11 Table of found playful experiences from game Blind Man's Buff.

5.2.3 User Testing

Interview questions:

1. What was the first think that got your attention when playing?
2. How you feel about gameplay mechanics of the game such as
movement and control of characters?
3. How useful would be the ability to communicate with play-
ers?
4. How was the graphical representation of power-ups useful
for understanding of their effects?
5. Have you discovered something that was wrong or you don’t
like about the game? Perhaps do you have ideas for improve-
ments?

Observations from interviews:

Graphics and audio nicely fit into the theme of the game. Fun and
humour as main emotions are highly encouraged with colourful
map, customization of players’ character and sound effects during
gameplay. For players with less experience in video games had prob-

30
lems with camera and character movement however those more ex-
perienced didn’t. Nevertheless, all players agreed on lack of indica-
tor or aiming tool for passing the buff to the others or any explana-
tion how this mechanic works. Insufficient graphical representation
of a power-ups and missing description of them discourage players
to use them even though they are crucial part for the gameplay.
Every player got interested in the game and come up with ideas for
the improvements as they would like to play this game again. Add-
ing map for better orientation, various indicators for the incoming
noise direction or communication tool. Additionally, the main focus
should be to improve the set up for starting a game, better match-
making tool as it is online game to find other players easily. Simple
emotes could be nice feature if they cannot be used for the harass-
ment or violated against players.

5.3 SwordMaster
Authors: David Osička, Martin Příborský, Pavol Valovič

Platform: Android

About: RPG with interactive battles for mobile phone. Help Lancelot
clear castle of enemies and save the imprisoned king. During journey
search for hidden rooms implemented with dynamic lightning
collect items to defeat your enemies. Enjoy battle aspects such as
attack and blocking is simulated with finger movement on the touch
screen of your phone. [2]

5.3.1 Heuristics Evaluation

# Playability Problem Description Violated Problem
Heuristic Severity
1 Small glitches with dynamic light and GU2 Cosmetic
allignment of visual aspect.
2 Indicators in combat mode includes GU4 Major
not usable ones. Areas in which
player should interact should be
highlighted.
3 Indicator of current damage taken GU4 Minor
from enemie and attack power
should be visible when different
power ups afftect this attributes.

31
 4 Missing Pause menu option in Battle GU6 Major
screen, player cannot pause, reset or
leave to main menu until battle is
over.
5 Bloking attacks coming from the GU8 Minor
corners is a bit inonvenient and
player have to remember to swipe
only vertically or horizontally even
though sword graphics would
suggest a diagonal swipe.
6 While in Pause Menu, confirmation GU10 Minor
should be displayed if player clicks
on Main Menu option. Which can be
hit by accident.
7 “Game characters” is not proper GU12 Minor
name of UI element and it’s not
intuitive that this element contains
Help and description of power ups.
8 Help is available only from the main GU12 Major
menu and not when pausing the
game. Player needs to quit current
progression to be able to see the Help.
9 Customized battle system is GP8 Minor
repetitive but this is done by design.
10 Customization or personalization is GP9 Minor
not available to player.
11 Playing game by progression order of GP14 Major
levels gives player higher attack and
bonuses callected in previous levels.
However starting on specific level
doesn’t give player equal strenght
according to power ups. So after
returning to Main Menu, Reseting
level or dying all bonuses collected
are lost – to get same state player
needs to play from start again.
Table 12 Table of violated heuristics for game SwordMaster.

5.3.2 PLEX
# Category Explanation

32
 1 Challenge Reaching the goal by battles with enemies,
navigating throughout the varios level and
finding key and gate to next level.
2 Completion Saving the king as master goal of game.
3 Cruelty Fighting enemies in order to kill them. Walking
through mouses and killing them.
4 Discovery Finding main goal objects, hidden rooms and
power ups.
5 Fantasy Being prince in medievel castle trying to save the
king and his kingdom.
6 Humour Small jokes are visible throughout the whole
gameplay.
7 Suffering After dying and need to start from the beginnng.
Table 13 Table of found playful experiences from game SwordMaster.

5.3.3 User Testing

Interview questions:

1. Was battle system explained accordingly to your needs and

understanding
2. What was the first that you got interested in or draw your at-
tention? And why?
3. Did you understand all game object and their effect in game?
4. What you think about movement mechanics in game?
5. Have you discovered something that was wrong or you don’t
like about the game? (bugs, mechanics, anything)

Observations from interviews:

Even though the game contains practical tutorial for the battle sys-
tem, all players had a problem to understand how precisely they
need to perform actions in any battle. Every player died on the first
encounter with real enemy. Biggest issue was using the swipe move-
ment other directions than vertical and horizontal. Tutorial could be
simplified a bit more to give clear idea for the battle mechanics. De-
spite the fact every player got issues with battle system, they see it
very interesting giving the potential of the game. The division be-
tween combat and roaming around is done well. Additionally, dy-
namic lightning adds the element of surprise and curiosity. Move-
ment control didn’t take a long time to master just players needed to
get used to it. Background music helped the atmosphere and fit into
the game genre. The critical part is loading between battles as well as

33
new map area. Loading indicator is not responsive and players
thought the game froze. In couple of times the game crashed com-
pletely during the loading of new level. This should be prioritized to
improve as well as the description of the effects of collectable objects
is insufficient. Currently available only from Main Menu and even
hard to find. In general, players were really excited at the beginning
after mastering the combat mechanics, but some found fighting and
running around as too repetitive without any twist after certain
amount of time played.

34
6 Conclusion
The aim of my work was to introduce the area of Quality Assurance
in the relation with the Game Development. To describ the main
steps in QA process are and how we can use the knowledge and
understanding in the testing and quality control of the video games.
To fully understand the meaning of testing process I outline different
approches and type of issues we can face in this process.
Additionally, I narrowed what are the crucial requirements for QA
tester and how important acknowledgement of game mechanics is.

Furthermore I introduced two expert evaluation techniques

(evaluation by Heuristics and PLEX framework) for the game
analysis, the precise concent of them and how to perform the
evaluation based on these techniques. Moreover the introdution to
User Testing process. Including the exact stages of this process used
further in the practical part of the work.

Practical part of thesis was to evaluate three specific games with

expert evaluation techniques and User Testing. To show how useful
these methods are in detection of issues in the games. I evaluated
each game with different set of testers with variours skills and game
experience level to achieve diversity in the results.

Game Amount of violated Amount of playful

heuristics experienes
SurvivAnt 11 4
Blind Man’s 10 7
Buff
SwordMaster 11 7
Table 14 Result table for expert evaluations.

Based on the results from Table 14, we can see that each game had
almost same amount of the violated heuristics which most of them
were minor. However issues related to these violations could be
fixed or at least understood if such evaluation would be used in
earlier stages of the development allowing developers make
changes. And allow them easier tool for setting focus on different
aspects of game. Related to the amount of playful experiences the
more game contains the higher is change for player to enjoy the game
and increase the rate of replayability. Developers and especially
designers should use these methods to design various aspects of the
game to invoke specific playful experience. To better understand
35
what could player expect to feel while playing the game.
Nevertheless a balance between negative and possitive experience
should be kept.

User Testing results can be matched to the violated heuristics as with

the similar elements of the game players had most troubles. All
games had in general the lack of information to the player or
insufficient interpretation of the information. Various issues related
to feedback to player action or absence of indicators are suchs
example. Despite that players always found certain aspect of game
they like, supporting their excitment to the game. Furthermore this
method is useful not only for detection the problematic aspects of
game as feedback from players. But to allow players to propose
various improvements which can be further managed with
developers.

To wrap it up Quality Assurance is important part of software

development and even more in game industry. Testing and
evaluation allows to detect issues. If used early enough or throught
the whole process of development it could prevent occurance of
issues and bugs in final product and make sure all requirements of
consumers are met. Being able to detect and work on issues in the
early stages of development also saves the amount of resources
required for these fixes. Additionally all mentioned techniques are
good to use while designing a game as guidane tools and to prevent
issues from occuring.

36
Literature
[1] KORHONEN, Hannu a Elina M. I. KOIVISTO. Playability heuristics
for mobile games. In: Proceedings of the 8th conference on Human-
computer interaction with mobile devices and services - MobileHCI
'06 [online]. New York, New York, USA: ACM Press, 2006, s. 9- [cit.
2017-04-20]. DOI: 10.1145/1152215.1152218. ISBN 1595933905.

[2] CHMELÍK, Jiří. PV255 Vývoj digitálních [online]. [cit. 2017-04-20].

Available from: http://www.fi.muni.cz/~xchmeli1/PV255/projects.cgi
[3] KORHONEN, Hannu a Elina M. I. KOIVISTO. Playability heuristics
for mobile multi-player games. In: Proceedings of the 2nd international
conference on Digital interactive media in entertainment and arts -
DIMEA '07 [online]. New York, New York, USA: ACM Press, 2007, s.
28- [cit. 2017-04-20]. DOI: 10.1145/1306813.1306828. ISBN
9781595937087.
[4] LUCERO, Andrés, Jussi HOLOPAINEN, Elina OLLILA, Riku
SUOMELA a Evangelos KARAPANOS. The playful experiences (PLEX)
framework as a guide for expert evaluation. In: Proceedings of the 6th
International Conference on Designing Pleasurable Products and
Interfaces - DPPI '13 [online]. New York, New York, USA: ACM Press,
2013, s. 221- [cit. 2017-04-20]. DOI: 10.1145/2513506.2513530. ISBN
9781450321921.
[5] ARRASVUORI, J., M. BOBERG a H. KORHONEN. Understanding
Playfulness - An Overview of the Revised Playful Experience (PLEX)
Framework [online]. In: Proc. Design & Emotion 2010, Design and
Emotion Society, 2010 [cit. 2017-04-20].
[6] LUCERO, Andrés a Juha ARRASVUORI. PLEX Cards.
In: Proceedings of the 3rd International Conference on Fun and Games -
Fun and Games '10 [online]. New York, New York, USA: ACM Press,
2010, s. 28-37 [cit. 2017-04-20]. DOI: 10.1145/1823818.1823821. ISBN
9781605589077.

[7] LUCERO, Andrés a Juha ARRASVUORI. The PLEX Cards and its
techniques as sources of inspiration when designing for
playfulness. International Journal of Arts and Technology [online].
2013, 6(1), 22- [cit. 2017-04-20]. DOI: 10.1504/IJART.2013.050688.
ISSN 17548853.
[8] LAITINEN, Sauli. Better Games Through Usability Evaluation and
Testing. Gamasutra [online]. 2005, 2 [cit. 2017-04-20]. Available from:
http://www.gamasutra.com/view/feature/130745/better_games_through_us
ability_.php

37
[9] INTERNATIONAL ORGANIZATION FOR STANDARDIZATION.
"ISO, 9241-11: guidance on usability". Ergonomic requirements for office
work with visual display terminals [online]. [cit. 2017-04-20]. Available
from:http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_de-
tail.htm?csnumber=16883.
[10] BRYANT, Aidan. Everything You Need to Know about User Testing
Your Game [online]. In: User Testing Blog, 2015 [cit. 2017-04-20]. Avail-
able from: https://www.usertesting.com/blog/2015/02/05/user-testing-
games/
[11] MORENO-GER, Pablo, Javier TORRENTE, Yichuan Grace HSIEH
a William T. LESTER. Usability Testing for Serious Games: Making
Informed Design Decisions with User Data. Advances in Human-
Computer Interaction [online]. 2012, 2012, 1-13 [cit. 2017-04-20]. DOI:
10.1155/2012/369637. ISSN 16875893. Available from:
http://www.hindawi.com/journals/ahci/2012/369637/
[12] Quality assurance (QA) [online]. In: BusinessDictionary.com. Web-
Finance, Inc. [cit. 2017-04-21]. Available from: http://www.businessdic-
tionary.com/definition/quality-assurance-QA.html
[13] ROUSE, Margaret. Quality assurance (QA) [online]. In: Tech Target
Network, 2007 [cit. 2017-04-21]. Available from: http://searchsoftware-
quality.techtarget.com/definition/quality-assurance
[14] ABIZZEL1. Defining AAA, AA, A/B, and ? games. In: N4G
blog [online]. [cit. 2017-04-22]. Available from: http://n4g.com/user/blog-
post/abizzel1/520211
[15] RABIN, Steve. Game Production: Video Game Industry [online]. In:
[cit. 2017-04-22]. Available from: http://www.cs.ucr.edu/~vbz/produc-
tion.pdf
[16] Game Development Process [online] [cit. 2017-04-22]. Available
from: https://e-games.tech.purdue.edu/GameDevProcess.asp

[17] Game Design Document. In: Wikipedia: the free encyclopae-

dia [online]. San Francisco (CA): Wikimedia Foundation, 2001- [cit.
2017-04-22]. Available from: https://en.wikipedia.org/wiki/Game_de-
sign_document
[18] YAMAN, Baris. Game Production Stages. In: ETohum Game Devel-
opers Summit [online]. 2013 [cit. 2017-04-22]. Available from:
https://www.slideshare.net/barisyaman/e-tohum-game-production-stages-
2013-11-public-28639975
[19] BETHKE, Erik. Game development and production. Plano, Tex.:
Wordware Pub., c2003. ISBN 1556229518.

38
[20] RUUSKA, Essi. QUALITY ASSURANCE TESTING IN VIDEO
GAMES. Bachelor’s thesis [online]. 2015, 13-19 [cit. 2017-04-23]. Avail-
able from: https://www.theseus.fi/bitstream/han-
dle/10024/94503/Ruuska_Essi.pdf?sequence=1
[21] Game Development Workflow Part 2: Production and Post-Produc-
tion. In: KinematicSoup Technologies Inc. [online]. 2016 [cit. 2017-04-
23]. Available from: http://www.kinemat-
icsoup.com/news/2016/11/10/game-development-workflow-part2
[22] Automated QA Testing at Electronic Arts. Communications of the
ACM, 2014, (vol. 57, no. 7), 50-57. DOI: 10.1145/2617754.
[23] UNSUNG HEROES OF THE GAMES INDUSTRY: QA TESTERS.
In: IGN US [online]. 2017 [cit. 2017-04-23]. Available from:
http://www.ign.com/articles/2017/02/01/unsung-heroes-of-the-games-in-
dustry-qa-testers
[24] SCHELL, Jesse. The art of game design: a book of lenses. Burlington:
Morgan Kaufmann, c2008. ISBN 9780123694966.

39
Appendix
Full interviews for each game:

SurvivAnt:
Testee # Gender Gamer
1. Male 3/10
2. Female 3/10
3. Male 10/10

Was tutorial explanation of main goal and how to achieve it ade-

quate?

1. Yes and no, the goal explanation was straightforward but the
way how achieve it was not. I needed to read help to fully un-
derstand how to use pheromones and control the ants.
2. Bit of confusion at the beginning of the game as I don’t like a
lot of text to read so I possibly skipped some relevant part
about controlling the ants. I didn’t know how to use phero-
mones.
3. I have played the game before so the understanding of goal
and how to achieve it was simple for me.

What was the first that you got interested in or got your attention?
And why?

1. I got confused how to even play the game, as there was a lack
of information in comparison of other games I played. I
needed to read help to fully understand how to use phero-
mones and control the ants.
2. I really liked the simplified look of the game and the back-
ground music that changed according to the day (relaxing)
and night (more aggressive as it’s attack time)
3. The autonomy of the ants, as they remember the found re-
source and return to it after the night pass.

Which boost category did you selected first? Did you balance the
boosts or focus in specific category? (First improving ants to collect
more resource

40
 1. At first my focus was on improved collecting of resources and
afterwards I spend most resource in the shouts for better de-
fence during night.
2. I don’t remember what my first category was but mostly I
bought shouts to protect myself during the night fight.
3. First choice was production of resources, when I wanted to
attack enemies during day I upgraded ants’ attributes but also
bought few shouts to secure surviving the night.

Have you discovered something that was confusing or you do not

like about the game? (Bugs, mechanics, anything)

1. I didn’t like that I needed to read help before I was able to

understand how to play the game. Also I didn’t know when
I can use the shouts again as they had no visible cooldowns.
2. It took me a while to discover that I can move the camera and
discover whole area to play.
3. The struggle with UI responsiveness when I bought some up-
grade that didn’t activate but I lost the resources according to
its cost anyway.

Blind Man’s Buff:

Testee # Gender Gamer
1. Male 7/10
2. Male 9.5/10
3. Male 5/10
4. Female 3/10

What was the first think that got your attention when playing?

1. Graphical aspects of the game are really nice.

2. Humorous and funny sound effects were really thematic.
3. Beautiful graphics as they nicely fit in the game. Furthermore
I actually think game was really nice as I didn’t expect much
out of student game.
4. I really liked how colourful the game was. Graphics invoked
the childhood times in me.

How you feel about gameplay mechanics of the game such as

movement and control of characters?

41
 1. Movement was good but giving buff to other players was
pretty confusing.
2. Range indicator for giving buff to others as it’s hard to aim
without it.
3. I had no idea how to give the buff to other player so I was just
running away.
4. I was confused with the camera control and movement espe-
cially when close to the objects and edges of map.

How useful would be the ability to communicate with players?

1. Chat in the lobby would be nice or simple emotes during the

gameplay.
2. In my opinion it should contain chat or push to talk as game
is online and we don’t need to be in same room.
3. Only in the lobby and after the end of round as during game
chat seems a bit useless to me.
4. I think it would be a really nice feature at least before the
match starts otherwise players need to use other communica-
tion tools.

How was the graphical representation of power-ups useful for un-

derstanding of their effects?

1. Lightning was pretty clear and trap after a while as well but I
had no idea what the last one does.
2. Besides the creature power up, I knew all of them.
3. No idea what’s the red one but others were quite easy to un-
derstand.
4. For example the lightning was quite obvious but about the
others I didn’t understand their effect until many tries to fig-
ure it out.

Have you discovered something that was wrong or you don’t like
about the game? Perhaps do you have ideas for improvements?

1. As its online game I would like to have some kind of match-

making or finding the nearly players. And when someone
used clapping I needed to be already looking at his direction
to be able to see him, possibly an arrow or indicator of direc-
tion from where the sound came.
2. Easier setting up would be nice as its online game and proba-
bly some stats and leader boards. Besides that a more maps

42
 would be nice improvement. And I think it would be also nice
idea to play it on mobile phone instead of the PC.
3. Mechanics of giving the buff to others was not really clear as
well as the camera movement was a bit too fast. And good
improvement would be a map with indicators or noise or
clapping for better navigation.
4. Sometimes I got stuck under the bridges. In my opinion a map
would be good feature. To know where I am as being close to
the edges of arena, the camera behaviour is confusing.

SwordMaster:

Testee # Gender Gamer

1. Male 6/10
2. Female 8/10
3. Female 5/10

What is your opinion on the battle system explanation according

to your needs and understanding?

1. Not adequate, it could be because I just quickly read through

the battle tutorial. But I didn’t understand how the battle
system works.
2. Understanding of tutorial was good but a bit confusing as it’s
just at the beginning and it was too much information at the
same time with colouring of the areas where to attack.
3. Only that it should be mentioned that you can’t swipe in the
diagonal way even its intuitive especially for phone.

What was the first thing that got your interest or draw your
attention? And why?

1. Battle system,as it was something new.

2. Graphical representation and dynamic lightning that was
hiding the enemies and objects.
3. Battle system as it’s personal and division between roaming
around and battle.

Did you understand all game objects and their effect in game?

1. Not clearly, as there was no text in the game. Only key but
others not at all.

43
 2. I have seen few items but I had no idea what is their effect
besides key.
3. I didn’t understand and almost killed myself with the poison
as it was red and I expected to heal me. However key was
intuitive.

4. What you think about movement mechanics in game?

1. Was ok, I am used to play games with similar movements.

2. At the beginning I was not sure if I need to swipe or tap or
simulate joystick movement, after a while I got it. But in
general it was good and new.
3. For me it was quite intuitive and easy to control the
movement.

5. Have you discovered something that was wrong or you don’t

like about the game? (bugs, mechanics, anything)

1. Game was quite same all the time and repetition of the same
action was making it boring after a while. Just fight, find the
key, move to other level.
2. The loading time of the battle was a bit too long and I didn’t
know if it’s loading or just froze. As improvement when
collecting a poison it would nice that screen flash red as when
you are hurt in the battle.
3. My game crashed when it was loading the new level. I didn’t
even know there is a help to the items, it definitely need to be
in the game not just hidden in the main menu.

44
Glossary
QA Quality Assurance

PLEX Playful Experience

UI User Interface

NPC Non-player character

RPG Role Play Game

RTS Real-time strategy

MMORPGs Massively-Multiplayer Online

Role-Playing Games

45