The Quantitative-Qualitative Research Dichotomy Revisited
The Quantitative-Qualitative Research Dichotomy Revisited
The Quantitative-Qualitative Research Dichotomy Revisited
COMMENTARY
H. KANIS*
1. Introduction
Hignett and Wilson (2004) present a case study into how qualitative methodology
can be used in ergonomics, as an incentive to explore theoretical issues. This explora-
tion consists of presumptions and views both at the start of the study and evolving
from the study. In addition, the authors set out to demonstrate the application of
a qualitative method, i.e. semi-structured interviewing on the basis of a staged
sampling procedure.
I will discuss some of the theoretical issues raised by the authors, as well as the
empirical study with respect to both the way in which it is carried out and its results.
My commentary is concerned with the extent to which ergonomics as a scientific
endeavour may support usage oriented product design. Here, usage is seen as the
arena for the successful design of products as effective, efficient and satisfactory
means to support, protect, replace and extend human activities.
I depart from the definition of ergonomics adopted by the authors, that is:
ergonomics concerned with ‘. . . the understanding of interactions among humans
and other elements of a system’ and the application of ‘. . . theoretical principles, data
and methods to design in order to optimise human well-being and overall system
performance’ (IEA, 2000). I will narrow down ‘the understanding of interactions’ to
insight into user-product interaction at the so-called ‘micro-level’ (rather than on a
‘meso-’ or ‘macro-level’). The micro-level is referred to by the authors as ‘. . . the
immediate setting in which the person was situated’, and ‘. . . the direct point of
contact to emphasise the individual nature of each interaction.’; interaction in
small groups is seen as the meso-level, while an organisational or population level
is indicated as being macro by the authors.
*e-mail: h.kanis@io.tudelft.nl
Theoretical Issues in Ergonomics Science ISSN 1463–922X print/ISSN 1464–536X online # 2004 Taylor & Francis Ltd
http://www.tandf.co.uk/journals
DOI 10.1080/1463922041233130303418
508 H. Kanis
2.4.1 Specifics not predictable from a generality: It may be argued that these quotes
from the Hignett-Wilson paper (see above) appear to be at odds with each other.
What matters here is how research can support usage oriented design. Our empirical
studies show that user activities i.e., perception, cognition, action including experi-
enced effort (see figure 1 in the present paper) tend to be largely unpredictable on the
basis of theoretical considerations (Kanis 1998). As it happens, actual usage of newly
designed products often comes as a surprise to designers. In many cases this surprise
is an unwelcome one, as unanticipated users’ operations deteriorate designed func-
tionalities or may lead to accidents.
510 H. Kanis
The unpredictability of user activities, however, does not mean that there are no
general mechanisms in the emerging of these activities. For example, the tendency to
adopt a low level of cognitive involvement as the default (i.e. automatic actions, or
skill-based in terms of Rasmussen’s model (1986)), the occurrence of habituation
(Lambert 2002), and the fixation of users trapped by their experience in terms of
situated knowledge (Standaert and Christiaans 2000). Nonetheless, to paraphrase
Garfinkel (1991), actual user activities ‘cannot be recovered by attempts, no matter
how thoughtful, to specify an examinable practice by detailing a generality.’ In other
words: on the basis of generic insights such as indicated above (i.e. low cognitive
involvement, habituation, fixation) one will never be able to specify the situatedness
of actual usage in context. Here, the term ‘situatedness’ is adopted from Suchman
(1987), indicating that user activities are prompted by what people encounter from
moment to moment in a partly self-made situation, rather than user activities emer-
ging as invariant structures applying across situations (see Suchman 1987: 67).
Similar to theoretical preconceptions, notions like ‘plan’ or ‘task’ can possibly pre-
dict tendencies in human behaviour (cf. Activity Theory in Nardi 1996). However,
such notions (let alone a ‘universal purpose’) appear to be largely non-effective for
anticipating actual user activities. Here, another observation of Garfinkel seems to
apply, i.e., his description of people as rule users, constantly trying to work out how
to match the specificities of particular situations to whatever general orientation
(quoted in Murphy et al. 1998: 53).
2.4.2 Usage oriented design in need of anticipatory research methods: What do the
observations in the previous paragraph mean in terms of ‘going general versus dig-
ging on the spot’?
First of all they mean that designers are in need of anticipatory methods which
can be readily applied during a design process, think of the use of early models or
prototypes in exploring future usage (Rooden 2001). The application of such antici-
patory methods should result in measurements and observations consisting of user’s
activities linked to featural (form) and functional characteristics of the particular
product to be (re)designed, since these characteristics are, ultimately, what product
design is about. Questions about future usage which a designer faces, in the end
always tend to be specific regarding the product and contextual regarding possible
environments, with an eye for the diverse individual activities of future users. There
is no such thing as the design of a ‘general product’, the functioning of which would
be invariant for diverging user operations under various circumstances. People do
not end up in hospital due to normal’ or ‘average’ accidents.
Ergonomics involving the optimisation ‘of human well-being and overall system
performance’ (see IEA-definition above) has never been keen on providing designers
with the insights or tools to cope with the situatedness of product usage (Green 2002).
The intended subservience of ergonomics to design seems to have largely been
restricted to the production of summative measures such as means and standard
deviations for properties, capacities and limitations of human beings, the number of
usability problems (or ‘errors’) and performance times. Such summative measures do
not concern actual usage, or only indirectly. In addition, cases ‘ironed out to blurred
averages’, as Murphy et al. (1998: 104) put it, will hardly be of help for designers
facing future usage as the ultimate arena, see Kanis et al. (1999) for examples of
empirical studies (one of these examples is summarised below, see 4.3.1). This arena
of future usage will inevitably be situated, and conditioned by the individual predis-
Quantitative-qualitative research dichotomy 511
positions of users. Such considerations make a strong case for doing qualitative
research.
seen by Seale (1999: 86). Were there any results of maximising the possibility of
finding disconfirming evidence, as Murphy et al. put it (1998: 97)?
4. Results
4.1. A representational model
The interpretation of the hierarchically coded data results in a so-called representa-
tional model of the role of qualitative methodology in ergonomics. This model seems
rather general (Hignett and Wilson 2004: figure 6). Does it exclude anything? Nega-
tive cases have been included it seems, so some falsification of previous versions
might have occurred. It may be wondered whether the conclusion that a ‘middle
ground position seems to be appropriate for ergonomics’ can be linked to the model,
on the basis of evidence from the interview data.
4.3.2 The design relevance of quantitative data (an example): At the request of a
car company, we studied feet movements in the footwell space of a car in order to
establish whether these movements would be constrained by the interior space and
lay-out (Peijs et al. 1999). The available footwell space in cars tends to become more
and more constricted due to safety considerations, costs, design trends etc.
Feet movements of participants (n ¼ 9) were video-recorded by their projection
on a field of checkered gridlines placed on the bottom of the footwell space. For each
participant the positions of the feet in a 55 minutes drive (about 40 minutes in the
city, the rest on the highway) were traced in order to make up the total area occupied
by the shoe sole per individual. In addition to these registrations, participants were
interviewed, during the drive and in a retrospective interview, about their experiences
regarding the footwell space.
On the basis of regression analyses, it is shown that drivers use more space the
larger their feet (see figure 2), but proportionally less space (see figure 3).
It may be argued that the pattern in figure 3 for the brake pedal simply represents
natural behaviour, rather than operations being constrained in space by the presence
of adjacent pedals. This explanation is not confirmed by the way the foot support is
used (at the left side of the car, freely accessible). For this support, the regression
coefficient resulting from the same analysis as presented in figure 2 for the break
Figure 2. Right foot operating the brake pedal: relationship shoe sole area and area used.
Quantitative-qualitative research dichotomy 515
Figure 3. Right foot operating the brake pedal: relationship shoe sole area and the ratio
area used/shoe sole area.
pedal amounts to 1.98 ( p ¼ 0.0002) with R2 ¼ 0.90, while the same regression as
presented in figure 3 for the break pedal results for the foor support in a statistically
non-significant coefficient of 0.11 ( p ¼ 0.5, with R2 ¼ 0.07), which means that the line
of regression for the foot support can be seen as running horizontally (‘in terms of’
figure 3).
For the car company these outcomes signified that, in the design of the footwell
space of cars, limits were being approached or even overstepped. There would have
been no reason for this conclusion if the only data available had been the reactions of
the participants in the interviews: none indicated any problems such as discomfort,
or could recall mistakes, i.e., operating the wrong pedal.
The lack of concordance between the qualitative data and the results of the
quantitative analysis may have an obvious reason: the fact that, the bigger people
are, the more they are used to having proportionally less room available. This
experience will have set reference points. Seen in this way, interviewing participants
has not produced wrong or biased answers. It is the quantitative analysis which in
this case yields a deeper insight.
Note that the quantitative analysis is based on the individuality of participants,
captured in shoe sole area and used area, in situ. This example shows that there is
nothing in the quantitativeness (of the data, of the analysis) detracting from the
design-relevance of the study. On the contrary! It would have been detrimental to
sacrifice the ‘richness’ of cases in terms of situated individuality to summative
descriptions such as averages and standard deviations, blurring interindividual
differences and being contextually stripped.
From a usage oriented design perspective it is not only the type of data which
determines their significance, but also the way in which the data are treated. The
readiness of quantitative data to being processed summatively may satisfy require-
ments for publication which are still dominant in the literature (cf. Green 2002).
However, the quantitative character of data per se, i.e., as countings of measurement
units constituting unambiguous and meaningful scales, cannot be blamed for this.
Qualitative data has no inherent claim to superiority.
516 H. Kanis
References
CROTTY, M. 1998, The Foundations of Social Research (London: Sage).
GARFINKEL, H. 1991, Respecification [. . .], in G. Button (ed), Ethnomethodology and the
Human Sciences (Cambridge: Cambridge University Press), 10–19.
GREEN, W. S. 2002, Ergonomics and the design of consumer products: moving beyond retro-
active fixes, Proceedings of the Australian Ergonomics Society, Melbourne (in press).
HIGNETT, S. and WILSON, J. R. 2004, The role for qualitative methodology in ergonomics: a
case study to explore theoretical issues, Theoretical Issues in Ergonomics Science, 5, 473–
493.
HIGNETT, S. 2001, Embedding ergonomics in hospital culture: top-down and bottom-up stra-
tegies, Applied Ergonomics, 32, 61–69.
IEA 2000, International Ergonomics Association, Triennial Report (Santa Monica, CA: IEA
Press).
KANIS, H. 1998, Usage centred research for everyday product design, Applied Ergonomics, 29,
75–82.
KANIS, H., WEEGELS, M. F. and STEENBEKKERS, L. P. A. 1999, The uninformativeness of
quantitative research for usability focused design of consumer products, Proceedings
of the Human Factors and Ergonomics Annual Meeting (Santa Monica, CA), 401–405.
LAMBERT, J. V. 2002, Designing more habituation-resistent warning systems: Sokolov’s cor-
tical model, Contemporary Ergonomics, Proceedings of the Ergonomics Society (London:
Taylor & Francis), 437–441.
MURPHY, E., DINGWALL, R., GREATBACH, D., PARKER, S. and WATSON, 1998, Qualitative
research methods in health technology assessment: a review of the literature (Health
Technology Assessment, http://www.soton.ac.uk/hts).
NARDI, B. A. 1996, Context and Consciousness (Cambridge, MA: MIT Press).
NEISSER, U. 1976, Cognition and reality (W.H. Freeman).
PEIJS, S., GREEN, W. S., KANIS, H. and MAARTENSE, J. 1999, Footwell Space, Automotive
Ergonomics and Safety, Proceedings of 32nd ISATA Conference (Oxford, UK), 171–
178.
RASMUSSEN, J. 1986, Information processing and Human-Machine Interaction: An Approach to
Cognitive Engineering (Amsterdam: North Holland).
ROODEN, M. J. 2001, Design models for anticipating future usage, Thesis (Delft University of
Technology).
SCHWANDT, T. A. 2000, Three Epistimological Stances for Qualitative Inquiry, in N. Denzin
and Y. S. Lincoln (eds), Handbook of Qualitative Research (Thousand Oaks, CA: Sage).
SEALE, C. F. 1999, The Quality of Qualitative Research (London: Sage).
STANDAERT, A. and CHRISTIAANS, H. 2000, Does fixation interfere with product usability?
Proceedings of the IEA/HFES Congress (Santa Monica, CA), 6-81–6-84.
SUCHMAN, L. 1987, Plans and Situated Actions: The Problem of Human-Machine
Communication (Cambridge: Cambridge University Press).
WEEGELS, M. F. 1996, Accidents involving consummer products, Thesis (Delft University of
Technology).
WEEGELS, M. F. and KANIS, H. 1998, Misconceptions of everyday accidents, Ergonomics in
Design, 6, 11–17.