See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/265586291

Activity Analysis of Expert and Novice Operators in a Semi-Automated

Manufacturing Process

Conference Paper · September 2014

DOI: 10.1145/2637248.2637271

CITATIONS READS
4 251

3 authors, including:

Camilo Charron Franck Mars

Rennes 2 University Laboratoire des Sciences du Numérique de Nantes
61 PUBLICATIONS 406 CITATIONS 114 PUBLICATIONS 2,047 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Franck Mars on 22 January 2016.

The user has requested enhancement of the downloaded file.

 Activity Analysis of Expert and Novice Operators in a
Semi-Automated Manufacturing Process
Marie Hoarau Camilo Charron Franck Mars
IRCCyN (UMR CNRS 6597), IRCCyN (UMR CNRS 6597), IRCCyN (UMR CNRS 6597),
École Centrale de Nantes, University of Rennes 2 CNRS,
1 rue de la Noë, 44321 Nantes, 1 rue de la Noë, 44321 Nantes, 1 rue de la Noë, 44321 Nantes,
France. France. France.
Marie.Hoarau@irccyn.ec- camilo.charron@uhb.fr franck.mars@irccyn.ec-
nantes.fr nantes.fr
ABSTRACT representation of the current situation. He cannot know
Designing a human-machine interface for a manufacturing everything about the situation; nor can he completely
process requires a good knowledge of both the work master the situation [4].
domain and the operators’ representation. Ecological
Interface Design (EID) offers some interesting tools that This work focuses on an activity analysis of operators in
can be of help in the design process. The literature on such a situation. More specifically, the case study involves
cognitive control also offers a good understanding of a frame stretching activity in an aeronautical factory. This
operators’ cognitive resources. Analysing the activity of analysis aims to make recommendations for the design of a
both expert and novice operators through these two human-machine device that could support operators during
frameworks may help us to better understand the decision making and process control.
differences between them. A three-step protocol was We will first briefly describe the two frameworks we used
followed: 1. the elaboration of a means-end hierarchy, 2. to describe the system and the operators’ activities, i.e.,
the extraction of schemes via interviews, and 3. the Ecological Interface Design (EID) [3] and cognitive control
evaluation of the behavioural manifestation of schemes. In [4]. We will then describe how we operationalized them
the present case study, interviews revealed that both the with regard to the case study. Finally, our results will be
novice and expert operators of a manufacturing process presented and discussed.
shared a representation of the global process. However, in
DESIGNING A HUMAN-MACHINE DEVICE TO SUPPORT
contrast with the expert operators, the novice operators did
ACTIVITY IN A DYNAMIC SITUATION
not develop an operative scheme that related to the
machine. The results will be used as a basis for the design Ecological Interface Design
of a human-machine interface that will aid them to do so. Ecological Interface Design (EID) aims to design easy-to-
use interfaces for both experts and novices. It considers that
Author Keywords
people’s capability to use perceptive treatment is a strength
Task analysis; cognitive control; expertise; interface design;
that interfaces should rely on. To give people the right
aeronautical factory.
affordance, the interface should represent the system’s
ACM Classification Keywords whole complexity, but also trigger expert shortcuts [1].
H.1.2 User/Machine Systems, H.5.2 User Interfaces
Abstraction Hierarchy
INTRODUCTION A means-end hierarchy is a tool for representing the
In a work context, a dynamic situation refers to complex system’s complexity as a whole. It links goals and the
tasks in which there are changing parameters over time, as means to achieve them using a five-level hierarchy,
well as uncertainties [4]. The operator must then choose his comprising functional goals, abstract functions, general
actions according to his goals and his knowledge or functions, physical process and physical forms [9].
Permission to make digital or hard copies of part or all of this work for Whereas traditional interfaces mainly focus on lower level,
personal or classroom use is granted without fee provided that copies are ecological interfaces also represent the higher levels and the
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for
connection between the details of the hierarchy [1].
components of this work owned by others than ACM must be honored. Skill-Rules-Knowledge (SRK) taxonomy
Abstracting with credit is permitted. To copy otherwise, to republish, to
post on servers, or to redistribute to lists, requires prior specific permission The second most important element is to convey any expert
and/or a fee. Request permissions from permissions@acm.org. shortcuts. Whereas a novice will sequentially inspect every
piece of information, an expert may create rules that allow
ECCE ’14, September 01 – 03 2014, Vienna, Austria him to directly link what he perceives with what he has to
Copyright is held by authors. Publication Rights Licensed to ACM.
do. In reality, three levels of behaviour can be identified
through the SRK taxonomy; these range from the more them was a student. On the final day, we observed one
symbolic to the less symbolic, i.e., from knowledge-based operator who no longer worked on this task but was here
behaviour and rules-based behaviour to skills-based when the programs were made. We have called him a
behaviour [6]. A proper interface should allow the user to “mid-expert”.
work within each subsequent level, and thus, include any
RESULTS
relevant symbols, signs and signals.
Workshop and Task Descriptions
Cognitive Control
Workshop Description
Definition The objective of the workshop under discussion is to bend
Whereas the former framework only considers what we straight aluminium frames. These frames are U or T shaped
identified on a symbolic/sub-symbolic dimension, cognitive in section, and can measure up to 10 meters long. They will
control may include a second aspect: the origin of data. eventually form the skeleton for planes.
Indeed, the subjects can support their action with external
or internal information. Cognitive control may be To achieve this goal, frames go through many
considered as the articulation of the two dimensions [4]. transformations: they are bent and stretched twice and
undergo two thermic treatments. These phases are
At any moment of his activity, in order to master the automatized but controlled by different operators. To reach
situation, the operator has to find a balance between the the exact required dimension, the frame is then manually
cognitive cost and an acceptable level of performance [2]. calibrated, trimmed by an automaton and manually
The cognitive compromise, headed by meta-knowledge, calibrated again. These manual phases are the longest and
will allocate the proper resources via cognitive control to most troublesome according to the operators; they are also
achieve the desired performance. the most expensive.
Schemes
Both the management and operators believe that an
The operator has access to resources that include helpful
improvement in the automated stretching and bending
elements from the environment. He also has internal
process could limit the calibration tasks.
cognitive resources, including representation. Among these
internal resources are schemes [7]. Stretching and Bending Task
In this task, two operators work together. A support
A scheme is an invariant organization of one’s activity in a
operator only helps with the handling of the frames; it is the
class of situations. It is made up of five elements: 1. rules
pilot’s job to control the process and the machine. He
for action; 2. information that people consider to be true or checks the frame tracking sheet for the references and
pertinent regarding their activity, i.e., operative invariants; correct stretching program, and the clamps and tools. He
3. goals or expected results of the activity, i.e., anticipation; also loads the program and can modify it, although the
4. elements to modulate the activity, i.e., inferences, and 5. latter is forbidden. The operators are also supposed to check
tools, i.e., artefacts [8]. the dimensions of the frame after each stretching process.
METHOD
Physical Process and Means-End Hierarchy
In order to collect the different information needed to use The objective of the operators engaged in the stretching and
both frameworks, we created a three-phase protocol. bending task is to achieve the exact dimension in terms of
The first phase allowed us to elaborate a means-end stretching, bending, planarity and straightness. Moreover,
hierarchy, as described in the EID framework [1], through the frame has to meet some quality standard concerning the
global observations, documentation analysis, and interviews final material state. These elements constitute the first level
with researchers in physics. of the means-end hierarchy (cf. figure 1, a).
A second phase was aimed at picking up the explicit We considered that the physical laws that determine the
representations of the people working on this task [4]. Thus, process represent the abstract function level. These laws
we interviewed two expert and two novice operators, and an actually determine the results of the stretching. We
industrial technician. extracted five elements for this level in consultation with
the researchers in physics (cf. figure 1, b). Indeed, after the
As we were expecting to find behavioural evidence of these stretching process, when the force deployed by the machine
schemes, we added a third phase. This was aimed at is discontinued, the frame shape will change according to
evaluating whether the actions of the operators differed five characteristics. The first one is the plastic limit, which
according to their mental representations. Thus, eight determines the minimum strength required to make the
months later, we returned to observe the operators’ activity frame reach a plastic phase. The second characteristic is
over the course of a week and using a detailed grid. The Young’s modulus, which depends on the alloy used and
data were recorded using a pad and ACTOGRAM software. determines how fast the material returns to its initial
We observed two pairs: two novices and two experts. The position. This return phenomenon, which is called the
support for the novice pair had changed; at the time, one of spring back, is the third characteristic; it depends on a
combination of both the elastic limit and Young’s modulus. However, they differed with regard to their operative
The fourth characteristic consists of the faults that can invariants for the operative side of the stretching: only the
appear in the frame’s alloy. Finally, there are edge expert operator knew how to modify the program and was
constraints, i.e., the forces applied on the frame by the tools aware that speed has no effect.
used during bending; in other words, the wedge and tracks
Both the novice and expert operators used the
that keep the frame’s section shape.
measurements as inferences to modify their actions.
To reach the plastic phase and anticipate the spring back, However, whereas the expert created rules, the novice
the operator has to stretch and bend the frame using the needed the symbolic level, implying a most important
hydraulic arms of the machine. The operator can change the cognitive cost according to cognitive control model [4]. For
position of these arms and the speed at which it moves. The instance, when a frame is not bent enough, the expert
speed should not impair the result of the process, except if operator can directly add a certain amount of stretching.
an internal defect exists. The operator can also change the However, the novice would be inclined to investigate the
clamp of hydraulic cylinders that hold the frame. In order to possible reasons for the shortfall, such as the speed of the
achieve the exact shape required, the frame is bent around a process. He may then call for help.
mould, the size and position of which may change. Finally,
Behavioural Evidence of Schemes and Cognitive Control
the operator can refrigerate the frame at any point in order A total of 54 different actions were made during the week.
to maintain the material’s characteristics after tempering. The appearance of each observed action per participant, per
With regard to edge constraints, the operator has to insert a session and within a fixed 120 seconds time slot was
wedge and tracks. The sizes of these two tools are fixed, but counted. Making the hypothesis that the number of actions
the operator could take another reference at this point in (denoted Y) follows a Poisson distribution, several
order to improve the process. It is worth noting that a Generalized Linear Models (GLM) were computed. This
project aimed at measuring these sections revealed that the allowed us to describe a relationship between a measure
dimensions of each frame are different. and one or more variables (also called predictors). Here, the
predictors were the time (in secs) from the beginning of the
Task Analysis
activity, the level of operator experience (expert (e), novice
Interviews: Schemes and Cognitive Control (n), “mid-expert” (me) and student (s)), the frame rank (the
We present the results obtained by interviewing two pilots rank of the frame during the session) and style (the frame
(one expert and one novice) and the industrial technician. reference), and the stretching (first vs. second). The models
We classified the interviews according to the scheme model were selected according to the Bayesian Index Criterion
[8]. Each operative invariant was then classified in terms of (BIC) using a stepwise procedure.
the means-end hierarchy.
The results showed that time had no effect. An operator
Both the novice and expert operators underwent a common effect was found for only five of the models: “Angle”,
initial training as coppersmiths; consequently, they shared “Change Speed”, “Manual Command”, “Look Away”, and
the same operative invariant relating to the general process. “Talk”.

Figure 1: Means-end hierarchy for the stretching and bending task.

 “Angle” means that the pilot checks the section’s lower level should be represented, taking physical laws into
straightness. Measuring the angle acts as a control of the account. Moreover, the most important elements according
activity and a desire to improve the process. The best model to the expert operator, such as angles or bending should be
for this action indicates an effect of the stretching process emphasized. The interface would then afford the novice the
(first vs. second) and of the operator. According to the appropriate corrective actions; he could change the tools’
model, the expert operator is more likely to measure the reference or, maybe, the program. Finally, the tools should
angle (expected mean for the expert me= 0.26 action per be taken into consideration. This is in contrast with what is
time slot) than the other operators (mme,n,s≈0), but only offered by the current simulation program. For the expert
during the second stretching. operator, it may reduce the number of iterations needed to
achieve a good result and may also allow the operator a
“Change speed” was coded each time the pilot turned the
greater capacity for anticipation.
speed button. It is an element of the operators’ scheme. The
best model indicated an effect of the operator only. The We are expecting to test such an interface in a subsequent
expert used this button less (me=0.14) than the mid-expert, phase of our work; indeed, a microworld [5] is developed.
the student and the novice (mme=0.93; mn 0.73; ms1.17). Furthermore, we are expecting to extend the qualitative
results presented here by testing on a larger scale.
“Manual command” was coded each time the pilot
manually directed the machine’s hydraulic arms. This REFERENCES
happened during the second stretching to help the support 1. Bennett, K.B. and Flach, J. Display and interface
operator. This action could indicate the reactive-anticipative design: subtle science, exact art. CRC Press, Boca
dimension of the activity for more manual commands imply Raton  ; London, 2011.
more trials. An effect of the stretching (first vs. second) and 2. Charron, C., Hoc, J.-M., and Milleville-Pennel, I.
the operator was found. During the second phase, the expert Cognitive control by brain-injured car drivers: an
used the manual command less often (me=0.57) than the exploratory study. Ergonomics 53, 12 (2010), 1434–
other operators (mme=1.71; mn=1.32; ms=1.83). 1445.
“Look away” was coded each time the pilot stopped 3. Flach, J.M., Tanabe, F., Monta, K., Vicente, K.J., and
focusing on the task to look around. The best model Rasmussen, J. An ecological approach to interface
specifies an effect of the operator only. The expert design. Proceedings of the Human Factors and
(me=0.45) and the novice (mn=0.42) looked away more than Ergonomics Society Annual Meeting, SAGE
the others (mme0.14; mn≈0). Publications (1998), 295–299.
“Talk” was coded each time the pilot talked to the support 4. Hoc, J.-M. and Amalberti, R. Cognitive control
operator or to anyone else. The operator was the only dynamics for reaching a satisficing performance in
predictor. The expert talked more (me=1.44) than the others complex dynamic situations. Journal of Cognitive
(mme=0.36; mn=0.39; ms=0.06). Both “look away” and Engineering and Decision Making 1, 1 (2007), 22–55.
“talk” are indicators of the operators’ distraction. Less 5. Omodei, M.M. and Wearing, A.J. The Fire Chief
distraction could indicate that more cognitive resources are microworld generating program: An illustration of
allocated to the task, suggesting a more symbolic activity. computer-simulated microworlds as an experimental
CONCLUSION paradigm for studying complex decision-making
All the interviewed operators seemed to share invariants behavior. Behavior Research Methods, Instruments, &
relating to the higher levels of the means-end hierarchy. Computers 27, 3 (1995), 303–316.
However, in contrast with the novice and the industrial 6. Rasmussen, J. Skills, rules, and knowledge; signals,
technicians, the expert operator developed an operative signs, and symbols, and other distinctions in human
scheme that relates to the machine. To do so, he used the performance models. Systems, Man and Cybernetics,
different levels of the means-end hierarchy. Thus, we were IEEE Transactions on, 3 (1983), 257–266.
able to observe behaviours that reflect the expert operator’s
7. Rogalski, J. Psychological analysis of complex work
invariants. On the contrary, the three other operators did not
environments. In M. Fischer, N. Boreham and B.
control the result of their activity. They were not capable of
Nyham, eds., European perspectives on learning at
modifying their activity to improve their outcomes; they
work: the acquisition of work process knowledge. Office
had probably changed their acceptable performance.
of official publications of the european communities,
According to EID [1], a good interface should link the Luxembourg, 2004, 218–236.
lower levels of the means-end hierarchy to the higher 8. Vergnaud, G. Multiplicative structures. Number
levels. This is precisely what the expert does. In this case concepts and operations in the middle grades 2, (1988).
study, this would allow the novice operators to be aware of
the possibilities for their actions on the machine and relate 9. Vicente, K.J. Cognitive work analysis: Toward safe,
them to their existing knowledge of the physical process. productive, and healthy computer-based work. CRC
We suggest that the consequences of each action on the Press, 1999.

View publication stats