1 s2.0 S0747563221002016 Main

Computers in Human Behavior 123 (2021) 106878
Contents lists available at ScienceDirect
Computers in Human Behavior

journal homepage: www.elsevier.com/locate/comphumbeh
The future of artificial intelligence at work: A review on effects of decision

automation and augmentation on workers targeted by algorithms and
third-party observers
Markus Langer a, *, Richard N. Landers b
a
Fachrichtung Psychologie, Universität des Saarlandes, Saarbrücken, Germany
b
Department of Psychology, University of Minnesota, Minneapolis, MN, USA
A R T I C L E I N F O A B S T R A C T
Keywords: Advances in artificial intelligence are increasingly leading to the automation and augmentation of decision
Automated and augmented decision-making processes in work contexts. Although research originally generally focused upon decision-makers, the perspec
Artificial intelligence tive of those targeted by automated or augmented decisions (whom we call “second parties”) and parties who
Algorithmic decision-making
observe the effects of such decisions (whom we call “third parties”) is now growing in importance and attention.
Perceptions
Attitudes
We review the expanding literature investigating reactions to automated and augmented decision-making by
Review paper second and third parties. Specifically, we explore attitude (e.g., evaluations of trustworthiness), perception (e.g.,
fairness perceptions), and behavior (e.g., reverse engineering of automated decision processes) outcomes of
second and third parties. Additionally, we explore how characteristics of the a) decision-making process, b)
system, c) second and third party, d) task, and e) outputs and outcomes moderate these effects, and provide
recommendation for future research. Our review summarizes the state of the literature in these domains,
concluding a) that reactions to automated decisions differ across situations in which there is remaining human
decision control (i.e., augmentation contexts), b) that system design choices (e.g., transparency) are important
but underresearched, and c) that the generalizability of findings might suffer from excessive reliance on specific
research methodologies (e.g., vignette studies).
1. Introduction decision-making in medicine and management (Burton et al., 2020;

Longoni et al., 2019).
For over half a century, research and practice have explored how With good design and adequate testing, decision automation and
decision-making automation, which refers to automating decision- augmentation systems can often provide better and more efficient de
processes without remaining human control, and augmentation, which cisions than even the most experienced human experts (Grove et al.,
refers to the addition of system-support for human decisions, can in 2000; Kuncel et al., 2013). However, these benefits can be undermined
crease decision quality and efficiency (Benbasat & Nault, 1990; Meehl, by poor system design, misuse, and reluctance to adopt systems by
1954; Parasuraman et al., 2000; Raisch & Krakowski, 2021). In psy first-party users (Dietvorst et al., 2015; Parasuraman & Riley, 1997). We
chology, those concepts date back until at least Meehl (1954), who use the term first party to refer to people who use or interact with the
argued that it could be possible to translate decisions made by humans in output of such systems to make decisions that affect other people.
a subjective and informal way (clinical or holistic decision-making) into First-party users are distinct from developers, who develop systems and
a structured and formal way (mechanical or actuarial decision-making). then monitor, maintain, and update them. They are also distinct from
Nowadays, advances in artificial intelligence (AI)1 help to realize this upper-level managers, who may be responsible for implementation in a
structured way of decision-making in many application areas. For more abstract way but do not work directly with the systems. Our
example, AI-based systems increasingly automate or augment aspects of definition of first parties refers to people who have at least some direct
* Corresponding author. Universität des Saarlandes, Arbeits- & Organisationspsychologie, Campus A1 3, 66123, Saarbrücken, Germany.
E-mail address: markus.langer@uni-saarland.de (M. Langer).
1
We use artificial intelligence as an umbrella term, subsuming both classical manifestations like expert systems and deterministic human-programmed algorithms
with more recent ones, like machine learning and deep learning.
https://doi.org/10.1016/j.chb.2021.106878
Received 31 October 2020; Received in revised form 13 May 2021; Accepted 24 May 2021
Available online 29 May 2021
0747-5632/© 2021 Elsevier Ltd. All rights reserved.
M. Langer and R.N. Landers Computers in Human Behavior 123 (2021) 106878
control over whether and to what degree an artificial system will alter Resulting from our review, we present effects (on attitudes, perceptions,
the decisions they personally make. A prototypical first party at work is a behavior) of decision automation and augmentation on second and third
manager who uses a system to augment aspects of their decision-making parties, including moderation by characteristics of the decision-making
process regarding the personnel they manage. First parties often have process, of the system, of the second and third parties themselves, of the
the freedom to question the quality of the systems they are employing tasks, and of outputs and outcomes. We conclude by highlighting limi
and deviate from their recommendations, relying instead or more tations observed across the literature and by providing recommenda
heavily upon their own judgment (Benbasat & Nault, 1990; Highhouse, tions for future research.
2008).
To date, the majority of research has investigated the perspectives of 2. Review methodology
first parties (Benbasat & Nault, 1990; Burton et al., 2020; Hoff & Bashir,
2015), despite them being only one part of a complex network of Prior to our search we defined the following inclusion criteria for
stakeholders for almost any automation or augmentation system records considered in our review. Specifically, we did not restrict our
(Jungmann et al., 2020; Langer et al., 2021). For this paper, two other search to any specific timeframe and only included records written in
types of stakeholders are central. We call them second parties and third English. Moreover, we determined to only include research referring to
parties. Second parties are people whose lives, work, and well-being are the use of systems automating or augmenting decision processes with
directly affected and targeted, often without their consent or knowledge, the potential to directly affect an identified second party. We also
by automated and augmented decisions. Second parties cannot choose restricted our review to research that collected or interpreted empirical
whether they want to be affected by systems, their outputs, or decisions data, whether qualitative or quantitative, investigating the effects of
based on those outputs unless they exit the decision-making context decision automation or augmentation on second- or third-party experi
entirely, such as by quitting their job. Prototypical second parties are ences and behavioral outcomes. Furthermore, we only included research
employees who receive work tasks from automated systems (e.g., Uber that presented a comprehensible description of their study procedures
drivers; Lee et al., 2015), employees whose shifts are automatically and analysis methodology, which was not universal. Given our research
determined (Uhde et al., 2020), and job applicants whose application questions, we focused our initial review on research about decisions
information is evaluated by first parties supported by automated systems affecting people in work-related contexts. However, in the course of that
(Langer, König, & Hemsing, 2020). Third parties are people who observe review, we found a variety of papers in the field of medical decision-
an automated or augmented decision without being directly affected by making that appeared to be relevant to work-related contexts. Medi
that decision. Third parties are not directly affected by a particular de cine is somewhat unique in that systems may automate or augment
cision but may feel that they could become a second party in the future medical employees’ (e.g., physicians, nurses, technicians) decisions
or are concerned for the well-being, privacy, or some other character about patients instead of about workers or job applicants. Yet because
istic of second parties. For example, prototypical third parties are people we observed that research on augmented and automated medical
reading news articles on automated personnel selection practices or decision-making can be more advanced than for managerial decision-
people who hear from friends who are working in jobs where they making, we decided to explicitly include medicine in this review.
receive performance evaluations by automated systems (e.g., in algo Recognizing potential challenges in the generalizability of this research
rithmic management contexts; Wesche & Sonderegger, 2019). At higher to management, we discuss these examples only where necessary to
levels of analysis, the label “third party” can even apply to average illustrate trends and theoretical avenues not yet explored in manage
group, cultural, or societal reactions to specific decisions or policies, ment yet relevant to management theory.
such as global reactions to Amazon’s failure to automate resume We conducted our primary literature search on the SCOPUS database
screening procedures without undue bias (Lecher, 2019). and followed up with a search on Google Scholar. On each database, we
Understanding the perspectives of second and third parties to auto first conducted a preliminary search in which we identified relevant
mated and augmented decision-making at work constitutes a crucial sources and search terms, and which revealed that research relevant to
emerging line of research as decision automation and augmentation this review could be found in a variety of disciplines with different
increasingly determine how work takes place (Kellogg et al., 2020; publishing traditions. For instance, in psychology, scholars generally
Murray et al., 2020). Even if decision automation and augmentation is value journal publications as scholarly contributions whereas confer
accepted by first parties, second and third parties can either foster or ence proceedings are often unavailable and conference presentations are
impair success in practice (Healy et al., 2020). Specifically, second minimally valued. In comparison, although computer science journals
parties might be influential in improving or sabotaging first-party trust exist, the majority of scholarly work is published in conference pro
in the accuracy, efficiency, and consistency of automated and ceedings. Given our goals for an interdisciplinary review, we acknowl
augmented decisions by providing direct feedback to those first parties edged these varying publishing traditions and included published or
(Lee et al., 2015). Similarly, third parties could protest or use negative accepted work in conference proceedings, academic journals, and books.
word-of-mouth on social media to attempt to shape public opinion. Such For work found in online repositories that also cover preprints (e.g.,
behavior can discourage first parties from employing automation or arXiv), the authors discussed whether those articles should be included
augmentation, can affect policy makers and regulators in a way that based upon their individual quality. Furthermore, we talked to subject
influences the application of decision automation and augmentation in matter experts from various disciplines to ensure that we identified the
practice, can diminish organizational reputation and even spur litiga most relevant outlets and conferences.
tion. Although previous research has investigated first-party perspec We required for inclusion at least one hit in each category among
tives on decision automation and augmentation (at least in certain search terms referring to a) the system, b) where the system is used or
application areas Benbasat & Nault, 1990; Burton et al., 2020; Hoff & who is affected, c) terms referring to reactions or perceptions by second
Bashir, 2015; Onnasch et al., 2014), researchers only began to explore and third parties, and d) the study methodology. Within each of the
effects on second and third parties in the beginning of the 2000s. following lists of final search terms, multi-word terms were treated as
We contend that this area of research is now at a stage where a re one search term and logically separated by “or.” For (a), we used:
view is crucial to reveal systematic issues and blind spots that need to be “automated, automation, algorithm*, artificial intelligence, autonomous
addressed in the future. Consequently, the primary research questions machine, computer-assisted, computer-based, decision support system,
targeted with this review are: (1) how do automated and augmented expert system, intelligent system, machine learning”. For (b), we used:
decisions affect second and third parties? (2) what moderates these ef “advisor*, employee, individuals, job, management, manager, mana
fects? and (3) what are the next crucial steps to advance this research gerial, office, organization, physician, patients, workplace.” For (c), we
area? In the following section, we describe our review methodology. used: “accept*, belief, fairness, perceiv*, perception, react*, respond*,
2
satis*“. For (d), we used: “applicants, employee, experiment, field study, systems may support first parties by, for instance, providing processed or
laboratory study, participants, subjects, worker.” Among papers deemed evaluated information (Acikgoz et al., 2020; Suen et al., 2019), leaving
relevant, we used a snowball technique (Wohlin, 2014), seeking addi the final decision at the discretion of the first party. As system control
tional relevant articles by scanning references within already-identified increases, there is greater interaction between system and human
articles, as well as scanning papers citing the respective paper (via (O’Neill et al., 2020). For instance, in human-in-the-loop augmentation,
Google Scholar) and repeating this process until relevant references first parties and systems exchange information before an action is taken,
were exhausted. Fig. 1 provides a flowchart outlining search steps taken meaning that the system might request information from the human or
and the number of sources at each stage of filtering. Table 1 summarizes vice-versa (van Dongen & van Maanen, 2013). As system control in
the final set of studies that served as the basis of this review and also creases further, human control diminishes, until reaching full system
provides contextual information regarding those studies. control. Although taxonomies regarding the automation and augmen
Virtually all research that we identified on automated and tation decision-making were developed with first parties in mind (see
augmented decision-making at work related to human resource man also Makarius et al., 2020; Parasuraman et al., 2000), these strategies
agement tasks such as personnel selection or scheduling. We also describing how decisions are made could also affect reactions by second
became aware of a substantial number of papers that came from the and third parties and were thus considered important for our review.
extensive area of algorithmic management. Duggan et al. (2020, p. 119) Specifically, we anticipated that second- and third-party experiences of
defined algorithmic management “as a system of control where automated decisions vary by decision agency as control shifts from a
self-learning algorithms are given the responsibility for making and human to an automated agent. In our review, we ultimately used this
executing decisions affecting labour, thereby limiting human involve framework as an organizational tool and moderator of interest, as
ment and oversight of the labour process.” Although we sought to classified in Table 1. Specifically, we categorize the reviewed papers into
include this research where relevant, it was often unclear whether re papers that investigate automated, system-controlled decision-making
ported findings referred to participant experiences with automated de (Auto), decision augmentation, where human and system interact in
cisions or with the general work environment (e.g., being self-employed, decision-making (Augm), and human decision-making (Human).
not having a contract; Galière, 2020; Möhlmann et al., in press), and we
only included research that was unambiguously focused upon reactions 3. Effects of decision automation and augmentation on second
to the augmented or automated decision-making itself per our inclusion and third parties
criteria. As a result, we included few algorithmic management papers
relative to the overall size of that research literature. Our review of the effects of decision automation and augmentation
One insight that emerged from our review was the many ways that based on the studies presented in Table 1 is summarized in Table 2. For
decision automation and augmentation could be realized, which might clarity, we structured the results of our research, both in that table and
ultimately affect second- and third-party reactions. Most critically, we in the remainder of this section, in three components: attitudes, per
found that a continuum between full human and full system control (or ceptions, and behavior. Considered from the perspective of the study
automation) can be conceptualized. To formalize this continuum, Kaber participant in which these constructs are studied, perceptions refer to
and Endsley (2004) presented a taxonomy defining it across ten distinct how people immediately feel about and understand a system’s actions,
levels, ranging from human control, in which a first party is in complete whereas attitudes refer to conscious evaluations of systems when
control of a decision and all information used to make it, to full system queried. Behavior is defined as actions taken by second and third parties
control, in which decisions are fully automated without human oversight in response to their experiences or observations, respectively, of auto
or intervention. Between those extremes, the specific role of both mated or augmented decision-making. Importantly, all three outcome
humans and systems vary widely. At the lower end of system control, categories are likely to recursively affect each other (Fazio & Williams,
Fig. 1. Flow diagram of search steps. Dashed lines indicate the paths where identified records were excluded.
3
Table 1
Overview of the number of participants, the context, the decision process, methodology, words used to refer to the system and system function in the reviewed studies.
Study Participants Context Auto Augm Human Methodology Words used to refer System functioning
to the system
Decision-makingat work (general)

Acikgoz et al., 2020 298 US MTurk Personnel selection (x) (x) (x) All quantitative AI software using Screens resumes, conducts
Study 1 vignette advanced algorithms interviews, rates
interviews, makes
recommendation to
manager
Study 2 225 US students Personnel selection (x) (x) (x) AI software using Conducts interviews, rates
advanced algorithms interviews, makes
recommendation to
manager
Binns et al. (2018) 19 UK, no further Promotion x All quantitative Computer system Evaluates whether a
Study 1 information vignette based on a computer person should receive a
model promotion
Study 2 + 3 Study 2: 325 Promotion x Computer system, Same as in Study 1
Study 3: 65 predictive model
Both UK Prolific
Bigman et al. (2020) 2090 American Personnel Selection x All quantitative Computer program Reviews resume and
Pilot Study Trends Panel vignette interview information,
participants gives applicant scores,
applicants are hired based
on these scores
Study 2A-D A: 122 MTurk Personnel selection x x CompNet an Hires applicants
B: 241 Mturk artificial-
C: 241 Mturk intelligence-based
US, Canada computer program
D: 1499 UK
representative
sample from
Prolific
Study 3 240 MTurk Personnel selection x x Same as in 2A-D Hires applicants
US, Canada
Study 4 964 MTurk Personnel selection x x Same as in 2A-D Hires applicants
US Canada
Study 5 155 Norwegian Personnel selection x x Same as in 2A-D Hires applicants
tech workers
Dineen et al. (2004) 76 US students Personnel selection x x Quantitative vignette Automated screening Reviews applications,
system informs applicants
whether they were chosen
Gonzalez et al. 192 US MTurk Personnel Selection x x Quantitative vignette AI/ML tool Makes hiring decision
(2019)
Höddinghaus et al. 333 German Compensation, x x Quantitative vignette Computer program Allocates and selects
(2020) workers training, workshops and training
promotion courses
Allocates and determines
monthly bonuses and
promotion decisions
Hong et al. (2020) 233 US Qualtrics Personnel Selection x Quantitative vignette AI, algorithm, Chats with participants,
program, system evaluates interview
questions and resume
information
Howard et al. (2020) 22 US physicians Scheduling x x Quantitative, reactions Automated approach Makes schedules, assigns
to actual decisions people to shifts
Langer, König, and 124 German Personnel selection x x Quantitative and Computer Analyzes and evaluates
Hemsing (2020) students participants record participants interview
interviews responses
Langer, König, and 123 German Personnel selection x x Quantitative and Virtual interview tool Evaluates people,
Papathanasiou students participants watch a provides feedback to
(2019) video people, decides whether
applicants proceed to next
stage
Langer, König, 148 German Personnel selection x x Quantitative and Virtual interview tool Interviews applicants
Sanchez, and students and participants watch a
Samadi (2019) working video
individuals
Langer et al. (2018) 120 German Personnel selection x x Quantitative and Virtual interview tool Interviews applicants
students participants watch a
(psychology and video
computer science)
M. K. Lee (2018) 228 US MTurk Personnel x x Quantitative and Algorithm Assigns tasks
selection, qualitative vignette Decides who should come
Scheduling, to work
Performance Reviews resumes and
(continued on next page)
4
Table 1 (continued )
to the system
evaluation selects top candidates

Work assignment Evaluates employees
Marcinkowski et al. 304 German Selection of (x) x Quantitative vignette AI technology Analyzes applicant data
(2020) students students for and recommends
university approval or rejection of
admission applicants
Mirowska (2020) 184 students Personnel selection x x Quantitative vignette AI interview Reviews interviews
assessment software
Nagtegaal (2021), 109 Dutch alumni Reimbursement, x x x Quantitative and Computer, using an Decides about travel
Study 1 performance qualitative vignette automated algorithm reimbursement
evaluation Evaluated performance of
employees
Study 2 126 UK Prolific, Pension x x x Quantitative and Computer, using an Calculates pensions
employees of calculation, hiring qualitative vignette automated algorithm Scans CVs, interviews
governmental candidates, hiring
organizations
Newman et al. 199 US MTurk Layoffs and x x All quantitative An algorithm (i.e., a Determines who gets
(2020), Study 1 promotions vignette computerized promoted or laid off
decision-making
tool)
Study 2 1654 US Employee x x Human resource Decisions regarding
university evaluation algorithm promotions, layoffs,
employees raises, pay cuts
Study 3 189 US MTurk Bonus payment x x x An algorithm (a Determines how
determination computerized employee bonuses should
decision-making be allocated
tool)
Study 4 197 US students Employee x x Algorithm Evaluates employee
evaluation performance data and
comes to a final decision
on the performance
review
Study 5 213 US students Personnel selection x x Algorithm Evaluates recorded
responses, top scorers will
be put on a short list
Nolan et al. (2016) 468 US MTurk Personnel selection (x) x Quantitative vignette Computer program Combines data and
Study 1 that uses a calculates overall scores
mathematical for candidates
formula
Ötting & Maier 149 German Task allocation x x Quantitative vignette Intelligent computer Decides about task
(2018) Study 1 students including pictures Humanoid robot allocation
Study 2 145 German Allocation of x x Quantitative vignette Intelligent computer Decides about allocation
students vocational training including pictures Humanoid robot of vocational training
Schmoll & Bader 144 German Personnel selection x x Quantitative vignette Self-learning Screens applicants’ social
(2019) students algorithm media profiles
Suen et al. (2019) 180 actual Chinese Personnel selection x x Quantitative, real AI algorithm Analyzes interviews,
interviewees interviews for serves as a reference for
simulated job hiring decisions
Uhde et al. (2020), 51 German Scheduling (who x x Quantitative and System Decides who gets vacation
Study 2 healthcare gets vacation) qualitative vignette but also encourages
workers workers to find a solution
for themselves
van Esch & Black 293 non-specified Recruitment x Quantitative vignette, AI Analyzes applicant
(2019) survey information
Wang et al. (2020) 579 US MTurk Work evaluation x Quantitative vignette Algorithm Processes MTurkers work
and promotion history, decides over who
will become a Master
worker
Decision-making at work (algorithmic management in gig and platform work)
Anwar et al. (2021) 19 Indian workers Algorithmic x Qualitative interviews Algorithm Assigns work tasks,
management provides information,
manages evaluation
process
Bucher et al. (2021) 12,294 posts on Algorithmic x Qualitative analysis of Algorithm Assigns work tasks,
forum management forum posts provides information,
manages evaluation
process
Galière (2020) 21 interviews with Algorithmic x Qualitative interviews Algorithm Assigns work tasks,
French workers management provides information,
manages evaluation
process
Griesbach et al. 955 US workers Algorithmic x Quantitative survey, Algorithm Assigns work tasks,
(2019) survey management qualitative interviews provides information,
55 US workers manages evaluation
interviews process
5
to the system
Jarrahi and 33 workers Algorithmic x Qualitative interviews, Algorithm Assigns work tasks,
Sutherland (2019) 98 threads from management documents websites, provides information,
forums forum posts manages evaluation
Probably all US process
Jarrahi et al. (2020) 20 probably US Algorithmic x Qualitative interviews, Algorithm Assigns work tasks,
workers management forum posts provides information,
Forum posts manages evaluation
process
Kinder et al. (2019) 20 workers Algorithmic x Qualitative interviews, Algorithm Assigns work tasks,
19 clients management forum posts provides information,
125 forum manages evaluation
discussions process
Lee et al. (2015) 21 US drivers Algorithmic x Qualitative data from Algorithm Assigns work tasks,
12 US passengers management semi-structured provides information,
128 posts in online interviews, analysis of manages evaluation
forums posts in online forums, process
132 official blog analysis of blog posts
posts by the companies
Möhlmann and 15 informal, 4 Algorithmic x Qualitative interviews, Algorithm Assigns work tasks,
Zalmanson (2017) formal interviews management post entries; informal provides information,
with US and UK and formal interviews manages evaluation
workers and blog posts process
Möhlmann et al. (in 15 informal, 19 Algorithmic x Qualitative interviews Algorithm Assigns work tasks,
press) formal interviews management provides information,
with US workers manages evaluation
8 formal process
interviews with
employees and
engineers
Myhill et al. (2021) 32 interviews with Algorithmic x Qualitative interviews Algorithm Assigns work tasks,
Scottish workers management provides information,
manages evaluation
process
Ravenelle (2019) 31 US workers Algorithmic x Qualitative interviews Algorithm Assigns work tasks,
from two management provides information,
platforms manages evaluation
process
Wood et al. (2019) 107 interviews Algorithmic x Qualitative interviews, Algorithm Assigns work tasks,
with workers management survey provides information,
679 workers in manages evaluation
survey process
Southeast Asia and
Sub Saharan
countries
Venn et al. (2020) 58 Australian Algorithmic x Qualitative interviews Algorithm Assigns work tasks,
workers management provides information,
manages evaluation
process
Decision-making in medicine
Arkes et al. (2007), 347 US students All diagnosis (x) x All quantitative Computer program Tells the physician
Study 1 vignette whether they should order
an X-ray
Study 2 128 US students (x) x Computer program Determines risk of
diseases, advices
treatment
Study 3 74 US patients (x) x Computer program Assigns likelihood of
diagnoses
Study 4 131 US medical (x) x Computer program Assigns likelihood of
students diagnoses
Araujo et al. (2020) 958 Dutch, Treatment (and x x Quantitative vignette Artificial Decides about medical
representative others) intelligence, treatment
sample computers, computer
programs
Bigman & Gray 240 US MTurk Treatment x x All quantitative HealthComp an Decides whether to
(2018), Study 3 vignette autonomous perform a surgery
statistics-based
computer system
Study 6 239 US MTurk x x Same as in Study 3 Same as in Study 3
Study 7 100 US MTurk x x x Same as in Study 3 Same as in Study 3
Study 8 240 US MTurk x x Same as in Study 3 Same as in Study 3
Study 9 Within: 201 x x Same as in Study 3 Same as in Study 3
Between: 409
Both US MTurk
Haan et al. (2019) 20 US patients Diagnosis (x) x Qualitative semi- AI system, computer Analyzes radiological
structured interviews images and autonomously
evaluates scans
6
to the system
Hamilton et al. 46 US patients Diagnosis and x Group interviews, IBM Watson for Analyzes patient data,
(2019) treatment participants watched a oncology gives treatment
video recommendation
Jutzi et al. (2020) 298 German Diagnosis x Qualitative and survey AI Analyzes melanoma
patients
Jonmarker et al. 2196 Swedish Diagnosis x x x Qualitative survey Computer-only Breast cancer diagnosis
(2019) women and actual
patients
Keel et al. (2018) 98 Australian Diagnosis x x Quantitative, reactions Automated system Diagnoses retinopathy
patients to actual decision
Longoni et al. (2019) 228 US students Diagnosis x x All quantitative Computer uses an Stress diagnosis
Study 1 vignette algorithm
Study 2 103 US MTurk Diagnosis x x Computer capable of Stress diagnosis
artificial intelligence
Study 3 3a: 205, 3b: 227, Diagnosis x x 3a: robotic 3a: skin cancer screening
3c: 235 all US dermatologist is a 3b: diagnosis
MTurk computer program 3c: helps human to
3b: robotic nurse, conduct surgery
interactive animated
avatar
3c: Robotic surgeon
Study 4 100 US MTurk Diagnosis x x Same as 3a Skin cancer screening
Study 5 286 US MTurk Diagnosis x x Robotic Skin cancer screening
dermatologist
Study 6 243 US MTurk Treatment x x Computer Recommendation of
bypass operation
Study 7 294 US MTurk Treatment x x Computer program Recommendation of
uses an algorithm bypass operation
Study 8 401 US MTurk Diagnosis x x Computer program Recommendation of
uses an algorithm bypass operation
Study 9 179 US MTurk Diagnosis x x x AI dermatologist that Skin cancer screening
is an algorithm
Study 10 No information Diagnosis Robotic Dry eyes diagnosis
ophthalmologist,
computer that uses
an algorithm
Study 11 92 US MTurk Various x x Well-trained Gives advice
algorithm
Nelson et al. (2020) 48 US patients Diagnosis x x x Qualitative semi- AI program, AI tool Automated diagnosis or
structured interviews support of diagnosis
Palmeira & Spassova 36 US panel University x x x All quantitative Computer program Provides a favorability
(2015), Study 1 admission vignette score to rank candidates
Study 2 Between: 117 US Diagnosis and (x) x Computer program Determines risk of
panel treatment and decision aid diseases advices treatment
Mixed: 75 US
panel
Study 3 Medical: 41 US Diagnosis, x x x Medical software Informs about symptoms/
panel university candidate characteristics
Admission: 42 US admission and probability of illness/
panel success
Palmisciano et al. 20 UK patients, Diagnosis and x Qualitative and AI system Analyzes images, works
(2020) qualitative treatment quantitative survey out surgical plan, alarms
107 UK patients with vignettes about risks, supports
and relatives surgeon, operates patient
autonomously
Pezzo & Pezzo 59 US students All diagnosis (x) x All quantitative Computer decision Combines test results
(2006), Study 1 vignette aid
Study 2 166 US medical (x) x Computer decision Makes a diagnosis
students and 154 aid
students
Promberger & Baron, 68 US panel Diagnosis and x x All quantitative Computer program Provides recommendation
2006, Study 1 treatment vignette
Study 2 80 US panel Diagnosis and x x Computer program Provides recommendation
treatment or autonomous decision
Shaffer et al. (2013), 434 US students Treatment (x) x All quantitative Decision aid Determines whether
Study 1 vignette [computer program] patient should have an X-
ray
Study 2 109 US students Treatment (x) x Decision aid Determines whether
[computer program] patient should have an X-
ray
Study 3 189 US students Diagnosis (x) x Computer-based Calculates a score that
diagnostic aid tells the probability for
appendicitis
Srivastava et al. 100 US MTurk Diagnosis x Quantitative, decision Data driven Skin cancer screening
(2019) between different algorithm
forms of algorithms
7
to the system
Stai et al. (2020) 264 US Diagnosis and x x Quantitative survey AI Analyzes images, gives
participants treatment recommendation on
further treatment
Tobia et al. (2021) 1356 US Treatment x Quantitative vignette AI Recommends
representative chemotherapy drug dosis
sample
Wolf (2014), Study 1 218 US IT students All diagnosis (x) x All quantitative Computer program Assigns likelihood of
vignette diagnoses
Study 2 101 US IT students (x) x Computer program Assigns likelihood of
diagnoses
Yokoi et al. (2020) 272 Japanese Diagnosis and x x Quantitative vignette AI Analyzes symptoms,
college graduates treatment identifies disease,
suggests medical
treatment
York et al. (2020) 216 UK patients Diagnosis and x x Quantitative survey AI Helps human to analyze
treatment X-rays, helps deciding
how to manage injuries
Note. Auto = this study investigates automated, system-controlled decision-making; Augm = this study investigates decision augmentation, where humans and systems
interact in decision-making; Human = this study investigates human decision-making. (x) = unclear description of decision-making situation. The column “Words used
to refer to the system” consists of quotes of the respective papers. For papers including vignette studies, this highlights the words used to describe the system to
participants within the respective vignette studies.
1986), so none should be considered in isolation. 2019; Jutzi et al., 2020; Nelson et al., 2020).
Further important with respect to ability, in the case of fully-
3.1. Attitudes automated decisions, second parties seem to be initially unsure about
the performance of automated systems in high-stakes decision contexts.
We identified three major types of attitudes in relation to decision For example, in qualitative studies, several representatives of second
automation and augmentation: evaluations of trustworthiness, reduc parties reported that they expect more accurate whereas others expect
tionism, as well as reluctance regarding automation of moral decisions. less accurate diagnoses when using decision automation in medicine
(Jutzi et al., 2020; Nelson et al., 2020). People also seem to question the
3.1.1. Evaluations of trustworthiness validity of automated decisions, and patients may even explicitly call for
A number of studies investigate second- and third party evaluations scientific evidence showing that they can trust automated systems (Haan
of trustworthiness facets that relate to the system or the decision-maker et al., 2019). Höddinghaus et al. (2020) found no differences regarding
receiving system support (J. D. Lee & See, 2004; Mayer et al., 1995). people’s evaluation of human versus system data processing capabilities
Specifically, in the terminology of Mayer et al.’s (1995) facets of trust in different managerial tasks. However, humans were evaluated as more
worthiness, the reviewed papers predominantly investigate evaluations adaptable to changing circumstances which the authors consider
of ability and integrity, and some papers also evaluate benevolence. In another facet of ability. Since automated systems might not only aim to
the reviewed studies, ability relates to perceived performance of a sys increase diagnostic accuracy but also decision-making efficiency, effi
tem or the abilities of a decision-maker supported by a system. Integrity ciency perceptions can also contribute to overall evaluations of the
refers to the believe that the first party or system is unbiased in their ability (i.e., performance) of automated systems. People’s expectations
decisions. Benevolence relates to the evaluation of how much a first are split regarding efficiency of automated systems as some studies
party or a system will consider humans’ needs and how much they showed that people expect increasing diagnostic speed and earlier
“care” about individuals. Second parties will consider the combination detection of diseases (Haan et al., 2019; Hamilton et al., 2019; Jutzi
of those trustworthiness facets when they decide whether to trust or rely et al., 2020; Nelson et al., 2020) whereas others found no differences in
on the respective automated or augmented decision (Höddinghaus et al., expectations regarding waiting time expectations (Arkes et al., 2007; V.
2020; J. D. Lee & See, 2004). A.; Shaffer et al., 2013; Wolf, 2014). Similarly, high expectations
Overall, results regarding assessment of the trustworthiness facet regarding efficiency were reported in studies investigating workers’
ability are inconclusive. Performance and ability evaluations are central beliefs about algorithmic management (Galière, 2020).
to the trustworthiness of first parties and of automated systems (J. D. Lee Regarding the trustworthiness facet integrity, people seem to pre
& See, 2004). In medical decision-making, there is research investi dominantly believe systems to possess high integrity. People seem to
gating second-party perceptions of the ability of human experts using believe that systems have less discrimination motivation than humans
systems for decision-making. For instance, research indicates that sec (Bigman et al., 2020) which seems to be related to beliefs that systems
ond and third parties can ascribe lower ability to first parties that use possess less agency (Bigman & Gray, 2018) and with the belief that
systems in their decisions (Arkes et al., 2007; Wolf, 2014). However, systems do not have an own agenda they follow (Myhill et al., 2021).
there is no consensus whether those reactions are specific to first parties Furthermore, Höddinghaus et al. (2020) also found that people perceive
consulting systems versus consulting other human experts (V. A. Shaffer systems to be less biased than human decision-makers. It is important to
et al., 2013), if the magnitude of reactions is dependent upon the level of stress that attitudes regarding integrity of automated systems are closely
automation (Palmeira & Spassova, 2015), or on the behavior of the first related to what we discuss under perceptions of consistency and objec
party (Pezzo & Pezzo, 2006). When explicitly comparing augmented tivity. Specifically, if people believe the decision agent to have less
decisions with unaided expert judgement, using systems can even result discrimination motivation, to be less biased, they might also expect (at
in better perceptions of the abilities of the first party (Hamilton et al., least initially) that decision processes will be more consistent and
2019; Palmeira & Spassova, 2015). In addition, second parties seem to objective. However, this is contingent on second- and third-party beliefs
be concerned about human deskilling, such as by worrying that physi that systems can lead to lower bias, and this attitude in the population at
cians will become less able to diagnose without augmentation systems large is, given the lack of familiarity with automated decision-making,
and eventually lose their ability to detect system errors (Hamilton et al., likely unstable.
8
Table 2
Effects of decision automation and augmentation on second and third parties.
Outcome Current consensus Sample research question Selected sources
Attitudes
Evaluations of People are sometimes unsure about the ability of humans using What are the contexts where people are Höddinghaus et al. (2020);
trustworthiness systems, tend to question the ability of automated systems for initially sceptical about ability of automated Nelson et al. (2020)
decisions, but also sometimes expect systems to be equally or systems for decisions?
more able and efficient as humans. Additionally, people believe
that systems have higher integrity and are less biased than
humans. Furthermore, systems are perceived as less benevolent,
but evidence is limited.
Reductionism Decision automation neglects unique conditions, qualitative Is it possible to alter attitudes regarding Longoni et al. (2019);
information, and decontextualizes as well as quantifies decision reductionism? Newman et al. (2020)
processes.
Reluctance regarding Evidence is limited but people do not appear to want decision Why are people sceptical about decision Bigman and Gray (2018)
automation of moral automation for decisions with obvious moral components. automation for decisions with moral
decisions components?
Perceptions
Fairness and Justice Mixed results; more studies have found negative fairness What moderates effects on fairness and Howard et al. (2020);
perceptions, but some have found positive effects or no effects, justice perceptions? Newman et al. (2020); Ötting
this seems to depend on the task at hand. Decision automation & Maier (2018)
generally leads to negative interpersonal justice perceptions, to
better perceptions of consistency, and to lower perceived
controllability. Effects regarding informational justice are
inconclusive.
Organizational Mixed findings regarding organizational attractiveness. How do other perceptions and attitudes Langer, König, &
attractiveness translate to organizational attractiveness? Papathanasiou (2019);
Newman et al. (2020)
Accountability and Research suggests that decision automation and augmentation What are practical implications of potentially Pezzo & Pezzo (2006); Tobia
responsibility affect accountability and perceived responsibility but findings are diffused responsibility between decision- et al. (2021)
limited to vignette studies and mostly stem from the medical makers and automated systems?
domain.
Being under system In algorithmic management, there is a tension between being How does the tension between control and Möhlmann & Zalmanson
control vs. autonomy under constant system control but at the same time feeling autonomy affect workers’ job satisfaction? (2017); Griesbach et al., 2019
autonomy because of having no human boss.
Privacy concerns Automated decision-making appears to lead to greater privacy Why do people perceive privacy concerns for Langer, König, and
concerns, but the literature is small. automated decisions? Papathanasiou (2019)
Behavior
Reverse engineering of Some second parties appear to try to reverse engineer (e.g., Does reverse engineering improve perceived Kinder et al. (2019)Lee et al.
automated decisions through experimentation) the functioning of automated systems. control of automated decisions? (2015)
Creating workarounds Some second parties try to use system peculiarities for their own How do automated systems and bypassing Möhlmann et al. (in press);
benefit or to circumvent control by the automated system if the behavior coevolve? Who seeks workarounds? Wood et al. (2019)
outcome is valued.
Collective action Some second parties use online forums to share knowledge, Would this kind of collective action also Möhlmann et al. (in press)
complain, and engage in collective action against automated translate to employees in organizations?
systems.
Embracing the system People try to keep their own ratings within systems high, embrace Does embracing the system undermine Galière (2020); Wood et al.
system functionality as being fair and efficient. scrutinizing of system outputs? (2019)
As a final facet of trustworthiness, reviewed papers also investigate decision-making, as well as given trustworthiness evaluations as po
benevolence. In general, there are only few papers investigating this tential antecedents regarding perceptions of automated and augmented
facet, but those find that people ascribe lower benevolence to systems decisions, this is as an important line of future research in the work
than to human decision-makers. Specifically, people believe that sys context.
tems do not consider individual needs and do not care about individuals
(Höddinghaus et al., 2020; Yokoi et al., 2020). This might also relate to 3.1.2. Reductionism
what we discuss in the section on reductionism as well as lower per Our review suggests that second and third parties believe that deci
ceptions of interpersonal justice. sion automation suffers from reductionism (Newman et al., 2020).
Evaluations of trustworthiness finally result in effects on intentions Specifically, people seem to believe that systems consider more factors
to trust. Regarding trust, automated decisions seem to predominantly overall but neglect unique conditions, qualitative information, and de
result in less trust compared to human decisions and this seems to apply contextualize as well as quantify decision processes (Longoni et al.,
to decision-making at work (Höddinghaus et al., 2020; M. K. Lee, 2018) 2019; Newman et al., 2020). This might also be a cause of several
and in medicine (Palmisciano et al., 2020; York et al., 2020). However, negative perceptions by second and third parties as Newman et al.
this lack of trust might be contingent on a variety of moderator such as (2020) found negative fairness perceptions to automated decisions due
characteristics of the task that is to be automated (e.g., personnel se to the fact that second parties believe that systems do not or cannot
lection vs. scheduling; M. K. Lee, 2018; image analysis vs. medical adequately use qualitative information (e.g., leadership skills). Simi
treatment recommendations; Palmisciano et al., 2020). larly, Longoni et al. (2019) found that people are reluctant to use
In sum, existing research regarding decision automation and automated systems for medical diagnoses as they think that those sys
augmentation and issues of trustworthiness as well as trust comes mostly tems neglect their unique individual conditions (but see Yokoi et al.,
from the field of medical decision-making. Although questions emerge 2020 who did not find significant differences regarding uniqueness
in decision-making at work (Höddinghaus et al., 2020; Nolan et al., neglect). Similarly, Hamilton et al. (2019) report that their participants
2016), it has not yet received the same empirical attention. Given un believed that only human physicians have a holistic view of individual
clear and sometimes contradictory results in the field of medical patients which makes systems inadequate for personalized care. In sum,
9
those findings speak towards the attitude that automated decisions are Second, procedural justice perceptions were affected positively and
reductionist and dehumanizing. Importantly, this is also a reason for negatively. This mixed picture makes sense since procedural justice re
resistance to use automated systems from the perspective of first parties fers to different facets where some are more likely to positively affected
(Dawes et al., 1989; Grove & Meehl, 1996). First, second, and third by decision automation and augmentation, whereas others are more
parties all appear to assume that automated decisions suffer from likely impaired (Nagtegaal, 2021). On the positive side, automated de
reductionism. cisions improve perceptions of processes as being based on accurate
information, being performed consistently, as well as free of bias (Col
3.1.3. Reluctance regarding automation of moral decisions quitt, 2001), for instance in the context of job interview performance
Although empirical research is limited, both second and third parties evaluations (Acikgoz et al., 2020; Langer, König, Sanchez, & Samadi,
do not appear to want decision automation for decisions with obvious 2019; Marcinkowski et al., 2020). Even decision augmentation can lead
moral components. Specifically, over a various set of studies, Bigman to more perceived consistency in decision-making (Jutzi et al., 2020;
and Gray (2018) found that people do not want systems to make moral Nelson et al., 2020; Nolan et al., 2016). Moreover, in algorithmic
decisions in medicine (e.g., decisions about treatments) and attributed management contexts, automated evaluation was perceived as an effi
this to reduced agency and reduced ability to feel and sense. cient and objective way to evaluate workers’ performance (Galière,
2020).
3.2. Perceptions On the negative side of procedural justice perceptions, automated
decisions seem to impair perceptions of whether it is possible to express
We identified five major types of perceptions: fairness and justice, one’s views and feelings about a process, appeal processes (e.g., appeal
organizational attractiveness, accountability and responsibility, super negative customer ratings; Griesbach et al., 2019; Möhlmann et al., in
vision vs. autonomy, and privacy invasion. press; Myhill et al., 2021), and more generally control decision processes
and outcomes. Specifically, reduced perceived control of automated
3.2.1. Fairness and justice decisions was found for people in personnel selection (Langer, König, &
In line with growing interest regarding algorithmic fairness, Papathanasiou, 2019; M. K. Lee, 2018), where they seem to be unsure
accountability, and transparency (Shin & Park, 2019), fairness and how to affect automated decision processes in a way that could improve
justice perceptions reflect the most commonly investigated outcomes in their performance ratings (Acikgoz et al., 2020; Langer, König, Sanchez,
the reviewed papers. For perceptions of fairness, observed effects were & Samadi, 2019). In algorithmic management processes, perceived lack
mixed but predominantly negative. Some of the studies in personnel of controllability was often associated with a lack of perceived trans
selection scenarios based on job interviews found no differences in parency of what contributes to decision outcomes by automated systems
fairness perceptions between human and automated decisions (Langer, (Möhlmann et al., in press; Myhill et al., 2021; Veen et al., 2020). In these
König, & Hemsing, 2020; Suen et al., 2019). In a school admission cases, the lack of transparency was usually described as an intentional
context, Marcinkowski et al. (2020) found stronger fairness perceptions design choice by platform providers in order to better control the
for automated compared to human decisions. Similarly, Howard et al. workforce and to prevent people from gaming the system (Galière, 2020;
(2020) reported stronger fairness perceptions after the implementation Möhlmann et al., in press). Future research could thus investigate
of an automated scheduling system for residents at a hospital compared methods to increase perceived controllability without enabling workers
to scheduling by human decision-makers. However, most of the to game respective systems.
reviewed studies found that people perceive automated decisions as less Third, results regarding informational justice, which refers to
fair than human decisions across personnel selection, performance perceived openness, honesty, and transparency in decision processes
evaluation, and scheduling scenarios (Dineen et al., 2004; Langer, (Colquitt, 2001), were inconclusive. As we have just mentioned, quali
König, & Papathanasiou, 2019; M. K. Lee, 2018; Newman et al., 2020; tative results in algorithmic management contexts indicate that people
Uhde et al., 2020; Wang et al., 2020). Additional qualitative findings might perceive low informational justice as they do not understand how
indicate that fairness was a concern in algorithmic management contexts algorithmic management decisions work (Griesbach et al., 2019; Lee
(Lee et al., 2015; Myhill et al., 2021) where different design choices et al., 2015 ; Möhlmann et al., in press; Myhill et al., 2021; Veen et al.,
resulted in more, or less perceived fairness (Griesbach et al., 2019). 2020). In personnel selection, there is tentative evidence that people
Overall, the mixed findings regarding fairness suggest that there are perceive automated decisions as less open towards the applicants than
moderators that have not yet been sufficiently investigated, a concept human decisions, although there were no differences in perceived in
we revisit in greater detail later in this review. formation known about the decision processes (Acikgoz et al., 2020).
Regarding justice, three of the four dimensions of organizational Whereas those findings indicate that informational justice might be
justice theory (Colquitt, 2001) were commonly studied: interpersonal, lower in the case of automated decisions, there is also hope that auto
procedural, and informational justice. Overall, people seem to expect mated processes could enhance informational justice (Höddinghaus
both positive and negative effects regarding the single justice facets. This et al., 2020). Specifically, transparency of decisions could increase if
seems to be closely related to what we have discussed regarding atti automated system were designed to be transparent (Lee et al., 2015) or
tudes towards decision automation (i.e., reductionism, evaluations of at least to provide better explanations of their recommendations than
trustworthiness) which supports previously found close relation be humans do (Jutzi et al., 2020).
tween trustworthiness evaluations and perceptions of justice (Colquitt &
Rodell, 2011). 3.2.2. Organizational attractiveness
First, interpersonal justice, which refers to the perception of being Organizational attractiveness refers to perceptions of how second
treated with dignity and human warmth in decision processes (Colquitt, and third parties perceive organizations that sponsor algorithmic de
2001), was generally harmed by automation. Studies on automated job cisions. In the case of personnel selection, organizational attractiveness
interviews found lower social presence, two-way communication, and seemed to be partly driven by fairness and justice perceptions of the
less adequate interpersonal treatment in automated interviews (Acikgoz algorithmic decision (Ötting & Maier, 2018) and were predominantly
et al., 2020; Langer, König, Sanchez, & Samadi, 2019; Langer, König, & negative. Similarly, Acikgoz et al. (2020) found lower organizational
Papathanasiou, 2019). Similar findings come from the area of medical attractiveness, job pursuit intentions, and stronger litigation intentions
decision-making where patients fear a lack of human touch, less ability when using decision automation in interviews with mediations princi
to ask questions, and that systems might not provide potentially nega pally through decreased chance to perform and decreased two-way
tive outcomes (e.g., diagnoses) in a sensitive manner (Haan et al., 2019; communication quality. Similarly, automated interviews have been
Nelson et al., 2020). associated with reduced organizational attractiveness compared to
10
videoconference interviews, and social presence and fairness seem to be system control can be perceived as providing more autonomy than
the most important mediators in this effect (Langer, König, Sanchez, & comparable supervision by a human boss (Anwar et al., 2021; Griesbach,
Samadi, 2019; Langer, König, & Papathanasiou, 2019). In contrast to the Reich, Elliott-Negri, & Milkman, 2019; Möhlmann et al., in press;
negative findings from selection contexts in organizations, in university Möhlmann & Zalmanson, 2017; Wood et al., 2019). We need to
admissions, the use of systems was associated with better organizational emphasize that parts of these perceptions could also be affected by the
reputation (Marcinkowski et al., 2020). overall work environment and by platform providers such as Uber or
This suggests unexplored moderators affecting people’s perceptions Upwork selling jobs on their platforms as being “entrepreneurial”. This
of organizational attractiveness. For instance, we could imagine that might contribute to workers’ self-identity as being autonomous and
interpersonal justice facets are especially important in the case of job having flexible working hours when they are actually following strict
interviews (Langer, König, & Papathanasiou, 2019) such that in this rules and algorithmic control (Galière, 2020; Jarrahi et al., 2020).
context, automated decisions impair organizational attractiveness. In
contrast, there might be situations where consistency and objectivity are 3.2.5. Privacy invasion
more important and where the perceived benefits of automated de Automated decision-making can lead to perceptions of privacy in
cisions could result in higher attractiveness (e.g., in resume screening as vasion under certain circumstances, but this conclusion is based on few
a situation where there is usually no human contact; Marcinkowski studies. Privacy concerns are commonly debated when using automated
et al., 2020). decisions, as the technologies enabling decision automation and
augmentation often rely on automatic evaluation of sensitive data pro
3.2.3. Accountability and responsibility vided by second parties. In the context of personnel selection, privacy
Accountability and responsibility are commonly discussed for the use concerns may be higher for decision automation (Gonzalez et al., 2019;
of automated systems for high-stakes decisions across disciplines (Flo Langer, König, Sanchez, & Samadi, 2019; Langer, König, & Papathana
ridi et al., 2018; Kellogg et al., 2020; Mittelstadt et al., 2016). A common siou, 2019). However, the reviewed literature provided no empirical
concern is that there will be an accountability gap when humans rely on evidence as to the reasons why second parties would be more concerned
automated systems as there might be situations where it is not clear who providing private data to an automated system than to a human
is accountable for errors (Raji et al., 2020). Whereas there is only scarce decision-maker evaluating the same data. Additionally, in algorithmic
research in work-related contexts, the question of who is accountable for management, there is significant discussion of worker concern with
failures of automated systems in medicine was raised by participants in constant surveillance, but privacy concerns seem to be of lesser impor
the reviewed studies, although developers and first parties were tance in this area (Galière, 2020; Griesbach et al., 2019).
commonly believed to be ultimately accountable (Haan et al., 2019;
Nelson et al., 2020). 3.3. Behavior
Whereas those results speak to perceptions of actual legal account
ability, other studies found that people attributed different re We identified four commonly studied classes of behavior in response
sponsibility to decision-makers who use automated systems for decision- to automated decision-making: reverse engineering of automated de
making. For instance, systems can deflect part of the blame for negative cisions, creating workarounds, collective action, and embracing the
medical outcomes but only if physicians follow system advice (Pezzo & system. Research that investigates behavioral reactions to decision
Pezzo, 2006; Tobia et al., 2021). Tobia et al. (2021) conclude that this automation and augmentation predominantly comes from the area of
may constitute an incentive to follow recommendations by automated algorithmic management in gig and platform work, which focuses on
systems as this could shield against legal accusations. On the side of behaviors related to perceived controllability of automated decisions
second parties, being affected by a decisions that was shared between (see Kellogg et al., 2020; or Möhlmann et al., in press, for overviews on
human and system can result in being unsure who to blame for negative control in relation to algorithmic management). Second parties try to
outcomes of decision automation and augmentation (Jutzi et al., 2020; enhance their understanding of automated management processes
Nolan et al., 2016). Specifically, some second parties perceived less through reverse engineering and knowledge sharing in online forums,
decrease in their own perceived responsibility when systems provided and building a personal or shared understanding this way may increase
them with recommendations about medical treatment in comparison to perceived controllability (Jarrahi et al., 2020; Lee et al., 2015;
when humans gave this recommendation (Longoni et al., 2019; Prom Möhlmann et al., in press; Myhill et al., 2021). Furthermore, based on
berger & Baron, 2006). Authors of the respective papers argued that this their understanding of automated processes, second parties may use
might be attributable to diffused responsibility; the second-party may peculiarities of automated systems to create workarounds that either
not know who to blame in the event of an unfavorable outcome resulting avoid penalties or increase their income (Kinder et al., 2019; Myhill
from following and automated recommendation. Future work could et al., 2021; Veen et al., 2020; Wood et al., 2019), and success with such
investigate whether those effects are moderated by characteristics of the methods may also increase perceived controllability. Other attempts to
system (e.g., system accuracy; Lowe et al., 2002) as previous research increase perceived controllability may involve initiating collective ac
was unspecific about conditions that affect perceived responsibility. tion in regard to automated processes that affect millions of other second
parties (Lee et al., 2015). However, in most studied situations, there was
3.2.4. Being under system control vs. autonomy a sufficient power imbalance so as to force second parties to embrace
The tension between being controlled by an automated system versus automated decisions or give up their incomes. This lack of control
perceived autonomy at work it especially prevalent in algorithmic sometimes even led workers to describe automated decisions as the
management contexts. Specifically, workers are under constant sur fairest possible way to work assignment and evaluation (Galière, 2020).
veillance and controlled by algorithmic management but at the same
time perceive flexibility and autonomy in their work (Galière, 2020; 3.3.1. Reverse engineering of automated decisions
Griesbach et al., 2019; Möhlmann et al., in press). These perceptions Second parties often do not understand how algorithmic manage
seem to be partly due to the fact that workers do not have a human boss ment systems work and do not know how they can affect system out
telling them what to do (Anwar et al., 2021; Griesbach et al., 2019; comes favorably (Lee et al., 2015; Myhill et al., 2021; Veen et al., 2020).
Möhlmann et al., in press; Möhlmann & Zalmanson, 2017; Wood et al., To increase understanding, there are examples of second parties trying
2019). Instead, there is an automated system providing instructions, to reverse engineer automated decision-making process through exper
information, and evaluation. Although this leads to workers being aware imentation (Kinder et al., 2019;Lee et al., 2015 ; Möhlmann et al., in
of being under constant evaluation (Bucher et al., 2021; Kinder et al., press). Workers may change inputs into the system in order to see how
2019; Ravenelle, 2019; Veen et al., 2020), it seems that being under outputs change and build a mental model of system functioning (Jarrahi
11
& Sutherland, 2019). Similarly, workers have logged in as alternative moderators: system versus human control and first-party behavior in
user types in algorithmically managed systems to see how their changes decision process. Whereas there is only scarce research on work-related
influence what potential customers see (Jarrahi & Sutherland, 2019). decisions regarding these topics, results from medical decision-making
Reverse engineering seems to serve the purpose of increasing under support them (Jutzi et al., 2020; Nelson et al., 2020). For future
standing of the automated management system to increase controlla research, it will be central to investigate psychological processes
bility of the system. explaining the effects reported in this section. For example, why do
people prefer human control in certain application contexts? An expla
3.3.2. Creating workarounds nation based on findings from the perspective of first parties (Grove &
Second parties may try to use bugs and (possibly undocumented) Meehl, 1996) might be that humans fear that machines decide the fates
features of algorithmic management systems to create workarounds. For of human beings. Other interpretations in line with findings regarding
instance, a person may try to bring the interaction with their clients attitudes and perceptions might be that human control will improve
away from the influence of the automated system, for instance, by single case decision-making, will be less dehumanizing, and will allow
directly interacting with customers (Jarrahi & Sutherland, 2019;Lee the consideration of qualitative factors (i.e., there might be hope that
et al., 2015). Additionally, people try to use technical findings to their humans can mitigate reductionism). Additionally, people might hope
own benefit. For instance, users may log out of a system that is tracking that human decision-makers can be influenced to one’s own benefit (i.e.,
them to receive better evaluations (e.g., because systems cannot track are more “controllable”). Furthermore, with a human decision-maker
reckless driving behavior or the actual time a worker spend on a task; available, second and third parties may perceive someone who is a
Jarrahi & Sutherland, 2019; Lee et al., 2015; Möhlmann & Zalmanson, contact person to whom they can direct complaints and who is
2017; Wood et al., 2019). In another example, Upwork workers who accountable for negative outcomes.
inputted five 1-h blocks of work discovered they would have greater
system benefits than if they inputted those same hours as a single 5-h 4.1.1. System vs. human control
block (Jarrahi & Sutherland, 2019). These ways of how to game the Earlier in this paper, we anticipated the distinction between
system are under constant change as platform providers try to fix augmented and automated decision-making to be of importance. Indeed,
respective loopholes which makes it necessary for workers to search for people call for human control in high-stake decisions and seem to pre
new ways to create workarounds (Galière, 2020; Möhlmann et al., in dominantly perceive augmented as favorable to automated decisions.
press). For instance, second parties perceive decisions where first parties only
have the option to consult an automated system as fairer compared to
3.3.3. Collective action full automation or to decisions where first parties can only slightly
Numerous studies provide evidence that second parties are inclined change automated decisions (Newman et al., 2020). Furthermore, Suen
to share knowledge and evidence with other second parties and to keep et al.’s (2019) finding of no negative reactions to algorithmic
up to date about potential changes in functions of the systems they work decision-making in personnel selection might be because participants
under (Jarrahi & Sutherland, 2019;Lee et al., 2015 ; Möhlmann & Zal were informed that the algorithmic evaluation only served as a reference
manson, 2017; Myhill et al., 2021). This is often done via online forums, for the human decision-maker. Another perspective on the effects of
which may also provide a feeling of social embeddedness. Second parties human control comes from Nagtegaal (2021). She found in hiring or
use such forums to onboard new workers, to complain about the system, performance evaluation that only automating decisions negatively
to form alliances in protest, and to raise their collective voice against the affected perceptions; augmentation had no negative effects. In contrast,
usually inaccessible layer of developers behind the system (Lee et al., less human control led to stronger fairness perceptions in mathematical
2015). In such forums, second parties also sometimes organize resis tasks (e.g., travel reimbursement decisions). This suggests that there are
tance against the automated management system or to try to collectively also tasks where more system control can be beneficial.
game the system (Tassinari & Maccarrone, 2020). For instance, Additional evidence on system versus human control is generally
Möhlmann and Zalmanson (2017) describes a case where Uber drivers lacking for decisions at work; however, research in medical decision-
tried to collectively reduce work effort in order to increase demand making also suggests that humans prefer having a human involved in
which would in turn increase surge pricing to the benefit of worker pay. decision-making for certain tasks. Second parties often specify that
Even more drastic manifestations of collective actions are worker strikes systems should be more of an informant or second opinion than the final
(Möhlmann et al., in press; Tassinari & Maccarrone, 2020). trigger of a decision (Haan et al., 2019; Hamilton et al., 2019; Jutzi et al.,
2020; Palmisciano et al., 2020; Stai et al., 2020) and call for
3.3.4. Embracing the system human-system symbiosis (Nelson et al., 2020). Supporting those find
Several studies report that workers in algorithmic management ings, Bigman and Gray (Study 8; 2018) found that if systems only advice
embrace algorithmic management systems. They try to keep their human physicians, there is a higher likelihood that people accept de
evaluations within the system high (Bucher et al., 2021; Kinder et al., cisions. However, many people still preferred the human expert to
2019; Ravenelle, 2019; Wood et al., 2019). Furthermore, they perceive decide without automated support (see also York et al., 2020) (for
systems as being efficient and objective and thus recommend other further support of the positive influence of human control see Jonmarker
workers to play along with the rules of the system as this is the fairest et al., 2019; Longoni et al., 2019; Palmeira & Spassova, 2015).
possible evaluation of workers’ performance (Galière, 2020).
4.1.2. First-party behavior in decision processes
4. Moderators of effects on second and third parties Following versus rejecting advice by automated systems seems to
affect second-party attitudes and perceptions with regard to the first
Table 3 provides an overview of characteristics of the decision- party. For instance, Arkes et al. (2007) found that rejecting automated
making process, of the system, of the second- and third parties them recommendations can lead to lower trustworthiness assessment of the
selves, of the tasks, and of the outcomes. In each case, we either report decision-maker. Furthermore, when augmented decision result in an
research directly testing these moderators or observed differences be unfavorable outcome for second parties, people seem to ascribe more
tween studies adopting different approaches. fault to physicians when they reject automated advice compared to
when they follow the advice (see also Tobia et al., 2021). This finding
4.1. Characteristics of the decision-making process might be grounded in attributions of responsibility that might be lower
for when humans follow automated systems compared to when they
We identified two process characteristics as potentially influential ignore the advice (Lowe et al., 2002).
12
Table 3
Factors affecting the effects of decision automation and augmentation on second and third parties.
Category Current consensus Sample research question Selected sources
Characteristics of the decision-making process

System vs. human control People call for human control in high-stake decisions and Why do people prefer human control in certain Nagtegaal (2021);
seem to perceive augmented as more favorable than tasks? Newman et al. (2020)
automated decisions. However, this evidence stems mainly
from medical decision-making and may depend on the focal
task.
First-party behavior in decision Rejecting vs. following advice by automated systems seems to What are psychological processes underlying Tobia et al. (2021);
process affect second-party perceptions of the first party. different perceptions of first parties who reject Pezzo & Pezzo (2006)
or follow advice?
System characteristics
System performance All else being equal, humans prefer human control but when How do different system performance measures Bigman & Gray (2018);
system accuracy becomes better, humans start to prefer (e.g., accuracy, bias) affect perceptions of Longoni et al. (2019)
system control. systems?
Understandability and Inconclusive findings. What are trade-offs of providing explanations Binns et al. (2018);
transparency through and transparency to second parties? Langer et al. (2018);
information and explanation Newman et al. (2020)
Information about the developer Information about the developer can affect reactions to the Do information about the development process Bigman et al. (2020);
system. (e.g., about training data) also affect reactions? Wang et al. (2020)
Characteristics of second and third parties
Experience, familiarity, Mixed effects with some studies finding no effects other Does experience, familiarity, and education Langer et al. (2018);
education finding better perceptions of automated decisions with more affect perceived control and understanding? Wang et al. (2020)
experience, familiarity, education.
Personality and traits Inconclusive effects of studies investigating a range of What are influential personality facets and Longoni et al. (2019)
potentially influential traits (e.g., perceived uniqueness, trait traits and why are they influential?
privacy concerns)
Gender There is a tendency that females react less favorable to What are the underlying influences that may Araujo et al. (2020);
automated decisions. contribute to gender differences? York et al. (2020)
Task characteristics The respective task affects reactions, but it is unclear what What are the dimensions of tasks that affect M. K. Lee (2018);
exactly it is about tasks that affect second- and third-party second and third parties with respect to Nagtegaal (2021)
reactions. Research indicates that it is a mix of stakes of the automated and augmented decisions?
task, tasks that require human versus mechanical skills,
perceived quantifiability, and familiarity with automated
systems for the respective tasks.
Output and outcome Different kinds of outputs (e.g., standard vs. non-standard) What are further characteristics of system Wang et al. (2020);
characteristics affect perceptions. Unfavorable outcomes lead to negative outputs (e.g., output comprehensibility) that Hong et al. (2020)
reactions irrespective of human or automated decisions. affect perceptions?
However, overall research on effects of outputs and outcomes
is scarce.
4.2. System characteristics when system accuracy becomes better, humans start to prefer system
control (Longoni et al., 2019). For instance, patients would prefer
System characteristics that affect reactions to automated and AI-based medical systems when they are more accurate than a human
augmented decision processes are system performance, understand expert (Haan et al., 2019; Jutzi et al., 2020). However, if systems are
ability and transparency through information and explanation, as well similarly accurate like humans, people indicated that they would prefer
as information about the developer. Overall, only a small number of human over automated decisions. For instance, Bigman and Gray (2018,
specific system design features have received attention. In the context of Study 9) showed that it is necessary to make accuracy advantages of
algorithmic management, research shows possible implications of automated systems salient (i.e., in a within-subject comparison to the
different design choices when comparing different algorithmic man human expert) and strongly different (i.e., 75% human vs. 95% system)
agement platforms (Griesbach et al., 2019; Ravenelle, 2019). For in order to find preference for system decisions. Potentially, this could
instance, design choices may affect perceived fairness, perceived au indicate that expectations regarding system performance are unrealis
tonomy, job satisfaction, and overall work performance (Galière, 2020). tically high (Merritt et al., 2015). The study by Bigman and Gray even
However, as respective papers compare different platforms, without showed that in the case of a clear advantage of the automated system,
systematically investigating design options (see Ravenelle, 2019), it is 28% of participants indicated that they still would prefer the inferior
hard to tell which design differences between the platforms led to human decision. In a similar vein, Longoni et al. (2019) showed that
different worker reactions. Consequently, systematically assessing ef people are less sensitive to accuracy when deciding between human and
fects of design options for automated systems in decision-making and automated decisions versus when deciding between two human
their effects on second and third parties could receive more attention decision-makers. Specifically, they found that accuracy differences lead
(Landers & Marin, 2021). to stronger effects when deciding between two humans (i.e., participants
more strongly favored the human showing higher accuracy) compared
4.2.1. System performance to when deciding between a human and an automated system where the
System performance affects second-party reactions to automated system shows higher accuracy.
decisions. Whereas, accuracy (e.g., prediction accuracy) is the pre
dominant measure of system performance in the reviewed studies, there 4.2.2. Understandability and transparency through information and
are others (e.g., efficiency, lack of bias) that could also be considered to explanation
be performance measures and that may become increasingly important Whereas research, practice, regulatory bodies, and ethical guidelines
in new application areas of decision automation and augmentation (e.g., commonly call for understandability and transparency of systems and
in management, medicine) (Srivastava et al., 2019). their outputs, and assume that this positively affects outcomes for sec
All else being equal, humans prefer human control in decisions but ond parties (e.g., Dineen et al., 2004; Jobin et al., 2019), the reviewed
13
findings in relation to understandability and transparency are incon 4.3.1. Experience, familiarity, education
clusive. For example, Wang et al. (2020) informed their participants that Overall, experience, familiarity, and education are related to un
there are “information online available” about an automated decision derstandability or perceived control as people with more experience and
process which led to participants perceiving biased procedures as even familiarity with computers, programming, and higher education might
less fair compared to where no such information was available. In a believe that they understand how systems work which might give them
more direct investigation of the effects of explanations, Binns et al.‘s more confidence that they can control system processes and outputs
(2018) between-subject study showed only weak effects of different kind (Jonmarker et al., 2019; Langer et al., 2018). Intuitively, this could lead
of explanations on justice perceptions. In their within-subject study, to better perceptions of automated decisions. However, there are mixed
explicitly comparing different kinds of explanations resulted in more effects as some studies found no effects whereas others found better
negative justice perceptions for case-based explanations compared to perceptions of automated decisions with more experience, familiarity,
other explanations. On the one hand, this indicates that without direct and education. For instance, Langer et al. (2018) found no correlation of
comparison between different versions of explanations, their effect on computer experience and any of their perceived justice variables. Wolf
justice perceptions might be negligible. On the other hand, it is impor (2014) showed that even IT students derogate physicians who use
tant to test the effects of different kinds of explanations as some might automated systems. In contrast, Wang et al. (2020) showed that com
lead to comparably negative outcomes. Furthermore, Langer et al. puter literacy positively correlates with fairness perceptions regarding
(2018) found that providing information regarding what input variables automated decisions. Similarly, Jonmarker et al. (2019) found that
an automated interview system uses increases perceived transparency better understanding of new technology is associated with better re
but at the same time decreases perceived organizational attractiveness. actions to decision automation in breast cancer screening. Furthermore,
Similarly, Newman et al. (2020) findings indicate that whereas for Gonzalez et al. (2019) showed that familiarity with AI can mitigate
human decisions high transparency leads to more fairness, for auto negative reactions to automated decisions. Additionally, Araujo et al.
mated systems this effect is reversed. (2020) showed that programming knowledge is associated with more
A possible explanation for the overall inconclusive effects is that positive evaluations of usefulness and fairness of automated decisions.
more understandability without also perceiving more controllability Additionally, Bigman et al. (2020) showed that people with stronger AI
might be negative (Ananny & Crawford, 2018). For instance, research knowledge were less morally outraged by algorithmic bias which could
shows that only providing information about variables the system con be interpreted as either more positive evaluations of automated de
siders as inputs might be detrimental if people see that they might not be cisions or as a positivistic bias in relation to automated systems as they
able to control those inputs (e.g., their voice tone) (Grgić-Hlača et al., assume that biased system outputs are not associated with the system
2018; Langer et al., 2018; Newman et al., 2020). Another possible but maybe just reflect actual differences between people. Furthermore,
interpretation is that people might have different expectations regarding Wang et al. (2020) found that less educated participants react more
transparency of systems compared to humans as decision-makers. For strongly to unfavorable outcomes by automated decision processes.
instance, people may not even expect explanations from systems Relatedly, higher education levels seem to be associated with better
whereas explanations to increase transparency of a decision are some reactions to automated decisions in medicine (Jonmarker et al., 2019;
thing natural in the interaction with human decision-makers (Lombrozo, York et al., 2020).
2011; Zerilli et al., 2018) – a lack of explanations could be interpreted as
intentional distortion in the case of a human decision-maker (Schlicker 4.3.2. Personality and traits
et al., 2021). Another possibility might be that people believe they un The reviewed papers investigated a range of traits potentially influ
derstand human decision-making processes although those are eventu ential (e.g., perceived uniqueness, trait privacy concerns, locus of con
ally also black box decisions (Zerilli et al., 2018). We see potential trol) with inconclusive effects. For instance, Araujo et al. (2020) showed
regarding research on understandability of systems which is also re that trait privacy concerns were negatively associated with fairness and
flected in the current boom in research on explainable artificial intelli positively with perceived risk of using automated decisions. V. A. Shaffer
gence (XAI) (Miller, 2019). Future research needs to investigate how to et al. (2013) showed that people with a high internal locus of control
optimally induce understanding through the provision of explanation react more negatively to physicians using computerized aid. Further
and information. In this regard, we need to understand under what more, Longoni et al. (2019) found that the more unique a person feels
conditions, in what contexts, and with what trade-offs (e.g., gaming the less they want to be assessed by automated systems.
respective systems; Jarrahi & Sutherland, 2019; (Lee et al., 2015);
Möhlmann et al., in press) increasing understanding of systems happens 4.3.3. Gender
(Langer et al., 2021). There is inconclusive evidence but a tendency that females react less
favorable to automated decisions. Whereas Hong et al. (2020) found no
4.2.3. Information about the developer differences between male and female participants with respect to
Information about the system developer might affect second and automated decisions involving unfair gender discrimination, Dineen
third parties. For instance, Wang et al. (2020) showed stronger negative et al. (2004) and York et al. (2020) showed stronger preferences for
effects of biased outcomes when a system was developed by an “out human versus automated decisions for female participants. Wang et al.
sourced team” compared to an in-house developer team. Similarly, (2020) showed that females react more strongly towards unfavorable
Bigman et al. (2020) found third parties perceive more moral outrage outcomes in automated decisions. Finally, Araujo et al. (2020) report
regarding biases by automated systems when the developing organiza that females tend to perceive automated decisions as less useful. Overall,
tion was describes as sexist. Finally, information about the prestige of it might be possible that unconsidered confounding variables (e.g., fa
the developer can lead to better reactions when experts use respective miliarity with respective systems) might have contributed to the re
systems to augment their decisions (Arkes et al., 2007). ported effects of gender on reactions to automated decisions.
4.3. Characteristics of second and third parties 4.4. Task characteristics
We identified experience, familiarity, education, personality and Previous research indicates that it is a mix of the perceived stakes of
traits as well as gender as characteristics of second and third parties that the task, perceptions of tasks requiring human versus mechanical skills,
moderate effects of automation and augmentation. perceived quantifiability of task-related information, and familiarity
with automated systems for the respective tasks that affect second- and
third-party perceptions. First, people might react differently to systems
14
used in high versus low-stakes contexts (Langer, König, & Papathana considering financial decisions would usually also be considered
siou, 2019). Specifically, third parties reacted more negatively to auto high-stakes. Instead, financial decisions may represent a quantifiable
mated decisions used for personnel selection (high-stakes) compared to task requiring mechanical skills, one done better by a system than by a
training situations (low-stakes) (Langer, König, & Papathanasiou, 2019). human. Familiarity with automated decisions might be another
Similarly, Longoni et al. (2019; Study 10,2 supplementary material) dimension which could explain the rather favorable reactions to auto
showed more resistance to automated systems in high-stakes medical mated decisions for movies and music. Overall, we conclude that a
decision (see also Palmisciano et al., 2020; York et al., 2020). In necessary theoretical advance to move forward is the development of a
contrast, Araujo et al. (2020) found that automated decisions are taxonomy of task differences relevant to algorithmic decision-making.
perceived as fairer in high-stakes situations in health and justice. Without such a taxonomy, distinctions like these will continue to pla
Additionally, Ötting and Maier (2018) found no differences in reaction gue unambiguous conclusions in such experiments.
to task allocation (low-stakes) and allocation of vocational training
(high-stakes) between human and automated decisions. A different 4.5. System output and decision outcome characteristics
perspective on the influence of the stakes comes from Srivastava et al.
(2019) who showed that in high-stakes contexts, people value accuracy System outputs and decision outcomes serve as stimuli for second-
more than unbiasedness of system decisions. party perceptions. We distinguish here between the outputs of a sys
Second, perceptions might differ in tasks that require human versus tem (i.e., the actual recommendation, decision) and outcomes for in
mechanical skills (M. K. Lee, 2018). For instance, human skills would be dividuals (e.g, favorable vs. unfavorable). Although research on the
subjective and intuitive judgement as well as emotional capabilities recursive effects of outputs and outcomes on attitudes, perceptions, and
whereas mechanical skills would be quick and potentially objective behaviors remains scarce, future research should investigate such effects
processing of large amounts of data (Castelo et al., 2019; M. K. Lee, given previous research on first-party perceptions showing that outputs
2018). For example, systems seem to be more accepted for image pro and outcomes affect first-party attitudes, perceptions, and behavior in
cessing tasks in medicine compared to treatment recommendations relation to systems (e.g., evaluations of trustworthiness, trust, accep
(Palmisciano et al., 2020; York et al., 2020). Furthermore, M. K. Lee tance, use) (Hoff & Bashir, 2015; J. D. Lee & See, 2004). It is likely that
(2018) found that humans perceive less fairness, less trust, and more these effects persist in the second- and third-party context.
negative emotional reactions when an automated system conducted a Hong et al. (2020) showed that biased outputs by systems can have
task that required human skills (i.e., personnel selection, work evalua comparably stronger negative effects on people’s perceptions of decision
tion) compared to tasks that required mechanical skills (i.e., work automation compared to descriptions referring to system quality.
assignment, scheduling). Unfortunately, in all above examples (imaging Additionally, Tobia et al. (2021) showed that following versus rejecting
vs. treatment recommendation; selection vs. scheduling) the respective standard versus non-standard treatment recommendations (i.e., out
tasks might not only differ with respect to requiring human skills, but puts) by automated systems differently affected physicians’ perceived
could also be perceived to differ with respect to the stakes involved. responsibility. On the side of outcomes for second parties, unfavorable
Third, related to the idea of human vs. mechanical skills, quantifi and unjust outcomes lead to negative reactions irrespective of human or
ability may influence how people react to decision automation (Nagte automated decisions (Gonzalez et al., 2019; Hong et al., 2020). For
gaal, 2021). Specifically, if it is difficult to measure task components (e. example, Gonzalez et al. (2019) found no difference in organizational
g., predictors, criteria) with face valid numbers, people seem to believe attractiveness in the case of negative outcomes (i.e., imagine a job
that it is a bad idea trying to automate it through systems using math application was rejected) but that participants preferred human de
ematical algorithms. This interpretation is in line with the reviewed cisions in the case of positive outcomes. Even expected outcome favor
findings (Lee et al., 2015; Newman et al., 2020; Ötting & Maier, 2018). ability seems to have an effect (Wang et al., 2020). Specifically, Wang
Especially, Nagtegaal’s (2021) studies support this distinction; for et al. (2020) showed that participants who expect to fail an automated
highly quantifiable, mathematically deterministic tasks (e.g., deter evaluation also perceive the general process to be less fair than the ones
mining travel reimbursement given provided documentation), she who expect to pass.
showed a decline in perceived fairness as human control increased. In
contrast, for less quantifiable (but also more complex; Nagtegaal, 2021) 5. Limitations and guidelines for future research
tasks (e.g., hiring), decision automation was perceived as least fair, and
there was no difference in perceived fairness between human and Overall, our review revealed inconsistencies in findings with respect
augmented decision-making, possibly because people did not believe to several commonly investigated attitudes and perceptions and high
that adding the system would provide any benefit to such a task. lighted important moderators (see Tables 2 and 3 for related sample
Additional support for this task dimension comes from research inves research questions). For instance, findings regarding trustworthiness
tigating first-party reactions. For first parties, previous research found attitudes (especially for the facet ability) differed between the studies,
that they reject automated decisions in uncertain domains (Dietvorst & perceptions of fairness seem to be moderated by the task at hand, and
Bharti, 2020) but prefer automated decisions for quantifiable tasks there are ambiguous effects of transparency and explanation provision.
(Castelo et al., 2019). Furthermore, few researchers explicitly examine the relationships
The complexities involved with task characteristics dimensions are among attitudes, perceptions, and behavior, generally focusing upon
illustrated by Longoni et al. (2019; Study 11, supplementary material), either perceptions, or attitudes, or behaviors. Moreover, although
who found that participants preferred human decisions throughout all research indicates that human control in decision processes alters sec
investigated application areas but that the strength of this preference ond- and third-party attitudes and perceptions, the psychological pro
differed across the areas in the following rank order from the greatest cesses underlying this effect remain open for investigation. In addition
difference between human and automated decisions to the smallest: to the inconsistencies found in the reviewed studies that hopefully
legal, health, home, fashion, home décor, restaurants, movies, music, stimulate future research, the following subsections present five broad
financial decisions. This rank order reveals that high versus low-stakes is observations regarding limitations of the reviewed studies together with
a potentially central distinction that affects human reactions to auto guidelines to advance future research on decision automation and
mated decisions, yet it is clearly not the only meaningful distinction, augmentation.
5.1. Research design that allows for generalizability

2
We thank the authors for providing us with additional unpublished results
from their study. Overall, the generalizability of the reviewed studies is unclear.
15
Specifically, most relied on vignettes, generally asking research partic automated systems at work (e.g., expert systems) might reoccur for
ipants to imagine being in or observing an automated or augmented current AI-based systems (Gill, 1995). In fact, despite expert systems
decision process. Vignette studies can be difficult to design in order to sparking significant attention in the 1980s and 1990s, they have not
draw meaningful conclusions, and they often do not accurately mirror been widely adopted in workplaces. Instead, problems transitioning
real-world processes (Atzmüller & Steiner, 2010). Furthermore, for most from development to implementation, and more problematically the
studies it remained unclear how familiar people were regarding decision lack of system acceptance by users, hindered the use of this earlier
automation or augmentation. For example, a vignette study may ask for manifestation of workplace AI (Gill, 1995). Although current AI-based
participant reactions to “AI” but if participants do not know what the automated systems may have significant advantages over expert sys
term AI means, knowledge of AI, novelty effects, technological anxiety, tems in terms of flexibility and accuracy, it is unclear if the fundamental
and a variety of other constructs are potentially confounded. Moving psychological processes affected by the use of these systems and the
forward, researchers should prioritize research designs that allow more design concerns faced in their development to address such processes
generalizable conclusions, especially because specific algorithm design are substantially different. Additionally, litigation against automated
and implementation details in real-world decision-making may have decision systems has become a concern that will likely affect attitudes
large effects on relevant outcomes for both second and third parties towards such systems (Harwell, 2019), national ethical and legal
(Galière, 2020). guidelines are evolving, and there have been several high profile ex
This reframing potentially influences interpretations of the reviewed amples of public outrage about the use of automated decisions for
research in two major ways. First, imagining how a second party might work-related decisions (Lecher, 2019). Furthermore, the new generation
feel when a decision is made about them by a first party and/or system of AI-based automated systems comes with new technical challenges,
may more directly reflect third-party reactions than second-party re such as the detection and response to biased decision-making (Raghavan
actions. A consequence of this might be that most vignette-based studies et al., 2020), the opacity of decision-processes (Burrell, 2016), and
show a preference for human over automated medical decisions, increased demand for explainability (Floridi et al., 2018).
whereas Keel et al.’s (2018) study of authentic second-party reactions To understand the effects of automated and augmented decision-
found the opposite. Second, systems are generally developed iteratively, making in this context, it is necessary to take a multi-stakeholder
refined over time through hundreds or thousands of versions to maxi view, considering their perspectives both holistically and individually
mize positive outcomes and minimize negative outcomes over a long when making research design decisions (Jungmann et al., 2020; Langer
time period for narrowly defined subpopulations (Landers & Marin, et al., 2021). As it stands currently, there is a notable stream of research
2021). Thus, development research conducted by technology companies assessing first-party acceptance (at least in certain areas of application;
and held privately often refines real-world systems for targeted outcome Burton et al., 2020; Hoff & Bashir, 2015), and our review summarizes
in ways that vignette studies of such systems cannot replicate. In short, research on second and third parties. In addition to the use of authentic
researchers may often be studying automated or augmented decision-making to enhance our understanding of those parties, a more
decision-making as researchers imagine it, not as it is realized in complete treatment of stakeholders would include supervisors of people
authentic organizational contexts. In these ways, a literature relying using automated decisions (e.g., how to best assess the work of direct
entirely upon vignettes risks being ungeneralizable to real-world reports when those use automated systems; Holstein et al., 2019), team
decision-making. members (e.g., how does the implementation of an automated decision
The most informative designs in the literature that we identified agent into a team affect team collaboration; O’Neill et al. 2020), de
were studies investigating algorithmic management in gig or platform velopers (e.g., how developers should be involved in redesign to
work and studies directly investigating the implementation of auto improve automated systems once they are implemented; Landers &
mated systems for decision (Howard et al., 2020; Keel et al., 2018; Marin, 2021), and regulatory bodies (e.g., how policy maker actions
Ravenelle, 2019). Yet, such insights predominantly stem from qualita influence these networks of relationships; Arrieta et al., 2020).
tive interviews with second parties or from the analysis of online forums
where workers exchange their experiences. Future studies might try to 5.3. Inconsistency in terminology
use experiments and quantitative analyses to broaden the insights. For
instance, algorithmic management could affect employees in organiza Substantial variation in terminology (see Table 1) between and even
tions who have more interaction with their colleagues and supervisors within studies sometimes harms the coherence of conclusions. The
compared to Uber drivers, who almost always work alone (Wesche & reviewed studies used terms including AI/ML (Gonzalez et al., 2019),
Sonderegger, 2019). Additionally, if people know that automated sys automated system (Dineen et al., 2004), super computer (Bigman &
tems will evaluate them in application processes or in performance Gray, 2018), and algorithm (Newman et al., 2020) to refer to similar
evaluations within organizations, this might lead to similar reactions concepts. Without explicitly defining terms, construct proliferation (J.
like the ones found in studies in our review (e.g., reverse engineer A. Shaffer et al., 2016) becomes a substantial and troubling risk, risking
algorithmic evaluation processes). Since the same automated system unnecessary splits in the literature and wasted researcher resources.
could be used in different organizations, it would be interesting to This problem especially applies to study designs themselves, when
investigate whether this could lead to collective action or attempts to considering the specific prompts provided to participants in vignette
share knowledge about the involved systems across organizations. For studies. Studies varied greatly in terms of the amount and clarity of
instance, similar behavior is already common in automated systems for explanatory information concerning the system studied. For instance, M.
personnel selection where there are online discussions where people K. Lee (2018) used the term “algorithm” with participants without
share knowledge and folk theories about to positively affect evaluation providing any additional information, whereas Newman et al.’s (2020)
by automated interview systems.3 participants read “algorithm (i.e., a computerized decision-making
tool)”. Similarly, if participants are more familiar with the word “com
5.2. Multi-stakeholder view puter program,” this might result in different reactions compared to
when describing the same system as an “algorithm” due to their personal
The stakes associated with accurately investigating real-world pro familiarity rather than the general idea of automated or augmented
cesses are high. Consider that many problems associated with earlier decision-making. Supporting this, Gray and Wegner (2012) show that
uneasy feelings about systems result from ascribing the ability to sense
and feel (i.e., experience) to machines but not from perceived agency of
3
https://www.reddit.com/r/recruitinghell/comments/e5eyw5/duke_univer systems. Ascribing sensing and feeling abilities might be more likely for
sity_is_preparing_students_for/. some terms (e.g., “robot”) compared to others (e.g., “computer
16
program”). As such, we believe that investigating the consequences of third parties can be exceptionally influential on organizations consid
terminological differences when referring to a system that enables de ering the implementation of systems enabling the automation or
cision automation or augmentation is important for future research, as augmentation of decisions; understanding second- and third-party re
effects across studies may vary due to such differences. sponses has never been more critical as society continues its march to
wards automation and augmentation.
5.4. Considering design-features of systems and levels of automation At the beginning of this review, we proposed three research ques
tions: (1) how do automated and augmented decisions affect second and
A large range of papers that refer to “the system” that augments or third parties? (2) what moderates these effects? and (3) what are the
automates decisions consider “the system” to be a monolithic concept. next crucial steps to meaningfully advance this research area? With
This ignores the large range of design possibilities for such systems that respect to (1) we can conclude that there are manifold influences of
might influence effects for second and third parties (Landers & Marin, decision automation and augmentation related to second and third
2021). For instance, automated systems vary in their development parties regarding topics such as trust, fairness, responsibility, control
process (e.g., what kind of training data are provided for the automated lability, and autonomy. With regard to (2), we found that those in
system), in their interface design (e.g., feedback affordances), or in their fluences appear to be qualified in numerous underexplored ways,
performance (e.g., predictive accuracy). The studies already available in especially by the characteristics of the decision process (i.e., different
this domain suggest that system design and system characteristics affect effects for automation compared to augmentation contexts), the system,
second- and third-party perceptions of automated and augmented de the second and third parties, the focal task, and outputs/outcome
cisions (Griesbach et al. 2019; Hong et al., 2020; Longoni et al., 2019; characteristics. Regarding (3), our review revealed that most of our
Ravenelle, 2019), and as such, studying this issue haphazardly is no current understanding of second- and third-party effects is driven by
longer informative to theory. vignette studies, by studies of independent contractors reporting their
Furthermore, decision processes vary in the human and system experiences with algorithmic management systems, and by generalizing
control and contribution to the decision, and the difference between from the field of medical decision-making. The literature neglects the
fully-automated and augmented decisions seems to strongly affect sec manifold system design choices that enable decision automation and
ond and third parties. However, only a subset of the various possible augmentation, making the generalizability of this literature to broader
configurations for decision augmentation (Kaber & Endsley, 2004) have work contexts with more complex decision-making systems unclear.
received attention, and even less so in the area of work-related decisions. Thus, it is critical for future research to tackle these issues of general
On the one hand, it could be possible that the exact configuration of how izability directly, by more carefully considering their choice of termi
humans and systems interact might not matter for second and third nology, research design, and methodology, as well as by integrating
parties who may have little insight into the specific decision-making interdisciplinary literature. Dramatic and substantial augmentation and
process. On the other hand, this might indicate a neglect of many of automation of work by “the system” is coming quickly, and researchers
the possible configurations between the extremes human control and must be better prepared.
full automation. Indeed, the few papers that allow a conclusion about
whether the extent of human control matters indicate that there are Credit author statement
psychological consequences that seem to be associated with either the
extent of human control or whether human decision-makers have the Markus Langer: Conceptualization, Methodology, Conducting the
final say over decisions (Newman et al., 2020). Future research should Review, Interpretation, Writing – original draft preparation, Writing –
thus explicitly investigate the implications of different levels of human Reviewing and Editing, Richard N. Landers: Conceptualization, Inter
control on second and third parties and thus contrast their potential pretation, Writing - Reviewing and editing.
psychological consequences.
Acknowledgements
5.5. Theoretical integration
Work on this paper was funded by the Volkswagen Foundation grant
The reviewed research either refrained from referring to a broader AZ 98513.
theoretical frame or mainly focused on justice or signalling theory
(Bangerter et al., 2012; Colquitt et al., 2001) to derive hypotheses and to References
embed into the research landscape. Work characteristics and work
design research (Parker & Grote, 2020) might provide another valuable Acikgoz, Y., Davison, K. H., Compagnone, M., & Laske, M. (2020). Justice perceptions of
artificial intelligence in selection. International Journal of Selection and Assessment, 28
theoretical lens for the implementation of decision automation and (4), 399–416. https://doi.org/10.1111/ijsa.12306
augmentation into work contexts (Langer, König, & Busch, 2020) but is Ananny, M., & Crawford, K. (2018). Seeing without knowing: Limitations of the
currently underutilized. Because those theories focus on work processes transparency ideal and its application to algorithmic accountability. New Media &
Society, 20(3), 973–989. https://doi.org/10.1177/1461444816676645
and not necessarily on system design or decision-process-based conse Anwar, I. A., Pal, J., & Hui, J. (2021). Watched, but moving: Platformization of beauty
quences that affect psychological processes, future researchers should work and its gendered mechanisms of control. Proceedings of the ACM on Human-
focus upon the creation of interdisciplinary theory. This involves Computer Interaction, 4, 1–20. https://doi.org/10.1145/3432949
Araujo, T., Helberger, N., Krulkmeier, S., de Vreese, C. H., et al. (2020). In AI we trust?
combining the technical and system design insights of computer science Perceptions about automated decision‑making by artificial intelligence. AI & Society,
and human-computer interaction with the human processes insights of 35(3), 611–623. https://doi.org/10.1007/s00146-019-00931-w
psychology and the applied theory insights of management to enable Arkes, H. R., Shaffer, V. A., & Medow, M. A. (2007). Patients derogate physicians who
use a computer-assisted diagnostic aid. Medical Decision Making, 27(2), 189–202.
more generalizable work for understanding decision automation and https://doi.org/10.1177/0272989X06297391
augmentation (Landers & Marin, 2021). Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A.,
Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020).
Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and
6. Conclusion
challenges toward responsible AI. Information Fusion, 58, 82–115. https://doi.org/
10.1016/j.inffus.2019.12.012
Research in the area of automated and augmented decision-making Atzmüller, C., & Steiner, P. M. (2010). Experimental vignette studies in survey research.
has focused principally upon first parties (Burton et al., 2020; Grove & Methodology, 6(3), 128–138. https://doi.org/10.1027/1614-2241/a000014
Bangerter, A., Roulin, N., & König, C. J. (2012). Personnel selection as a signaling game.
Meehl, 1996; Hoff & Bashir, 2015). This review extends this research by Journal of Applied Psychology, 97(4), 719–738. https://doi.org/10.1037/a0026078
investigating the effects of decision automation and augmentation on
second and third parties. In our highly connected society, second and
17
Benbasat, I., & Nault, B. R. (1990). An evaluation of empirical research in managerial Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus
support systems. Decision Support Systems, 6(3), 203–226. https://doi.org/10.1016/ mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30.
0167-9236(90)90015-J https://doi.org/10.1037//1040-3590.12.1.19
Bigman, Y. E., & Gray, K. (2018). People are averse to machines making moral decisions. Haan, M., Ongena, Y. P., Hommes, S., Kwee, T. C., & Yakar, D. (2019). A qualitative
Cognition, 181, 21–34. https://doi.org/10.1016/j.cognition.2018.08.003 study to understand patient perspective on the use of artificial intelligence in
Bigman, Y. E., Gray, K., Waytz, A., Arnestad, M., & Wilson, D. (2020). Algorithmic radiology. Journal of the American College of Radiology, 16(10), 1416–1419. https://
discrimination causes less moral outrage than human discrimination [Preprint]. https:// doi.org/10.1016/j.jacr.2018.12.043
doi.org/10.31234/osf.io/m3nrp. PsyArXiv. Hamilton, J. G., Genoff Garzon, M., Westerman, J. S., Shuk, E., Hay, J. L., Walters, C., …
Binns, R., van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018). “It”s Kris, M. G. (2019). “A tool, not a crutch”: Patient perspectives about IBM Watson for
reducing a human being to a percentage’; Perceptions of justice in algorithmic oncology trained by memorial sloan kettering. Journal of Oncology Practice, 15(4),
decisions. Proceedings of the CHI 2018 Conference on Human Factors in Computing e277–e288. https://doi.org/10.1200/JOP.18.00417
Systems. https://doi.org/10.31235/osf.io/9wqxr Harwell, D. (2019). Rights group files federal complaint against AI-hiring firm HireVue, citing
Bucher, E. L., Schou, P. K., & Waldkirch, M. (2021). Pacifying the algorithm – ‘unfair and deceptive’ practices. The Washington Post. https://www.washingtonpost.
Anticipatory compliance in the face of algorithmic management in the gig economy. com/technology/2019/11/06/prominent-rights-group-files-federal-complaint-aga
Organization, 1, 44–67. https://doi.org/10.1177/1350508420961531 inst-ai-hiring-firm-hirevue-citing-unfair-deceptive-practices/.
Burrell, J. (2016). How the machine “thinks”: Understanding opacity in machine Healy, J., Pekarek, A., & Vromen, A. (2020). Sceptics or supporters? Consumers’ views of
learning algorithms. Big Data & Society, 3(1), Article 205395171562251. https://doi. work in the gig economy. New Technology, Work and Employment, 35(1), 1–19.
org/10.1177/2053951715622512 https://doi.org/10.1111/ntwe.12157
Burton, J. W., Stein, M., & Jensen, T. B. (2020). A systematic review of algorithm Highhouse, S. (2008). Stubborn reliance on intuition and subjectivity in employee
aversion in augmented decision making. Journal of Behavioral Decision Making, 33(2), selection. Industrial and Organizational Psychology, 1(3), 333–342. https://doi.org/
220–239. https://doi.org/10.1002/bdm.2155 10.1111/j.1754-9434.2008.00058.x
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019). Task-dependent algorithm aversion. Höddinghaus, M., Sondern, D., & Hertel, G. (2020). The automation of leadership
Journal of Marketing Research, 56(5), 809–825. https://doi.org/10.1177/ functions: Would people trust decision algorithms? Computers in Human Behavior,
0022243719851788 116, 106635. https://doi.org/10.1016/j.chb.2020.106635
Colquitt, J. A. (2001). On the dimensionality of organizational justice: A construct Hoff, K. A., & Bashir, M. (2015). Trust in automation: Integrating empirical evidence on
validation of a measure. Journal of Applied Psychology, 86(3), 386–400. https://doi. factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/
org/10.1037/0021-9010.86.3.386 10.1177/0018720814547570
Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O. L. H., & Ng, K. Y. (2001). Justice Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019).
at the millennium: A meta-analytic review of 25 years of organizational justice Improving fairness in machine learning systems: What do industry practitioners
research. Journal of Applied Psychology, 86(3), 425–445. https://doi.org/10.1037// need? Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems,
0021-9010.86.3.425 1–16. https://doi.org/10.1145/3290605.3300830
Colquitt, J. A., & Rodell, J. B. (2011). Justice, trust, and trustworthiness: A longitudinal Hong, J.-W., Choi, S., & Williams, D. (2020). Sexist AI: An experiment integrating CASA
analysis integrating three theoretical perspectives. Academy of Management Journal, and ELM. International Journal of Human-Computer Interaction, 36(20), 1928–1941.
54(6), 1183–1206. https://doi.org/10.5465/amj.2007.0572 https://doi.org/10.1080/10447318.2020.1801226
Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Howard, F. M., Gao, C. A., & Sankey, C. (2020). Implementation of an automated
Science, 243(4899), 1668–1674. https://doi.org/10.1126/science.2648573 scheduling tool improves schedule quality and resident satisfaction. PloS One, 15(8),
Dietvorst, B. J., & Bharti, S. (2020). People reject algorithms in uncertain decision Article e0236952. https://doi.org/10.1371/journal.pone.0236952
domains because they have diminishing sensitivity to forecasting error. Psychological Jarrahi, M. H., & Sutherland, W. (2019). Algorithmic management and algorithmic
Science, 31(10), 1302–1314. https://doi.org/10.1177/0956797620948841 competencies: Understanding and appropriating algorithms in gig work. In
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People N. G. Taylor, C. Christian-Lamb, M. H. Martin, & B. Nardi (Eds.), Information in
erroneously avoid algorithms after seeing them err. Journal of Experimental contemporary society, lecture notes in computer science (11420th ed., pp. 578–589).
Psychology: General, 144(1), 114–126. https://doi.org/10.1037/xge0000033 Springer. https://doi.org/10.1007/978-3-030-15742-5_55.
Dineen, B. R., Noe, R. A., & Wang, C. (2004). Perceived fairness of web-based applicant Jarrahi, M. H., Sutherland, W., Nelson, S. B., & Sawyer, S. (2020). Platformic
screening procedures: Weighing the rules of justice and the role of individual management, boundary resources for gig work, and worker autonomy. Computer
differences. Human Resource Management, 43(2–3), 127–145. https://doi.org/ Supported Cooperative Work, 29(1–2), 153–189. https://doi.org/10.1007/s10606-
10.1002/hrm.20011 019-09368-7
van Dongen, K., & van Maanen, P.-P. (2013). A framework for explaining reliance on Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines.
decision aids. International Journal of Human-Computer Studies, 71(4), 410–424. Nature Machine Intelligence, 1(9), 389–399. https://doi.org/10.1038/s42256-019-
https://doi.org/10.1016/j.ijhcs.2012.10.018 0088-2
Duggan, J., Sherman, U., Carbery, R., & McDonnell, A. (2020). Algorithmic management Jonmarker, O., Strand, F., Brandberg, Y., & Lindholm, P. (2019). The future of breast
and app-work in the gig economy: A research agenda for employment relations and cancer screening: What do participants in a breast cancer screening program think
HRM. Human Resource Management Journal, 30(1), 114–132. https://doi.org/ about automation using artificial intelligence? Acta Radiologica Open, 8(12). https://
10.1111/1748-8583.12258 doi.org/10.1177/2058460119880315, 205846011988031.
Fazio, R. H., & Williams, C. J. (1986). Attitude accessibility as a moderator of the Jungmann, F., Jorg, T., Hahn, F., Pinto dos Santos, D., Jungmann, S. M., Düber, C.,
attitude–perception and attitude–behavior relations: An investigation of the 1984 Mildenberger, P., & Kloeckner, R. (2020). Attitudes toward artificial intelligence among
presidential election. Journal of Personality and Social Psychology, 51(3), 505–514. radiologists, IT specialists, and industry. Academic Radiology. https://doi.org/
https://doi.org/10.1037/0022-3514.51.3.505 10.1016/j.acra.2020.04.011. S1076633220302038.
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Jutzi, T. B., Krieghoff-Henning, E. I., Holland-Letz, T., Utikal, J. S., Hauschild, A.,
Madelin, R., Pagallo, U., Rossi, F., Schafer, B., Valcke, P., & Vayena, E. (2018). Schadendorf, D., Sondermann, W., Fröhling, S., Hekler, A., Schmitt, M., Maron, R. C.,
AI4People—an ethical framework for a good AI society: Opportunities, risks, & Brinker, T. J. (2020). Artificial intelligence in skin cancer diagnostics: The
principles, and recommendations. Minds and Machines, 28(4), 689–707. https://doi. patients’ perspective. Frontiers of Medicine, 7. https://doi.org/10.3389/
org/10.1007/s11023-018-9482-5 fmed.2020.00233
Galière, S. (2020). When food-delivery platform workers consent to algorithmic Kaber, D. B., & Endsley, M. R. (2004). The effects of level of automation and adaptive
management: A Foucauldian perspective. New Technology, Work and Employment, 35 automation on human performance, situation awareness and workload in a dynamic
(3), 357–370. https://doi.org/10.1111/ntwe.12177 control task. Theoretical Issues in Ergonomics Science, 5(2), 113–153. https://doi.org/
Gill, T. G. (1995). Early expert systems: Where are they now? MIS Quarterly, 19(1), 51. 10.1080/1463922021000054335
https://doi.org/10.2307/249711 Keel, S., Lee, P. Y., Scheetz, J., Li, Z., Kotowicz, M. A., MacIsaac, R. J., & He, M. (2018).
Gonzalez, M. F., Capman, J. F., Oswald, F. L., Theys, E. R., & Tomczak, D. L. (2019). Feasibility and patient acceptability of a novel artificial intelligence-based screening
“Where’s the I-O?” Artificial intelligence and machine learning in talent model for diabetic retinopathy at endocrinology outpatient services: A pilot study.
management systems. Personnel Assessment and Decisions, 3, 5. https://doi.org/ Scientific Reports, 8(1). https://doi.org/10.1038/s41598-018-22612-2
10.25035/pad.2019.03.005 Kellogg, K. C., Valentine, M. A., & Christin, A. (2020). Algorithms at work: The new
Gray, K., & Wegner, D. M. (2012). Feeling robots and human zombies: Mind perception contested terrain of control. The Academy of Management Annals, 14(1), 366–410.
and the uncanny valley. Cognition, 125(1), 125–130. https://doi.org/10.1016/j. https://doi.org/10.5465/annals.2018.0174
cognition.2012.06.007 Kinder, E., Jarrahi, M. H., & Sutherland, W. (2019). Gig platforms, tensions, alliances and
Grgić-Hlača, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018). Human ecosystems: An actor-network perspective. Proceedings of the ACM on Human-
perceptions of fairness in algorithmic decision making: A case study of criminal risk Computer Interaction, 3, 1–26. https://doi.org/10.1145/3359314
prediction. In Proceedings of the 2018 world wide web conference on world wide web Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus
(pp. 903–912). https://doi.org/10.1145/3178876.3186138 clinical data combination in selection and admissions decisions: A meta-analysis.
Griesbach, K., Reich, A., Elliott-Negri, L., & Milkman, R. (2019). Algorithmic control in Journal of Applied Psychology, 98(6), 1060–1072. https://doi.org/10.1037/a0034156
platform food delivery work. Socius: Sociological Research for a Dynamic World, 5, Landers, R. N., & Marin, S. (2021). Theory and technology in organizational psychology:
Article 237802311987004. https://doi.org/10.1177/2378023119870041 A review of technology integration paradigms and their effects on the validity of
Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, theory. Annual Review of Organizational Psychology and Organizational Behavior, 8(1),
impressionistic) and formal (mechanical, algorithmic) prediction procedures: The 235–258. https://doi.org/10.1146/annurev-orgpsych-012420-060843
clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293–323. Langer, M., König, C. J., & Busch, V. (2020). Changing the means of managerial work:
https://doi.org/10.1037/1076-8971.2.2.293 Effects of automated decision-support systems on personnel selection tasks. In
18
Journal of business and psychology. Advance Online Publication. https://doi.org/ Newman, D. T., Fast, N. J., & Harmon, D. J. (2020). When eliminating bias isn’t fair:
10.1007/s10869-020-09711-6. Algorithmic reductionism and procedural justice in human resource decisions.
Langer, M., König, C. J., & Fitili, A. (2018). Information as a double-edged sword: The Organizational Behavior and Human Decision Processes, 160, 149–167. https://doi.
role of computer experience and information on applicant reactions towards novel org/10.1016/j.obhdp.2020.03.008
technologies for personnel selection. Computers in Human Behavior, 81, 19–30. Nolan, K., Carter, N., & Dalal, D. (2016). Threat of technological unemployment: Are
https://doi.org/10.1016/j.chb.2017.11.036 hiring managers discounted for using standardized employee selection practices?
Langer, M., König, C. J., & Hemsing, V. (2020). Is anybody listening? The impact of Personnel Assessment and Decisions, 2(1), 4. https://doi.org/10.25035/pad.2016.004
automatically evaluated job interviews on impression management and applicant O’Neill, T., McNeese, N., Barron, A., & Schelble, B. (2020). Human–autonomy teaming: A
reactions. Journal of Managerial Psychology, 35(4), 271–284. https://doi.org/ review and analysis of the empirical literature. Human Factors. Advance Online
10.1108/JMP-03-2019-0156 Publication.. https://doi.org/10.1177/0018720820960865, 001872082096086.
Langer, M., König, C. J., & Papathanasiou, M. (2019). Highly-automated job interviews: Onnasch, L., Wickens, C. D., Li, H., & Manzey, D. (2014). Human performance
Acceptance under the influence of stakes. International Journal of Selection and consequences of stages and levels of automation: An integrated meta-analysis.
Assessment, 27(3), 217–234. https://doi.org/10.1111/ijsa.12246 Human Factors, 56(3), 476–488. https://doi.org/10.1177/0018720813501549
Langer, M., König, C. J., Sanchez, D. R.-P., & Samadi, S. (2019). Highly automated Ötting, S. K., & Maier, G. W. (2018). The importance of procedural justice in human-
interviews: Applicant reactions and the organizational context. Journal of Managerial machine-interactions: Intelligent systems as new decision agents in organizations.
Psychology, 35(4), 301–314. https://doi.org/10.1108/JMP-09-2018-0402 Computers in Human Behavior, 89, 27–39. https://doi.org/10.1016/j.
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & chb.2018.07.022
Baum, K. (2021). What do we want from explainable artificial intelligence (XAI)? A Palmeira, M., & Spassova, G. (2015). Consumer reactions to professionals who use
stakeholder perspective on XAI and a conceptual model guiding interdisciplinary decision aids. European Journal of Marketing, 49(3/4), 302–326. https://doi.org/
XAI research. Artificial Intelligence, 296, 103473. https://doi.org/10.1016/j. 10.1108/EJM-07-2013-0390
artint.2021.103473 Palmisciano, P., Jamjoom, A. A. B., Taylor, D., Stoyanov, D., & Marcus, H. J. (2020).
Lecher, C. (2019). How Amazon automatically tracks and fires warehouse workers for Attitudes of patients and their relatives toward Artificial Intelligence in
‘productivity. The Verge. https://www.theverge.com/2019/4/25/18516004/amazo neurosurgery. World Neurosurgery, 138, e627–e633. https://doi.org/10.1016/j.
n-warehouse-fulfillment-centers-productivity-firing-terminations. wneu.2020.03.029
Lee, M. K. (2018). Understanding perception of algorithmic decisions: Fairness, trust, and Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse,
emotion in response to algorithmic management. Big Data & Society, 5(1). https:// abuse. Human Factors, 39(2), 230–253. https://doi.org/10.1518/
doi.org/10.1177/2053951718756684, 205395171875668. 001872097778543886
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels
Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50.30392 of human interaction with automation. IEEE Transactions on Systems, Man, and
Lee, M. K., Kusbit, D., Metsky, E., & Dabbish, L. (2015). Working with machines: The Cybernetics - Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/
impact of algorithmic and data-driven management on human workers. Proceedings 3468.844354
of the 2015 CHI Conference on Human Factors in Computing Systems. https://doi.org/ Parker, S., & Grote, G. (2020). Automation, algorithms, and beyond: Why work design
10.1145/2702123.2702548 matters more than ever in a digital world. Applied Psychology. Advance Online
Lombrozo, T. (2011). The instrumental value of explanations. Philosophy Compass, 6(8), Publication.. https://doi.org/10.1111/apps.12241
539–551. https://doi.org/10.1111/j.1747-9991.2011.00413.x Pezzo, M. V., & Pezzo, S. P. (2006). Physician evaluation after medical errors: Does
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019). Resistance to medical artificial having a computer decision aid help or hurt in hindsight? Medical Decision Making,
intelligence. Journal of Consumer Research, 46(4), 629–650. https://doi.org/ 26(1), 48–56. https://doi.org/10.1177/0272989X05282644
10.1093/jcr/ucz013 Promberger, M., & Baron, J. (2006). Do patients trust computers? Journal of Behavioral
Lowe, D. J., Reckers, P. M. J., & Whitecotton, S. M. (2002). The effects of decision-aid use Decision Making, 19(5), 455–468. https://doi.org/10.1002/bdm.542
and reliability on jurors’ evaluations of auditor liability. The Accounting Review, 77 Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020). Mitigating bias in
(1), 185–202. https://doi.org/10.2308/accr.2002.77.1.185 algorithmic hiring: Evaluating claims and practices. Proceedings of the 2020 FAT*
Makarius, E. E., Mukherjee, D., Fox, J. D., & Fox, A. K. (2020). Rising with the machines: conference on fairness, accountability and transparency. https://doi.org/10.1145/
A sociotechnical framework for bringing artificial intelligence into the organization. 3351095.3372828
Journal of Business Research, 120, 262–273. https://doi.org/10.1016/j. Raisch, S., & Krakowski, S. (2021). Artificial intelligence and management: The
jbusres.2020.07.045 automation-augmentation paradox. Academy of Management Review, 46(1), 192–210.
Marcinkowski, F., Kieslich, K., Starke, C., & Lünich, M. (2020). Implications of AI (un-) https://doi.org/10.5465/amr.2018.0072
fairness in higher education admissions: The effects of perceived AI (un-)fairness on Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., … Barnes, P.
exit, voice and organizational reputation. Proceedings of the 2020 FAT* conference on (2020). Closing the AI accountability gap: Defining an end-to-end framework for
fairness, accountability and transparency. https://doi.org/10.1145/3351095.3372867 internal algorithmic auditing. Proceedings of the 2020 FAT* conference on fairness,
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of accountability and transparency. https://doi.org/10.1145/3351095.3372873
organizational trust. Academy of Management Review, 20(2), 709–726. https://doi. Ravenelle, A. J. (2019). “We’re not uber:” control, autonomy, and entrepreneurship in
org/10.2307/258792 the gig economy. Journal of Managerial Psychology, 34(4), 269–285. https://doi.org/
Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review 10.1108/JMP-06-2018-0256
of the evidence. University of Minnesota Press. Schlicker, N., Langer, M., Ötting, S. K., König, C. J., Baum, K., & Wallach, D. (2021).
Merritt, S. M., Unnerstall, J. L., Lee, D., & Huber, K. (2015). Measuring individual What to expect from opening “black boxes”? Comparing perceptions of justice
differences in the perfect automation schema. Human Factors, 57(5), 740–753. between human and automated agents. Computers in Human Behavior, 122, Article
https://doi.org/10.1177/0018720815581247 106837. https://doi.org/10.1016/j.chb.2021.106837
Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Schmoll, R., & Bader, V. (2019). Who or what screens which one of me? The differential
Artificial Intelligence, 267, 1–38. https://doi.org/10.1016/j.artint.2018.07.007 effects of algorithmic social media screening on applicants’ job pursuit intention.
Mirowska, A. (2020). AI evaluation in selection: Effects on application and pursuit Proceedings of the ICIS 2019.
intentions. Journal of Personnel Psychology, 19(3), 142–149. https://doi.org/ Shaffer, J. A., DeGeest, D., & Li, A. (2016). Tackling the problem of construct
10.1027/1866-5888/a000258 proliferation: A guide to assessing the discriminant validity of conceptually related
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of constructs. Organizational Research Methods, 19(1), 80–110. https://doi.org/
algorithms: Mapping the debate. Big Data & Society, 3(2), Article 205395171667967. 10.1177/1094428115598239
https://doi.org/10.1177/2053951716679679 Shaffer, V. A., Probst, C. A., Merkle, E. C., Arkes, H. R., & Medow, M. A. (2013). Why do
Möhlmann, M., & Zalmanson, L. (2017). Hands on the Wheel: Navigating algorithmic patients derogate physicians who use a computer-based diagnostic support system?
management and Uber drivers’ autonomy. In Proceedings of the 2017 international Medical Decision Making, 33(1), 108–118. https://doi.org/10.1177/
conference on information system. https://aisel.aisnet.org/icis2017/DigitalPlatforms/ 0272989X12453501
Presentations/3. Shin, D., & Park, Y. J. (2019). Role of fairness, accountability, and transparency in
Möhlmann, M., Zalmanson, L., & Gregory, R. W. (in press). Algorithmic management of algorithmic affordance. Computers in Human Behavior, 98, 277–284. https://doi.org/
work on online labor platforms: When matching meets control. MIS Quarterly. 10.1016/j.chb.2019.04.019
Advance Online Publication. Srivastava, M., Heidari, H., & Krause, A. (2019). Mathematical notions vs. human
Murray, A., Rhymer, J., & Sirmon, D. G. (2020). Humans and technology: Forms of perception of fairness: A descriptive approach to fairness for machine learning.
conjoined agency in organizations. Academy of Management Review. Advance Online Proceedings of the 2019 ACM SIGKDD International Conference on Knowledge Discovery
Publication. https://doi.org/10.5465/amr.2019.0186 & Data Mining, 2459–2468. https://doi.org/10.1145/3292500.3330664
Myhill, K., Richards, J., & Sang, K. (2021). Job quality, fair work and gig work: The lived Stai, B., Heller, N., McSweeney, S., Rickman, J., Blake, P., Vasdev, R., Edgerton, Z.,
experience of gig workers. In International journal of human resource management. Tejpaul, R., Peterson, M., Rosenberg, J., Kalapara, A., Regmi, S.,
Advance Online Publication. https://doi.org/10.1080/09585192.2020.1867612. Papanikolopoulos, N., & Weight, C. (2020). Public perceptions of artificial
Nagtegaal, R. (2021). The impact of using algorithms for managerial decisions on public intelligence and robotics in medicine. Journal of Endourology, 34(10), 1041–1048.
employees’ procedural justice. Government Information Quarterly, 38(1), 101536. https://doi.org/10.1089/end.2020.0137
https://doi.org/10.1016/j.giq.2020.101536 Suen, H.-Y., Chen, M. Y.-C., & Lu, S.-H. (2019). Does the use of synchrony and artificial
Nelson, C. A., Pérez-Chada, L. M., Creadore, A., Li, S. J., Lo, K., Manjaly, P., intelligence in video interviews affect interview ratings and applicant attitudes?
Pournamdari, A. B., Tkachenko, E., Barbieri, J. S., Ko, J. M., Menon, A. V., Computers in Human Behavior, 98, 93–101. https://doi.org/10.1016/j.
Hartman, R. I., & Mostaghimi, A. (2020). Patient perspectives on the use of artificial chb.2019.04.012
intelligence for skin cancer screening: A qualitative study. JAMA Dermatology, 156
(5), 501. https://doi.org/10.1001/jamadermatol.2019.5014
19
Tassinari, A., & Maccarrone, V. (2020). Riders on the storm: Workplace solidarity among Wesche, J. S., & Sonderegger, A. (2019). When computers take the lead: The automation
gig economy couriers in Italy and the UK. Work, Employment & Society, 34(1), 35–54. of leadership. Computers in Human Behavior, 101, 197–209. https://doi.org/
https://doi.org/10.1177/0950017019862954 10.1016/j.chb.2019.07.027
Tobia, K., Nielsen, A., & Stremitzer, A. (2021). When does physician use of AI increase Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a
liability? Journal of Nuclear Medicine, 62(1), 17–21. https://doi.org/10.2967/ replication in software engineering. Proceedings of the 18th International Conference
jnumed.120.256032 on Evaluation and Assessment in Software Engineering - EASE, 14, 1–10. https://doi.
Uhde, A., Schlicker, N., Wallach, D. P., & Hassenzahl, M. (2020). Fairness and decision- org/10.1145/2601248.2601268
making in collaborative shift scheduling systems. Proceedings of the 2020 CHI Wolf, J. R. (2014). Do IT students prefer doctors who use IT? Computers in Human
Conference on Human Factors in Computing Systems, 1–13. https://doi.org/10.1145/ Behavior, 35, 287–294. https://doi.org/10.1016/j.chb.2014.03.020
3313831.3376656 Wood, A. J., Graham, M., Lehdonvirta, V., & Hjorth, I. (2019). Good gig, bad gig:
van Esch, P., & Black, J. S. (2019). Factors that influence new generation candidates to Autonomy and algorithmic control in the global gig economy. Work, Employment &
engage with and complete digital, AI-enabled recruiting. Business Horizons, 62(6), Society, 33(1), 56–75. https://doi.org/10.1177/0950017018785616
729–739. https://doi.org/10.1016/j.bushor.2019.07.004 Yokoi, R., Eguchi, Y., Fujita, T., & Nakayachi, K. (2020). Artificial intelligence is trusted
Veen, A., Barratt, T., & Goods, C. (2020). Platform-capital’s ‘app-etite’ for control: A less than a doctor in medical treatment decisions: Influence of perceived care and
labour process analysis of food-delivery work in Australia. Work, Employment & value similarity. International Journal of Human-Computer Interaction, 1–10. https://
Society, 34(3), 388–406. https://doi.org/10.1177/0950017019836911 doi.org/10.1080/10447318.2020.1861763
Wang, R., Harper, F. M., & Zhu, H. (2020). Factors influencing perceived fairness in York, T., Jenney, H., & Jones, G. (2020). Clinician and computer: A study on patient
algorithmic decision-making algorithm outcomes, development procedures, and perceptions of artificial intelligence in skeletal radiography. BMJ Health & Care
individual differences. Proceedings of the 2018 CHI Conference on Human Factors in Informatics, 27(3), Article e100233. https://doi.org/10.1136/bmjhci-2020-100233
Computing Systems, 14. https://doi.org/10.1145/3313831.3376813 Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2018). Transparency in algorithmic
and human decision-making: Is there a double standard? Philosophy & Technology, 32
(4), 661–683. https://doi.org/10.1007/s13347-018-0330-6
20

1 s2.0 S0747563221002016 Main

Uploaded by

Copyright:

Available Formats

1 s2.0 S0747563221002016 Main

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1 s2.0 S0747563221002016 Main

Uploaded by

Copyright:

Available Formats

Computers in Human Behavior 123 (2021) 106878

Contents lists available at ScienceDirect

Computers in Human Behavior

The future of artificial intelligence at work: A review on effects of decision

1. Introduction decision-making in medicine and management (Burton et al., 2020;

Decision-makingat work (general)

evaluation selects top candidates

Characteristics of the decision-making process

4.3. Characteristics of second and third parties 4.4. Task characteristics

5.1. Research design that allows for generalizability

You might also like