Explainable AI: Beware of Inmates Running the Asylum

Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences

Tim Miller∗ and Piers Howe† and Liz Sonenberg∗

∗
School of Computing and Information Systems
†
Melbourne School of Psychological Sciences
University of Melbourne, Australia
{tmiller,pdhowe,l.sonenberg}@unimelb.edu.au
arXiv:1712.00547v2 [cs.AI] 5 Dec 2017

Abstract portant in the 80s and 90s in expert systems

particularly; see [Chandrasekaran et al., 1989],
In his seminal book The Inmates are Running [Swartout and Moore, 1993], and
the Asylum: Why High-Tech Products Drive Us [Buchanan and Shortliffe, 1984]. High visibility of
Crazy And How To Restore The Sanity [2004,
the term, sometimes abbreviated XAI, is seen in
Sams Indianapolis, IN, USA], Alan Cooper ar-
grant solicitations [DARPA, 2016] and in the popular
gues that a major reason why software is of-
press [Nott, 2017]. One area of explainable AI receiving
ten poorly designed (from a user perspective)
attention is explicit explanation, on which we say more
is that programmers are in charge of design de-
below.
cisions, rather than interaction designers. As a
result, programmers design software for them- While the title of the paper is deliberately tongue-in-
selves, rather than for their target audience; a cheek, the parallels with Cooper [2004] are real: leaving
phenomenon he refers to as the ‘inmates run- decisions about what constitutes a good explanation of
ning the asylum’. This paper argues that ex- complex decision-making models to the experts who un-
plainable AI risks a similar fate. While the re- derstand these models the best is likely to result in fail-
emergence of explainable AI is positive, this ure in many cases. Instead, models should be built on an
paper argues most of us as AI researchers understanding of explanation, and should be evaluated
are building explanatory agents for ourselves, using data from human behavioural studies.
rather than for the intended users. But ex- In Section 2, we describe a simple scan of the 23 ar-
plainable AI is more likely to succeed if re- ticles posted as ‘Related Work’ on the workshop web
searchers and practitioners understand, adopt, page. We looked at two attributes: whether the pa-
implement, and improve models from the vast pers were built on research from philosophy, psychology,
and valuable bodies of research in philosophy, cognitive science, or human factors; and whether the re-
psychology, and cognitive science; and if evalu- ported evaluations involved human behavioural studies.
ation of these models is focused more on people The outcome of this scan supports the hypothesis that
than on technology. From a light scan of litera- ideas from social sciences and human factors are not suf-
ture, we demonstrate that there is considerable ficiently visible in the field.
scope to infuse more results from the social and In Section 3, we present some key bodies of work on ex-
behavioural sciences into explainable AI, and planation and related topics from social and behavioural
present some key results from these fields that sciences that will be of interest to those in explainable
are relevant to explainable AI. AI, and briefly discuss what their impact could be.

1 Introduction 2 Explainable AI Survey

“Causal explanation is first and foremost a To gather some data to test the hypothesis that the so-
form of social interaction. One speaks of giving cial sciences and human behavioural studies are not hav-
causal explanations, but not attributions, per- ing enough impact in explainable AI, a short literature
ceptions, comprehensions, categorizations, or survey was undertaken. This survey is not intended to
memories. The verb to explain is a three- be even close to comprehensive – it is merely illustrative.
place predicate: Someone explains something However, the results that it shows are reflective of many
to someone. Causal explanation takes the form other papers in the area that the authors have read.
of conversation and is thus subject to the rules
of conversation.” — Hilton [1990]. 2.1 Selected Papers
The term “explainable AI” has regained trac- The articles surveyed were taken from the ‘Related
tion again recently, after being considered im- Work’ list that was posted on the website for the IJ-
CAI 2017 Explainable AI workshop1 as of 16 May 2017 A score of 2 was given if and only if (a), (b), and
— the workshop to which this paper is submitted. In (c) above held, and the survey article (not the refer-
total 23 articles were on the list, although one was not enced article) described an algorithm for automat-
included in the results as described later. This list can ically generating explanations and this algorithm
be found in Appendix A. was derived from data from the social sciences. In
As noted already, this list is far from comprehensive, other words, the algorithm is explicitly based on a
however, it is a useful list for two reasons: model from one or more of the references.
1. First, it was compiled by the explainable AI com- A score of 0 was given for any other paper; that is,
munity: the organisers of the conference requested no references satisfying (a), (b), and (c).
that people send related papers to be added to the 3. Validation: Each paper was given a binary 0/1. A
list. As such, it represents at least a subset of what score of 1 was given if and only if the evaluation
the community see at the moment as highly relevant in the survey article (note, not the referenced ar-
papers for researchers in explainable AI. ticle) was based on data from human behavioural
2. Second, it is objective from the perspective of the studies. Even if the algorithm is categorised as data
authors of this paper. We did not contribute to the driven, we argue that it is still important to test
list, so the selection is not biased by our argument. that the assumptions and trade-offs made are suit-
able. It is therefore necessary to (eventually) per-
While the authors of some of the listed papers may form behavioural studies to test if the explanations
not consider their work as explainable AI, almost all of produced by the algorithm are appropriate for hu-
the papers were describing methods for automatically mans.
generating explanations of some type.
The paper that was excluded is Tania Lombrozo’s sur- 2.3 Results
vey paper on explanation research in cognitive science
Table 1 shows the results for the survey. Results for each
[Lombrozo, 2012]. This is not an explainable AI paper –
of the surveyed articles are available in Appendix B. Five
indeed, it summarises one of the bodies of work of which papers were deemed ‘off topic’, however, the results are
we argue people should be more aware.
included because we could not know the intent of those
who submitted articles to the reading list. For the ‘Data
2.2 Survey Method driven’ entry, column ‘N’ means that we were unsure
The survey was lightweight: it only looked for evidence about the reference. In this case, one paper had a ref-
that the presented research was somehow influenced by erence to a cognitive science article of which we were
a scientific understanding of explanations, and that the unable to locate a copy. For the ‘Validation’ entry, col-
evaluations were performed using data derived from hu- umn ‘N’ means ‘not applicable’: three papers were cat-
man behaviour studies or similar. We categorised the egorised as not applicable because their status were not
papers on the three items of interest, with the criteria research articles, but review articles or position papers,
for the scores as follows: and thus, they did not present any algorithm or model
to evaluate.
1. On topic: Each paper was categorised as either be-
ing about explainable AI or not, based on our un-
derstanding of the topic. It is possible that some On topic Off topic
papers were included on the workshop website be- (17 articles) (5 articles)
cause they presented good challenges or potentially Criterion N 0 1 2 N 0 1 2
useful approaches, but were not papers about ex-
planation per se, in which case they were ‘off topic’. Data driven 1 11 4 1 0 4 0 1
Validation 3 10 4 — 0 4 1 —
2. Data Driven: Each paper was given a score from
0–2 inclusive. Table 1: Results on small survey
A score of 1 was given if and only if one or more
of the references of the paper was an article on ex-
These results show that for the on-topic papers, only
planation in social science, meaning that: (a) expla-
four articles referenced relevant social science research,
nation or causal attribution as done by humans is
and only one of them truly built a model on this. Fur-
one of the main topics of the referenced article(s);
ther, serious human behavioural experiments are not
(b) the referenced article(s) validated their claims
currently being undertaken. For off topic papers, the re-
using data collected from human behaviour experi-
sults are similar: limited input from social sciences and
ments; and (c) the referenced article(s) appear in a
limited human behavioural experiments.
non-computer science venue or in a computer sci-
ence venue but contributed to the understanding of
2.4 Discussion
explanation in general (outside of AI).
The results, while only on a small set of papers, provide
1
evidence that many models being used in explainable
See http://home.earthlink.net/~ dwaha/research/meetings/ijcai17-xai/.
AI research are not building on current scientific under- However, it is also an opportunity: as Lipton [1990]
standing of explanation. Further, human behavioural argues, answering a contrastive question is often eas-
experiments are rare — something that needs to change ier than giving a full cause attribution because one
for us to produce useful explanatory agents. only needs to understand the difference between the two
It is important to note that we are not interpreting the cases, so one can provide a complete explanation without
above observations to say that there is not a lot of excel- determining or even knowing all causes of the event.
lent research on explainable AI. For example, consider
Ribeiro et al. [2016], who have done some remarkable 3.2 Attribution Theory
work on explaining classifiers, and yet scored ‘0’ on the Attribution theory is the study of how people attribute
‘Data Driven’ criteria. Instead, they have constructed causes to events; something that is necessary to pro-
their own understanding of how people evaluate expla- vide explanations. It is common to divide the types of
nations for their particular field over a series of human attribution into two classes: (1) causal attribution of so-
behavioural experiments. However, developing such an cial behaviour (called social attribution); and (2) general
understanding will not always be required or even pos- causal attribution.
sible for many researchers, so in these cases, building on Social Attribution The book from Malle [2004],
social science research is a sound place to start. based on a large body of work from himself and other
researchers in the field, describes a mature model of how
3 Where to? A Brief Pointer to people explain behaviour of others using folk psychol-
Relevant Work ogy. He argues that people attribute behaviour based
on the beliefs, desires, intentions, and traits of people,
In the different sub-fields of social sciences, there are and presents theories for why failed actions are described
several hundred articles on explanation, not to mention differently than successful actions; the former often re-
another entire field on causality. It is not feasible to ex- ferring to some precondition that could not be satisfied.
pect that AI researchers and practitioners can navigate Malle’s work provides a solid foundation on which to
this entire field in addition to their own field of expertise, build social attribution and explainable AI models for
especially considering that the relevant literature is writ- many sub-fields of artificial intelligence. Social attribu-
ten for a different audience. However, there are some key tion is important for systems in which intentional ac-
areas that should be of interest to those in explainable tion will be cited as a cause; in particular, it is im-
AI, which we outline in this section. portant for systems doing deliberative reasoning, and
Miller [2017] provides an in-depth survey of all articles the concepts used in his work are closely linked to
cited in this section plus many other relevant articles, that of systems such as belief-desire-intention models
and draws parallels between this work and explainable [Rao and Georgeff, 1995] and AI planning.
AI. Here, we present several key ideas from that work Causal Connection Research on how people
to demonstrate ways that models of explainable AI can connect causes shows that they do so by under-
benefit from models of human explanation. taking a mental simulation of what would have
happened had some other event turned out differently
3.1 Contrastive Explanation [Kahneman and Tversky, 1982; Hilton et al., 2005;
Perhaps the most important result from this work is McCloy and Byrne, 2000].
that explanations are contrastive; or more accurately, However, simulating an entire causal chain is in-
why–questions are contrastive. That is, why–questions feasible in most cases, so cognitive scientists and
are of the form “Why P rather than Q? ”, where P is social psychologists have studied how people de-
the fact that requires explanation, and Q is some foil cide which events to ‘undo’ (the counterfactuals)
case that was expected. Most philosophers, psychol- to determine cause. For example, people tend to
ogists, and cognitive scientists in this field assert that undo more proximal causes over more distal causes
all why–questions are contrastive (e.g. see [Hilton, 1990; [Miller and Gunasegaram, 1990], abnormal events
Lombrozo, 2012; Miller, 2017]), and that when people over normal events [Kahneman and Tversky, 1982],
ask for an explanation “Why P? ”, there is an implicit and events that are considered more ‘controllable’
contrast case. Importantly, the contrast case helps to [Girotto et al., 1991].
frame the possible answers and make them relevant For explainable AI models, these heuristics are useful
[Hilton, 1990]. For example, explaining “Why did Mr. from a computational perspective in large causal chains,
Jones open the window? ” with the response “Because in which causal attribution is intractable in many cases
he was hot ” is not useful if the implied foil is Mr. Jones [Eiter and Lukasiewicz, 2002]. Effectively, they can be
turning on the air conditioner, as this explains both the used to ‘skip-over’ or discount some events and not con-
fact and the foil; or if the implied foil was why Ms. Smith, sider their counterfactuals, while being consistent with
who was sitting closer to the window, did not open it in- what an explainee would expect.
stead, as the cited cause does not refer to a cause of Ms.
Smith’s lack of action. 3.3 Explanation Selection
This is a challenge for explainable AI, because it may An important body of work is concerned with explana-
not be easy to elicit a contrast case from an observer. tion selection. People rarely expect an explanation that
consists of an actual and complete cause of an event. strates people do follow these maxims, as discussed by
Instead, explainers select one or two causes and present Miller [2017].
these as the explanation. Explainees are typically able to Note that we are not arguing that explanations must
‘fill in’ their own causal understanding from just these. be text or verbal. However, explanations presented in a
Thus, some causes are better explanations than oth- visual way, for example, should have similar properties,
ers: events that are ‘closer’ to the fact in question in and these maxims offer a useful set of objective criteria.
the causal chain are preferred over more distal events
[Miller and Gunasegaram, 1990], but people will ‘trace 3.6 Where not to go
through’ closer events to more distal events if those dis- Finally, we discuss work that we believe should be dis-
tal events are human actions [Hilton et al., 2005] or ab- counted in explainable AI. Specifically, two well-known
normal events [Hilton and Slugoski, 1986]. theories of explanation, sometimes cited and used in ex-
In AI, perhaps some models are simple enough that plainable AI articles, are the logically deductive model of
explanation selection would not be valuable, or visuali- explanation [Hempel and Oppenheim, 1948], and the co-
sation would provide a powerful medium to show many variation model [Kelley, 1967]; both of which have had
causes at once. However, for causal chains with than a significant impact. However, since its publication, re-
handful of causes, we argue that explanation selection searchers found that the logically-deductive model was
can be used to simplify and/or prioritise explanations. inconsistent in many ways, and instead derived new
models of explanation. Similarly, the co-variation model
3.4 Explanation Evaluation was found be problematic and did not account for many
facets of human explanation [Malle, 2011], so was refined
The work discussed in this section so far looks at how ex- into other models, such as those of abnormality described
plainees generate and select explanations. There is also a in Section 3.3.
body of work that studies how people evaluate the qual- While these models are still cited as part of the history
ity of explanations provided to them. The most impor- of research in explanation, they are no longer considered
tant finding from this work is that the probability that valid models of human explanation in cognitive and so-
the cited cause is actually true is not the most important cial science. We contend, therefore, that explainable AI
criteria people use [Hilton, 1996]. Instead, people judge models should build on these newer models, which are
explanations based on so-called pragmatic influences of widely accepted, rather than these earlier models.
causes, which include criteria such as usefulness, rele-
vance, etc. [Slugoski et al., 1993]. 4 Conclusions
Recent work shows that people prefer ex-
planations that are simpler (cite few causes) We argued that existing models of how people gener-
[Lombrozo, 2007], more general (they explain more ate, select, present, and evaluate explanations are highly
events) [Lombrozo, 2007], and coherent (consistent relevant to explainable AI. Via a brief survey of articles,
with prior knowledge) [Thagard, 1989]. In particular, we provide evidence that little research on explainable AI
Lombrozo [2007] shows that the people dispropor- draws on such models. Although the survey was limited,
tionately prefer simpler explanations over more likely it is clear from our readings that the observation holds
explanations. more generally. We pointed to a handful of key articles
that we believe could be important, but for a proper
These criteria are important to any work in explain-
presentation and discussion of these, see Miller [2017].
able AI. Giving simpler explanations that increase the
We encourage researchers and practitioners in explain-
likelihood that the observer both understands and ac-
able AI to collaborate with researchers and practition-
cepts the explanation may be more useful to establish
ers from the social and behavioural sciences, to inform
trust, if this is the primary goal of the explanation.
both model design and human behavioural experiments.
Learning from these and adding them as objective crite-
We do not advocate that every paper on explainable
ria to models of explainable AI is important.
AI should be accompanied by human behavioural ex-
periments — proxy studies are valid ways to evaluate
3.5 Explanation as Conversation models of explanation, especially those in early devel-
Finally, it is important to remember that explanations opment, and computational problems are also of in-
are interactive conversations, and that people typically terest. However, we support the emphasis in the re-
abide by certain rules of conversation [Hilton, 1990]. cent DARPA solicitation [DARPA, 2016] on reaching
Grice’s maxims [Grice, 1975] are the most well-known “human-in-the-loop techniques that developers can use
and widely accepted rules of conversation. In short, they ... for more intensive human evaluations,’ and agree with
say that in a conversation, people consider the following: Doshi-Velez and Kim [2017] that to have a real-world im-
(a) quality; (b) quantity; (c) relation; and (d) manner. pact, “it is essential that we as a community respect the
Coarsely, these respectively mean: only say what you time and effort involved to do such evaluations.”
believe; only say as much as is necessary; only say what We hope that readers of this paper and participants in
is relevant; and say it in a nice way. Hilton [1990] ar- the workshop agree with our position and, where feasible,
gues that as explanations are conversations, they follow adopt existing models and methods to reduce the risk
these maxims. There is body of research that demon- that it is only the inmates that are running the asylum.
References [Kelley, 1967] Harold H Kelley. Attribution theory in
[Buchanan and Shortliffe, 1984] Bruce Buchanan and social psychology. In Nebraska symposium on motiva-
Edward Shortliffe. Rule-based expert systems: the tion, pages 192–238. Uni. Nebraska Press, 1967.
MYCIN experiments of the Stanford Heuristic Pro- [ Lipton, 1990] Peter Lipton. Contrastive explanation.
gramming Project. Addison-Wesley, 1984. Royal Institute of Philosophy Supplement, 27:247–266,
[Chandrasekaran et al., 1989] B Chandrasekaran, 1990.
Michael C. Tanner, and John R. Josephson. Ex- [Lombrozo, 2007] Tania Lombrozo. Simplicity and prob-
plaining control strategies in problem solving. IEEE ability in causal explanation. Cognitive psychology,
Expert, 4(1):9–15, 1989. 55(3):232–257, 2007.
[Cooper, 2004] Alan Cooper. The inmates are running [Lombrozo, 2012] Tania Lombrozo. Explanation and ab-
the asylum: Why high-tech products drive us crazy and ductive inference. Oxford handbook of thinking and
how to restore the sanity. Sams, IN, USA, 2004. reasoning, pages 260–276, 2012.
[DARPA, 2016] DARPA. Explainable ar- [Malle, 2004] Bertram F Malle. How the mind explains
tificial intelligence (XAI) program. behavior: Folk explanations, meaning, and social in-
teraction. MIT Press, 2004.
http://www.darpa.mil/program/explainable-artificial-intelligence,
2016. Full solicitation at [Malle, 2011] Bertram F Malle. Time to give up the
http://www.darpa.mil/attachments/DARPA-BAA-16-53.pdf. dogmas of attribution: An alternative theory of be-
[Doshi-Velez and Kim, 2017] F. Doshi-Velez and havior explanation. Advances in Experimental Social
B. Kim. Towards A Rigorous Science of Interpretable Psychology, 44(1):297–311, 2011.
Machine Learning. ArXiv e-prints, 1702.08608, 2017. [McCloy and Byrne, 2000] Rachel McCloy and
[Eiter and Lukasiewicz, 2002] Thomas Eiter and Ruth MJ Byrne. Counterfactual thinking about
Thomas Lukasiewicz. Complexity results for controllable events. Memory & Cognition, 28(6):1071–
structure-based causality. Artificial Intelligence, 1078, 2000.
142(1):53–89, 2002. [Miller and Gunasegaram, 1990] Dale T Miller and
[Girotto et al., 1991] Vittorio Girotto, Paolo Legrenzi, Saku Gunasegaram. Temporal order and the perceived
and Antonio Rizzo. Event controllability in counter- mutability of events: Implications for blame assign-
factual thinking. Acta Psychologica, 78(1):111–133, ment. Journal of personality and social psychology,
1991. 59(6):1111, 1990.
[Grice, 1975] Herbert P Grice. Logic and conversation. [ Miller, 2017] Tim Miller. Explainable AI: Insights from
In Syntax and semantics 3: Speech arts, pages 41–58. the social sciences. ArXiv e-prints, 1706.07269, 2017.
New York: Academic Press, 1975. https://arxiv.org/abs/1706.07269.
[Hempel and Oppenheim, 1948] Carl G Hempel and [Nott, 2017] George Nott. ‘Explainable Arti-
Paul Oppenheim. Studies in the logic of explanation. ficial Intelligence’: Cracking open the black
Philosophy of Science, 15(2):135–175, 1948. box of AI. Computer World, 4 2017.
https://www.computerworld.com.au/article/617359/.
[Hilton and Slugoski, 1986] Denis J Hilton and Ben R
[Rao and Georgeff, 1995] Anand S Rao and Michael P
Slugoski. Knowledge-based causal attribution: The
abnormal conditions focus model. Psychological re- Georgeff. Bdi agents: From theory to practice. In
view, 93(1):75, 1986. ICMAS, volume 95, pages 312–319, 1995.
[Ribeiro et al., 2016] Marco Tulio Ribeiro, Sameer
[Hilton et al., 2005] Denis J. Hilton, John L. McClure,
Singh, and Carlos Guestrin. Why should I trust you?:
and R. Slugoski, Ben. The course of events: Coun-
Explaining the predictions of any classifier. In Pro-
terfactuals, causal sequences and explanation. In The
ceedings of the Int. Conf.x on Knowledge Discovery
Psychology of Counterfactual Thinking. 2005.
and Data Mining, pages 1135–1144. ACM, 2016.
[Hilton, 1990] Denis J Hilton. Conversational pro- [Slugoski et al., 1993] Ben R Slugoski, Mansur Lalljee,
cesses and causal explanation. Psychological Bulletin,
Roger Lamb, and Gerald P Ginsburg. Attribution in
107(1):65–81, 1990.
conversational context: Effect of mutual knowledge on
[Hilton, 1996] Denis J Hilton. Mental models and causal explanation-giving. European Journal of Social Psy-
explanation: Judgements of probable cause and ex- chology, 23(3):219–238, 1993.
planatory relevance. Thinking & Reasoning, 2(4):273– [Swartout and Moore, 1993] William R Swartout and
308, 1996. Johanna D Moore. Explanation in second generation
[Kahneman and Tversky, 1982] Daniel Kahneman and expert systems. In Second generation expert systems,
Amos Tversky. The simulation heuristic. In P. Slovic pages 543–585. Springer, 1993.
D. Kahneman and A. Tversky, editors, Judgment un- [Thagard, 1989] Paul Thagard. Explanatory coherence.
der Uncertainty: Heuristics and Biases. New York: Behavioral and Brain Sciences, 12(03):435–467, 1989.
Cambridge University Press, 1982.
A List of Papers Surveyed 11. Letham, B., Rudin. C., McCormick, T., and Madi-
gan, D. (2015). Interpretable classifiers using rules
Taken from the ‘Related Work’ list posted on the website
and Bayesian analysis: Building a better stroke pre-
for the IJCAI 2017 Explainable AI workshop2 as of 16
diction model. Annals of Applied Statistics, 9(3),
May 2017.
1350-137.
1. Chakraborti, T., Sreedharan, S., Zhang, Y., & 12. Lombrozo, T. (2012). Explanation and abductive
Kambhampati, S. (2017). Plan explanations as inference. Oxford Handbook of Thinking And Rea-
model reconciliation: Moving beyond explanation soning (pp. 260-276).
as soliloquy. To appear in Proceedings of the
Twenty-Sixth International Joint Conference on Ar- 13. Martens, D., & Provost, F. (2014). Explaining
tificial Intelligence. Melbourne, Australia: AAAI data-driven document classifications. MIS Quar-
Press. terly, 38(1), 73-99.
14. Ribeiro, M.T., Singh, S., & Guestrin, C. (2016).
2. Cheng, H., et al. (2014) SRI-Sarnoff Aurora at
”Why should I trust you?” Explaining the pre-
TRECVid 2014: Multimedia event detection and
dictions of any classifier. Human Centered Ma-
recounting.
chine Learning: Papers from the CHI Workshop.
3. Doshi-Velez, F., & Kim, B. (2017). A (arXiv:1602.04938v1)
roadmap for a rigorous science of interpretability. 15. Rosenthal, S., Selvaraj, S. P., & Veloso, M. (2016).
(arXiv:1702.08608) Verbalization: Narration of autonomous mobile
4. Elhoseiny, M., Liu, J., Cheng, H., Sawhney, H., & robot experience. In Proceedings of the Twenty-
Elgammal, A. (2015). Zero-shot event detection Fifth International Joint Conference on Artificial
by multimodal distributional semantic embedding Intelligence. New York, NY: AAAI Press.
of videos. Proceedings of the Thirtieth AAAI Con- 16. Sheh, R.K. (2017). “Why did you do that?” Ex-
ference on Artificial Intelligence (pp. 3478-3486). plainable intelligent robots. In K. Talamadupula, S.
Phoenix, AZ: AAAI Press. Sohrabi, L. Michael, & B. Srivastava (Eds.) Human-
5. Hendricks, L.A, Akata, Z., Rohrbach, M., Donahue, Aware Artificial Intelligence: Papers from the AAAI
J., Schiele, B., & Darrell, T. (2016). Generating Workshop (Technical Report WS-17-11). San Fran-
visual explanations. (arXiv:1603.08507v1) cisco, CA: AAAI Press.
6. Kofod-Petersen, A., Cassens, J., & Aamodt, A. 17. Si, Z. and Zhu, S. (2013). Learning AND-OR tem-
(2008). Explanatory capabilities in the CREEK plates for object recognition and detection. IEEE
knowledge-intensive case-based reasoner. Frontiers Transactions On Pattern Analysis and Machine In-
in Artificial Intelligence and Applications, 173, 28- telligence, 35(9), 2189-2205.
35. 18. Shwartz-Ziv, R. & Tishby, N. (2017). Opening the
7. Kulesza, T., Burnett, M., Wong, W. K., & Stumpf, black box of deep neural networks via information.
S. (2015). Principles of explanatory debugging to (arXiv:1703.00810 [cs.LG])
personalize interactive machine learning. Proceed- 19. Sormo, F., Cassens, J., & Aamodt, A. (2005). Ex-
ings of the Twentieth International Conference on planation in case-based reasoning: Perspectives and
Intelligent User Interfaces (pp. 126-137). Atlanta, goals. Artificial Intelligence Review, 24(2), 109-143.
GA: ACM Press. 20. Swartout, W., Paris, C., & Moore, J. (1991). Expla-
8. Lake, B.H., Salakhutdinov, R., & Tenenbaum, J.B. nations in knowledge systems: Design for explain-
(2015). Human-level concept learning through able expert systems. IEEE Expert, 6(3), 58-64.
probabilistic program induction. Science, 350, 1332- 21. van Lent, M., Fisher, W., & Mancuso, M. (2004).
1338. An explainable artificial intelligence system for
9. Langley, P., Meadows, B., Sridharan, M., & Choi, small-unit tactical behavior. Proceedings of the
D. (2017). Explainable agency for intelligent au- Nineteenth National Conference on Artificial Intel-
tonomous systems. In Proceedings of the Twenty- ligence (pp. 900-907). San Jose, CA: AAAI Press.
Ninth Annual Conference on Innovative Applica- 22. Zahavy, T., Zrihem, N.B., & Mannor, S. (2017).
tions of Artificial Intelligence. San Francisco: AAAI Graying the black box: Understanding DQNs.
Press. (arXiv:1602.02658 [cs.LG])
10. Lécué, F. (2012). Diagnosing changes in an ontol- 23. Zhang, Y., Sreedharan, S., Kulkarni, A.,
ogy stream: A DL reasoning approach. In Proceed- Chakraborti, T., Zhuo, H.H., & Kambhampati, S.
ings of the Twenty-Sixth AAAI Conference on Arti- (2017). Plan explicability and predictability for
ficial Intelligence. Toronto, Ontario, Canada: AAAI robot task planning. To appear in Proceedings
Press. of the International Conference on Robotics and
2
Automation. Singapore: IEEE Press.
See http://home.earthlink.net/~ dwaha/research/meetings/ijcai17-xai/.
B Detailed Results

Paper On topic Data Driven Validation Comments

1 1 1 0
2 0 0 0
3 1 1 N/A A position paper, so Validation not applicable.
4 0 0 0
5 1 0 1
6 1 1 0
7 1 0 1
8 0 2 1 Off topic, but is mature work
9 1 0 N/A
10 0 0 0
11 1 ? 0 Could not locate reference Jennings et al. (1982)
12 N/A N/A N/A Survey paper on explanation in the social sciences
13 1 0 0
14 1 0 1
15 1 0 0
16 1 0 0
17 0 0 0
18 1 0 0
19 1 2 N/A Survey paper, so Validation not applicable
20 1 0 0
21 1 0 1
22 1 0 0
23 1 1 0