Human-Centred Explainability

Kacper Sokol

Inmates Running the Asylum

Operational Vacuum

  • Focused on technology
  • Little consideration for the real world
  • Implicit target audience – researchers and technologists

Example use case: explanatory debugging (Kulesza et al. 2015)

Who’s at the Other End?

  • Explanations arise to

    • address inconsistency with the explainee’s beliefs, expectations or mental model, e.g., an unexpected ML prediction causing a disagreement
    • support learning or provide information needed by an explainee to solve a problem or complete a task
  • Delivered at a request of a human explainee

An explanation is an answer to a “Why?” question (Miller 2019)

Enter Human-Centred Agenda

Humans expect the explanations to be (Miller 2019)

  1. contrastive
  2. selective
  3. not overly technical
  4. social

Composing an Explanation

  • topic – what should be explained
  • stakeholder – to whom something should be explained
  • goal – why something should be explained
  • instrument – how something should be explained


  • Diversity of human goals and expectations
  • Human biases, e.g., The Illusion of Explanatory Depth (Rozenblit and Keil 2002)

Counterfactual explanations are specific to a data point

Have you been 5 years older, your loan application would be accepted.

Contrastive Statements


  • Easy to understand (sparse by default)
  • Common in everyday life
  • Available in diverse formats (e.g., textual or visual)
  • Actionable from a technical perspective
  • Compliant with regulatory frameworks (e.g., GDPR) (Wachter, Mittelstadt, and Russell 2017)


Had you been 10 years younger,
your loan application would be accepted.

Example of an image counterfactual explanation

Duck Test

If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.

  • Counterfactuals are not necessarily causal!


Had you been 10 years younger,
your loan application would be accepted.

Had you paid back one of your credit cards,
your loan application would be accepted.



Algorithmic Recourse

  • Where to go and how to get there
  • A sequence of steps (actions) guiding the explainee towards the desired goal with explanations phrased as actionable recommendations

  • Independent manipulation of individual attributes is undesirable – think feature correlation
  • Instead, these actions can be modelled as (causal) interventions

Social Process

Human Explainability

  • Bi-directional explanatory process
  • Questioning and explanatory utterances
  • Conversational

Interactive Explainability

Conversational Explainability

  • Interactive personalisation and tuning of the explanations
  • Guiding the explainer to retrieve tailored insights

Naïve Realisation

Dialogue-based personalisation

Why was my loan application denied?

Instead of increasing my income. Is there anything I can do about my outstanding debt to get this loan approved?

Because of your income. Had you earned £5,000 more, it would have been granted.

If you cancel one of your three credit cards, you will receive the loan.

Interactive ≠ Social

  • Interactivity is insufficient, e.g., static explanation + dynamic user interface

  • Vehicle to personalise content (and other aspects)

  • Bespoke explanatory experience driven by context

Wrap Up


Producing explanations is necessary but insufficient for human-centred explainability

These insights need to be relevant and comprehensible (context) to explainees


Explainers are socio-technical constructs, hence we should strive for seamless integration with humans as well as technical correctness and soundness


Each (real-life) explainability scenario is unique and requires a bespoke solution


The Blind Men and the Elephant


Buchholz, Oliver. 2022. “A Means-End Account of Explainable Artificial Intelligence.” arXiv Preprint arXiv:2208.04638.
Karimi, Amir-Hossein, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. 2022. “A Survey of Algorithmic Recourse: Contrastive Explanations and Consequential Recommendations.” ACM Computing Surveys 55 (5): 1–29.
Kulesza, Todd, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. “Principles of Explanatory Debugging to Personalize Interactive Machine Learning.” In Proceedings of the 20th International Conference on Intelligent User Interfaces, 126–37.
Miller, Tim. 2019. “Explanation in Artificial Intelligence: Insights from the Social Sciences.” Artificial Intelligence 267: 1–38.
Miller, Tim, Piers Howe, and Liz Sonenberg. 2017. “Explainable AI: Beware of Inmates Running the Asylum or: How i Learnt to Stop Worrying and Love the Social and Behavioural Sciences.” arXiv Preprint arXiv:1712.00547.
Poyiadzi, Rafael, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. 2020. FACE: Feasible and Actionable Counterfactual Explanations.” In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 344–50.
Rozenblit, Leonid, and Frank Keil. 2002. “The Misunderstood Limits of Folk Science: An Illusion of Explanatory Depth.” Cognitive Science 26 (5): 521–62.
Schneider, Johannes, and Joshua Peter Handali. 2019. “Personalized Explanation for Machine Learning: A Conceptualization.”
Sokol, Kacper, and Peter Flach. 2020. “One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparency.” KI-Künstliche Intelligenz 34 (2): 235–50.
Sokol, Kacper, and Peter A Flach. 2018. Glass-Box: Explaining AI Decisions with Counterfactual Statements Through Conversation with a Voice-Enabled Virtual Assistant.” In IJCAI, 5868–70.
Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841.