Human-Centred Explainability

Kacper Sokol

Topics

• Inmates Running the Asylum
• Contrastive Statements
• Social Process
• Wrap Up

Inmates Running the Asylum

Operational Vacuum

• Focused on technology
• Little consideration for the real world
• Implicit target audience – researchers and technologists

Example use case: explanatory debugging (Kulesza et al. 2015)

(Miller, Howe, and Sonenberg 2017)

Who’s at the Other End?

• Explanations arise to

  • address inconsistency with the explainee’s beliefs, expectations or mental model, e.g., an unexpected ML prediction causing a disagreement
  • support learning or provide information needed by an explainee to solve a problem or complete a task

• Delivered at a request of a human explainee

An explanation is an answer to a “Why?” question (Miller 2019)

• Building up on the human-centred revolution mentioned in the preliminaries and the collection of human-centred properties listed in the taxonomy

• More definitions of explainability to follow in the definitions section

• Who the recipient is – making explanations human-centred

Enter Human-Centred Agenda

Humans expect the explanations to be (Miller 2019)

1. contrastive
2. selective
3. not overly technical
4. social

• selective – just enough information to address the explanatory needs, as opposed to a full causal account
• non-technical – communicating specific probabilities of events may not be helpful
• explanations as a social process – bidirectional, interactive, communication-based

Composing an Explanation

• topic – what should be explained
• stakeholder – to whom something should be explained
• goal – why something should be explained
• instrument – how something should be explained

(Buchholz 2022)

Complications

• Diversity of human goals and expectations
• Human biases, e.g., The Illusion of Explanatory Depth (Rozenblit and Keil 2002)

---

Counterfactual explanations are specific to a data point

Have you been 5 years older, your loan application would be accepted.

• People believe that they understand more than they actually do

• Overconfidence may lead people to generalise a counterfactual explanation beyond the instance for which it was generated (to unseen and possibly unrelated cases)

Contrastive Statements

Why?

• Easy to understand (sparse by default)
• Common in everyday life
• Available in diverse formats (e.g., textual or visual)
• Actionable from a technical perspective
• Compliant with regulatory frameworks (e.g., GDPR) (Wachter, Mittelstadt, and Russell 2017)

• Technical actionability stems from counterfactuals prescribing a change in the data point that leads to a different prediction

Example

Had you been 10 years younger,
your loan application would be accepted.

Duck Test

If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.

• Counterfactuals are not necessarily causal!

Actionability

Had you been 10 years younger,
your loan application would be accepted.

---

Had you paid back one of your credit cards,
your loan application would be accepted.

• Evolution of human-centredness: non-actionable → actionable → feasible → provided through algorithmic recourse

Feasibility

• Is the generated instance realistic, i.e., from the data distribution?
• Can it be reached from the explained data point?

(Poyiadzi et al. 2020)

Algorithmic Recourse

• Where to go and how to get there
• A sequence of steps (actions) guiding the explainee towards the desired goal with explanations phrased as actionable recommendations

---

• Independent manipulation of individual attributes is undesirable – think feature correlation
• Instead, these actions can be modelled as (causal) interventions

• action – independent shift in feature values

(Karimi et al. 2022)

Social Process

Human Explainability

• Bi-directional explanatory process
• Questioning and explanatory utterances
• Conversational

Interactive Explainability

• One of the tenets of human-centred explainability
• Largely neglected and unavailable (Schneider and Handali 2019)

Conversational Explainability

• Interactive personalisation and tuning of the explanations
• Guiding the explainer to retrieve tailored insights

Naïve Realisation

Dialogue-based personalisation

Why was my loan application denied?

Instead of increasing my income. Is there anything I can do about my outstanding debt to get this loan approved?

Because of your income. Had you earned £5,000 more, it would have been granted.

If you cancel one of your three credit cards, you will receive the loan.

(Sokol and Flach 2020; Sokol and Flach 2018)

Interactive ≠ Social

• Interactivity is insufficient, e.g., static explanation + dynamic user interface

• Vehicle to personalise content (and other aspects)

• Bespoke explanatory experience driven by context

Wrap Up

Summary

Producing explanations is necessary but insufficient for human-centred explainability

These insights need to be relevant and comprehensible (context) to explainees

Summary    

Explainers are socio-technical constructs, hence we should strive for seamless integration with humans as well as technical correctness and soundness

Summary    

Each (real-life) explainability scenario is unique and requires a bespoke solution

Summary    

(The Blind Men and the Elephant)

Bibliography

Buchholz, Oliver. 2022. “A Means-End Account of Explainable Artificial Intelligence.” arXiv Preprint arXiv:2208.04638.

Karimi, Amir-Hossein, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. 2022. “A Survey of Algorithmic Recourse: Contrastive Explanations and Consequential Recommendations.” ACM Computing Surveys 55 (5): 1–29.

Kulesza, Todd, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. “Principles of Explanatory Debugging to Personalize Interactive Machine Learning.” In Proceedings of the 20^th International Conference on Intelligent User Interfaces, 126–37.

Miller, Tim. 2019. “Explanation in Artificial Intelligence: Insights from the Social Sciences.” Artificial Intelligence 267: 1–38.

Miller, Tim, Piers Howe, and Liz Sonenberg. 2017. “Explainable AI: Beware of Inmates Running the Asylum or: How i Learnt to Stop Worrying and Love the Social and Behavioural Sciences.” arXiv Preprint arXiv:1712.00547.

Poyiadzi, Rafael, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. 2020. “FACE: Feasible and Actionable Counterfactual Explanations.” In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 344–50.

Rozenblit, Leonid, and Frank Keil. 2002. “The Misunderstood Limits of Folk Science: An Illusion of Explanatory Depth.” Cognitive Science 26 (5): 521–62.

Schneider, Johannes, and Joshua Peter Handali. 2019. “Personalized Explanation for Machine Learning: A Conceptualization.”

Sokol, Kacper, and Peter Flach. 2020. “One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparency.” KI-Künstliche Intelligenz 34 (2): 235–50.

Sokol, Kacper, and Peter A Flach. 2018. “Glass-Box: Explaining AI Decisions with Counterfactual Statements Through Conversation with a Voice-Enabled Virtual Assistant.” In IJCAI, 5868–70.

Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841.

Questions

kacper.sokol@rmit.edu.au
k.sokol@bristol.ac.uk