XML Preliminaries

Kacper Sokol

Topics

• Brief History of Explainability
• Why We Need Explainability
• Explanation Types
• Important Developments
• Examples of Explanations
• Useful Resources
• Wrap Up

Brief History of Explainability

Expert Systems (1970s & 1980s)

Transparent Machine Learning Models

Rise of the Dark Side (Deep Neural Networks)

• No need to engineer features (by hand)
• High predictive power
• Black-box modelling

DARPA’s XAI Concept

https://www.darpa.mil/program/explainable-artificial-intelligence

Why We Need Explainability

Expectations Mismatch

We tend to request explanations mostly when a phenomenon disagrees with our expectations.

For example, an ML model behaves unlike we envisaged and outputs an unexpected prediction.

Stakeholders

(Belle and Papantonis 2021)

Purpose or Role

• Fairness
• Privacy
• Reliability and Robustness
• Causality
• Trust

• Trustworthiness / Reliability / Robustness / Causality

  No silly mistakes & socially acceptable

• Fairness

  Does not discriminate & is not biased

• Privacy – no data leakage
• Reliability and Robustness – no undesired predictive behaviour, e.g., large changes for small changes in input
• Causality – model only relies on true causal relations, and not spurious correlations

(Doshi-Velez and Kim 2017)

Benefits

• New knowledge

  Aids in scientific discovery

• Legislation

  Does not break the law

  • EU’s General Data Protection Regulation
  • California Consumer Privacy Act

• Debugging / Auditing

  Identify modelling errors and mistakes

• Human–AI co-operation

  Help humans complete tasks

• debugging and auditing – husky vs. wolf in presence of background snow; (we will see later why this conclusion may have been incorrect)

Drawbacks

• Safety / Security

  Abuse transparency to steal a (proprietary) model

• Manipulation

  Use transparency to game a system, e.g., credit scoring

Pitfalls

Copy machine study done by Langer, Blank, and Chanowitz (1978):

• The danger of “empty explanations” – statements that look like explanations (…because…) but carry no meaningful explanation

• When asking for a favor, providing a reason increases the chances of success

1. “Excuse me, I have 5 pages. May I use the Xerox machine?” (request only)
2. “Excuse me, I have 5 pages. May I use the xerox machine, because I’m in a rush?” (request with a real reason)
3. “Excuse me, I have 5 pages. May I use the xerox machine, because I have to make copies?” (request with a fake reason)

Explanation Types

• We will look into more details characterising explanations in the taxonomy module
• These remarks are just to facilitate the following discussion

Explainability Source

• ante-hoc – intrinsically transparent predictive models (transparency by design)
• post-hoc – derived from a pre-existing predictive models that may themselves be unintelligible (usually requires an additional explanatory modelling step)

• Post-hoc build an understandable model of the relation between inputs and outputs

• These techniques can be model-specific or model-agnostic (based on input–output paris / observations)

Explanation Provenance

    Ante-hoc does not necessarily entail explainable or human-understandable

• endogenous explanation – based on human-comprehensible concepts operated on by a transparent model
• exogenous explanation – based on human-comprehensible concepts constructed outside of the predictive model (by the additional modelling step)

• Information lineage – the sources of explanatory information

• In the definition module, we will see why ante-hoc transparency may not entail explainability / interpretability (human understanding)

• Just a brief note: We may preserve all the desired properties of ante-hoc interpretability (intrinsic transparency of the model) but still derive explanations not directly from its transparency

• For example, process a complex decision tree – transparent by design – to extract a counterfactual from adjacent leaves, thus making it explainable

Explanation Domain

Original domain

Transformed domain

• Explanation domain is linked to explanation provenance

Explanation Types

• model-based – derived from model internals
• feature-based – regarding importance or influence of data features
• instance-based – carried by rael or fictitious data point

---

• meta-explainers – one of the above, but not extracted directly from the predictive model being explained (using an additional explainability modelling step, e.g., surrogate)

• The categories are not clear cut; a linear model explained through its coefficients is both model-based and feature-based

Explanation Family

• associations between antecedent and consequent

• feature importance
• feature attribution / influence
• rules

• exemplars (prototypes & criticisms)

• Alternative classification of explanation types

Explanation Family    

• contrasts and differences

  • (non-causal) counterfactuals
    i.e., contrastive statements
  • prototypes & criticisms

Explanation Family    

• causal mechanisms

  • causal counterfactuals
  • causal chains
  • full causal model

Explanatory Medium

• (statistical / numerical) summarisation
• visualisation
• textualisation
• formal argumentation

• These insights can be communicated via different media (as we will see later)

  • numerical summarisation – a single number or their collection (e.g., a table)
  • visualisation
  • textualisation

Communication of Explanations

• Static artefact

• Interactive (explanatory) protocol

  • interactive interface
  • interactive explanation

Explainability Scope

	global	cohort	local
data	a set of data	a subset of data	an instance
model	model space	model subspace	a point in model space
prediction	a set of predictions	a subset of predictions	a individual prediction

---

• algorithmic explanation – the learning algorithm, not the resulting model; e.g., modelling assumptions, caveats, compatible data types, etc.

• local explanations do not generalise to other instances

• while data points may appear similar, our perception of similarity is likely to be different from the similarity the underlying model is using

• a collection of cohort explanations may be easier to comprehend than a single global – recall the difference between transparency and explainability

• this requires a suitable explanation aggregation policy that humans can execute

Explainability Target

• Focused on a single class (technically limited)

  • implicit context

    Why \(A\)? (…and not anything else, i.e., \(B \cup C \cup \ldots\))

  • explicit context

    Why \(A\) and not \(B\)?

• Multi-class explainability (Sokol and Flach 2020b)

  If 🌧️, then \(A\); else if ☀️ & 🥶, then \(B\), else ☀️ & 🥵, then \(C\).

Important Developments

Where Is the Human? (circa 2017)

(Miller 2019)

Humans and Explanations

• Human-centred perspective on explainability
• Infusion of explainability insights from social sciences
  • Interactive dialogue (bi-directional explanatory process)
  • Contrastive statements (e.g., counterfactual explanations)

Exploding Complexity (2019)

• High-stakes vs. low-stakes
• Ante-hoc vs. post-hoc

(Rudin 2019)

Ante-hoc vs. Post-hoc

• Predictive performance vs. explainability / transparency

Black Box + Post-hoc Explainer

1. Chose a well-performing black-box model
2. Use explainer that is
   • post-hoc (can be retrofitted into pre-existing predictors)
   • and possibly model-agnostic (works with any black box)

Caveat: The No Free Lunch Theorem

Post-hoc explainers have poor fidelity

• Explainability needs a process similar to KDD, CRISP-DM or BigData
  
• Focus on engineering informative features and inherently transparent models

It requires effort

XAI process

A generic eXplainable Artificial Intelligence process is beyond our reach at the moment

• XAI Taxonomy spanning social and technical desiderata:
  • Functional • Operational • Usability • Safety • Validation •
  (Sokol and Flach, 2020. Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches)

• Framework for black-box explainers
  (Henin and Le Métayer, 2019. Towards a generic framework for black-box explanations of algorithmic decision systems)
  

Examples of Explanations

Permutation Feature Importance

()     https://www.kaggle.com/code/dansbecker/permutation-importance

Individual Conditional Expectation & Partial Dependence

()     ()

FACE Counterfactuals

()     (Poyiadzi et al. 2020)

RuleFit

()     https://christophm.github.io/interpretable-ml-book/rulefit.html

Useful Resources

📖   Books

• Survey of machine learning interpretability in form of an online book
• Overview of explanatory model analysis published as an online book

📝   Papers

• General introduction to interpretability (Sokol and Flach 2021)
• Introduction to human-centred explainability (Miller 2019)
• Critique of post-hoc explainability (Rudin 2019)
• Survey of interpretability techniques (Guidotti et al. 2018)
• Taxonomy of explainability approaches (Sokol and Flach 2020a)

💽   Software

• Microsoft’s Interpret
• Oracle’s Skater
• IBM’s Explainability 360
• FAT Forensics
• DALEX

• alibi
• iml
• LIME (Python, R)
• SHAP (Python, R)

Wrap Up

Summary

• The landscape of explainability is fast-paced and complex
• Don’t expect universal solution
• The involvement of humans – as explainees – makes it all the more complicated

Bibliography

Belle, Vaishak, and Ioannis Papantonis. 2021. “Principles and Practice of Explainable Machine Learning.” Frontiers in Big Data, 39.

Doshi-Velez, Finale, and Been Kim. 2017. “Towards a Rigorous Science of Interpretable Machine Learning.” arXiv Preprint arXiv:1702.08608.

Guidotti, Riccardo, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. “A Survey of Methods for Explaining Black Box Models.” ACM Computing Surveys (CSUR) 51 (5): 1–42.

Langer, Ellen J, Arthur Blank, and Benzion Chanowitz. 1978. “The Mindlessness of Ostensibly Thoughtful Action: The Role of ‘Placebic’ Information in Interpersonal Interaction.” Journal of Personality and Social Psychology 36 (6): 635.

Miller, Tim. 2019. “Explanation in Artificial Intelligence: Insights from the Social Sciences.” Artificial Intelligence 267: 1–38.

Poyiadzi, Rafael, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. 2020. “FACE: Feasible and Actionable Counterfactual Explanations.” In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 344–50.

Rudin, Cynthia. 2019. “Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead.” Nature Machine Intelligence 1 (5): 206–15.

Sokol, Kacper, and Peter Flach. 2020a. “Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 56–67.

———. 2020b. “LIMEtree: Consistent and Faithful Surrogate Explanations of Multiple Classes.” arXiv Preprint arXiv:2005.01427.

———. 2021. “Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligence.” arXiv Preprint arXiv:2112.14466.

Questions

kacper.sokol@rmit.edu.au
k.sokol@bristol.ac.uk