Machine Learning Explainability

Exploring Automated Decision-Making Through Transparent Modelling and
Peeking Inside Black Boxes

Kacper Sokol


About Me


  • Master of Engineering in Mathematics and Computer Science
    (University of Bristol)
  • Doctor of Philosophy in Computer Science – Artificial Intelligence and Machine Learning
    (University of Bristol)

About Me    

Work Experience

  • Research Fellow,
    ARC Centre of Excellence for Automated Decision-Making and Society
    (RMIT University)
  • Honorary Research Fellow,
    Intelligent Systems Laboratory
    (University of Bristol)

About Me    

Work Experience    

  • Senior Research Associate,
    TAILOR: European Union’s AI Research Excellence Centre
    (University of Bristol)
  • Research Associate,
    Human-Centred Artificial Intelligence in collaboration with THALES
    (University of Bristol)
  • Research Assistant,
    (University of Bristol)

About Me    

Work Experience    

  • Visiting Research Fellow,
    AI and Humanity Summer Cluster, Simons Institute for the Theory of Computing
    (UC Berkeley)
  • Visiting Researcher,
    Ubiquitous Computing Research Group
    (Università della Svizzera italiana)
  • Visiting Researcher,
    Machine Learning research group
    (University of Tartu)

About Me    


  • AI & ML transparency, interpretability and explainability

  • Taxonomy of XAI & IML

  • Human-centred, interactive and personalised XAI & IML

About Me    


  • Robustness, accountability, modularity and parameterisation of XAI & IML
  • Post-hoc methods, e.g., surrogates

About Me    


  • XAI & IML open-source software

  • XAI & IML freely available, online, interactive training materials

About You

  • Who are you?
  • What’s your background?
  • Why are you taking this course?
  • What do you expect to get out of this course?

Organisational Details


  • When: 2 weeks – w/c 6th and 13th of February 2023

  • What: 10 sessions comprising of

    • 45-minute lecture
    • 45-minute supervised lab session (project / assignment / coursework)
    • 30-minute open office (general questions & project discussions)
  • Time Commitment: 20 hours (self-study)


  • Where:

    What Time Location (week 1) Location (week 2)
    lecture 9.30–10.15am D0.03 D1.14
    discussion 10.15–10.30am D0.03 D1.14
    lab 10.30–11.15am D0.03 D1.14
    open office 11.30am–12pm D0.03 D1.14


  • Python programming
  • Basic mathematical concepts (relevant to machine learning)
  • Machine learning techniques for tabular data
  • Prior experience with machine learning approaches for images and text (e.g., deep learning) or other forms of data modelling (e.g., time series forecasting, reinforcement learning) if you decide to pursue a project in this direction




  • Written in Markdown
  • Built with Quarto into reveal.js (web-based slides)
  • Computational elements and figures coded in Python (with matplotlib)
  • Source can be compiled into Jupyter Notebooks (to experiment, modify, adapt and reuse the code chunks)

About the Course


  • Wealth of XAI and IML learning resources…

  • …but mostly limited to

    • summary descriptions
    • code examples
    • explanation examples
    • interpretation tips


  • Deconstruct each method
  • Inspect its assumptions and operationalisation
  • Learn to tune explainers for the problem at hand
  • Learn to interpret explanations in view of their theoretical properties and (limitations of) algorithmic implementation

  • Develop critical thinking about XAI and IML techniques

General Learning Objectives

  • Understand the landscape of AI and ML explainability techniques
  • Identify explainability needs of data-driven machine learning systems
  • Recognise the capabilities and limitations of explainability approaches, both in general and in view of specific use cases
  • Apply these skills to real-life AI and ML problems
  • Communicate explainability findings through interactive reports and dashboards

Practical Learning Objectives

  • Identify self-contained algorithmic components of explainers and understand their functions
  • Connect these building blocks to the explainability requirements unique to the investigated predictive system
  • Select appropriate algorithmic components and tune them to the problem at hand
  • Evaluate these building blocks (in this specific context) independently and when joined together to form the final explainer
  • Interpret the resulting explanations in view of the uncovered properties and limitations of the bespoke explainability algorithm


Introduction to explainability

  • History of explainability
  • Types of explanations
  • Ante-hoc vs. post-hoc discussion, and information lineage (endogenous and exogenous sources of explanatory information)
  • Multi-class explainability
  • Taxonomy and classification of explainability approaches
  • Defining explainability
  • Human-centred perspective
  • Evaluation of explainability techniques
  • Models and data used for this course


A brief overview of data explainability

  • Data as an (implicit) model
  • Data summarisation and description
  • Dimensionality reduction (e.g., t-SNE)
  • Exemplars, prototypes and criticisms


Transparent modelling

  • Linear models
  • Logistic models
  • Generalised additive models
  • Decision trees
  • Rule lists and sets; scoped rules
  • \(k\)-nearest neighbours and \(k\)-means


Feature importance

  • Permutation Importance

  • Partial Dependence-based feature importance

  • Meta-approaches

    • LIME-based feature importance
    • SHAP-based feature importance


Feature influence

  • Individual Conditional Expectation

  • Partial Dependence

  • Marginal Effect

  • Accumulated Local Effect

  • Meta-approaches

    • LIME (linear surrogate)
    • SHAP



  • surrogate explainers

    • local, cohort and global
    • linear and tree-based
  • rules

    • ANCHOR
    • RuleFit
  • SHAP


Instance-based explanations

  • Exemplar explanations
  • Counterfactuals
  • Prototypes and criticisms


  1. Bring-your-own-project

  2. Explain a predictive model (you are working with)

    • develop a bespoke explainability suite for a predictive model of your choice (e.g., for a project you are currently working on, or a model accessible via an API)
    • use multiple explainability techniques and identify the sources of explanation (dis)agreements


  1. Dissect an explainability method – choose an explainability method, identify its core (algorithmic) building blocks and articulate its assumptions, exploring how these different aspects affect the explanations

  2. Build a model-specific or model-agnostic explainer or a transparent model

    • new explainability technique (from existing building blocks)
    • new composition of an existing explainability technique
    • new visualisation of an explanation type


  • Individual or in small groups
  • Projects to be presented or demoed in the last class
  • Some project ideas


Objective: The journey is more important than the outcome

  • Reproducibility of the results (research best practice, open science)

  • Quality of the investigation

  • Due diligence

    • Assumptions
    • Choices (theoretical, algorithmic, implementation and otherwise)
    • Justifications
  • Project results


💽 Interactive visualisation / reporting / dashboarding / presentation software

Wrap Up