(Feature Influence)

Kacper Sokol

ICE captures the

response of a predictive modelfor asingle instancewhenvarying one of its features(Goldstein et al. 2015).

It communicates

local(with respect to a single instance)feature influence.

Property |
Individual Conditional Expectation |
---|---|

relation |
post-hoc |

compatibility |
model-agnostic |

modelling |
regression, crisp and probabilistic classification |

scope |
local (per instance; generalises to cohort or global) |

target |
prediction (generalises to model) |

Property |
Individual Conditional Expectation |
---|---|

data |
tabular |

features |
numerical and categorical |

explanation |
feature influence (visualisation) |

caveats |
feature correlation, unrealistic instances |

\[ X_{\mathit{ICE}} \subseteq \mathcal{X} \]

\[ V_i = \{ v_i^{\mathit{min}} , \ldots , v_i^{\mathit{max}} \} \]

\[ f \left( x_{\setminus i} , x_i=v_i \right) \;\; \forall \; x \in X_{\mathit{ICE}} \; \forall \; v_i \in V_i \]

\[ f \left( x_{\setminus i} , x_i=V_i \right) \;\; \forall \; x \in X_{\mathit{ICE}} \]

Original notation (Goldstein et al. 2015)

\[ \left\{ \left( x_{S}^{(i)} , x_{C}^{(i)} \right) \right\}_{i=1}^N \]

\[ \hat{f}_S^{(i)} = \hat{f} \left( x_{S}^{(i)} , x_{C}^{(i)} \right) \]

Centres ICE curves by anchoring them at a fixed point, usually the lower end of the explained feature range.

\[ f \left( x_{\setminus i} , x_i=V_i \right) - f \left( x_{\setminus i} , x_i=v_i^{\mathit{min}} \right) \;\; \forall \; x \in X_{\mathit{ICE}} \]

or

\[ \hat{f} \left( x_{S}^{(i)} , x_{C}^{(i)} \right) - \hat{f} \left( x^{\star} , x_{C}^{(i)} \right) \]

Visualises

interaction effectsbetween the explained and remaining features by calculating the partial derivative of the explained model \(f\) with respect to the explained feature \(x_i\).

- When no interactions are present, all curves overlap.
- When interactions exist, the lines will be heterogeneous.

\[ f \left( x_{\setminus i} , x_i \right) = g \left( x_i \right) + h \left( x_{\setminus i} \right) \;\; \text{so that} \;\; \frac{\partial f(x)}{\partial x_i} = g^\prime(x_i) \]

or

\[ \hat{f} \left( x_{S} , x_{C} \right) = g \left( x_{S} \right) + h \left( x_{C} \right) \;\; \text{so that} \;\; \frac{\partial \hat{f}(x)}{\partial x_{S}} = g^\prime(x_{S}) \]

**Easy to generate and interpret**- Spanning multiple instances allows to capture the diversity (heterogeneity) of the model’s behaviour

- Assumes
**feature independence**, which is often unreasonable - ICE may not reflect the true behaviour of the model since it displays the behaviour of the model for
**unrealistic instances** - May be
**unreliable for certain values**of the explained feature when its values are not uniformly distributed (abated by a**rug plot**) - Limited to explaining
**one feature at a time**

- Averaging ICEs gives
*Partial Dependence (PD)* - Generating ICEs may be computationally expensive for
*large sets of data*and*wide feature intervals*with a*small “inspection” step* - Computational complexity: \(\mathcal{O} \left( n \times d \right)\), where
- \(n\) is the number of instances in the designated data set and
- \(d\) is the number of steps within the designated feature interval

Under certain (quite restrictive) assumptions, ICE is admissible to a causal interpretation (Zhao and Hastie 2021).

See Causal Interpretation of Partial Dependence (PD) for more detail.

Model-focused (global) “version” of

Individual Conditional Expectation, which is calculated byaveragingICE across a collection of data points (Friedman 2001). It communicates the average influence of a specific feature value on the model’s prediction byfixing the value of this featureacross a designated set of instances.

It communicates the influence of a specific feature value – or similar values, i.e., an interval around the selected value – on the model’s prediction by

only considering relevant instancesfound in the designated data set. It is calculated asthe average prediction of these instances.

It communicates the influence of a specific feature value on the model’s prediction by quantifying the average (accumulated) difference between the predictions at the boundaries of a (small)

fixed intervalaround the selected feature value (Apley and Zhu 2020). It is calculated by replacing the value of the explained feature with the interval boundaries forinstances found in the designated data setwhose value of this feature is within the specified range.

Python | R |
---|---|

scikit-learn (`>=0.24.0` ) |
iml |

PyCEbox | ICEbox |

alibi | pdp |

DALEX |

- ICE paper (Goldstein et al. 2015)
*Interpretable Machine Learning*book- scikit-learn example
- FAT Forensics example and tutorial

Apley, Daniel W, and Jingyu Zhu. 2020. “Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models.” *Journal of the Royal Statistical Society: Series B (Statistical Methodology)* 82 (4): 1059–86.

Friedman, Jerome H. 2001. “Greedy Function Approximation: A Gradient Boosting Machine.” *Annals of Statistics*, 1189–1232.

Goldstein, Alex, Adam Kapelner, Justin Bleich, and Emil Pitkin. 2015. “Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation.” *Journal of Computational and Graphical Statistics* 24 (1): 44–65.

Zhao, Qingyuan, and Trevor Hastie. 2021. “Causal Interpretations of Black-Box Models.” *Journal of Business & Economic Statistics* 39 (1): 272–81.