ML Interpretability using LIME in R

Overview

• Merely building the model is not enough without stakeholders not being to interpret the outputs of your model
• In this article, understand how to interpret your model using LIME in R

Introduction

I thought spending hours preprocessing the data is the most worthwhile thing in Data Science. That is what my misconception was, as a beginner. Now, I realize, that even more rewarding is being able to explain your predictions and model to a layman who does not understand much about machine learning or other jargon of the field.

Consider this scenario – your problem statement deals with predicting if a patient has cancer or not. Painstakingly, you obtain and clean the data, build a model on it, and after much effort, experimentation, and hyperparameter tuning, you arrive at an accuracy of over 90%. That’s great You walk up to a doctor and tell him that you can predict with 90% certainty that a patient has cancer or not.

However, one question the doctor asks that leaves you stumped – “How can I and the patient trust your prediction when each patient is different from the other and multiple parameters can decide between a malignant and a benign tumor?”

This is where model interpretability comes in – nowadays, there are multiple tools to help you explain your model and model predictions efficiently without getting into the nitty-gritty of the model’s cogs and wheels. These tools include SHAP, Eli5, LIME, etc. Today, we will be dealing with LIME.

In this article, I am going to explain LIME and how it makes interpreting your model easy in R.

What is LIME?

LIME stands for Local Interpretable Model-Agnostic Explanations. First introduced in 2016, the paper which proposed the LIME technique was aptly named “Why Should I Trust You?” Explaining the Predictions of Any Classifier by its authors, Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.

Source

Built on this basic but crucial tenet of trust, the idea behind LIME is to answer the ‘why’ of each prediction and of the entire model. The creators of LIME outline four basic criteria for explanations that must be satisfied:

• The explanations for the predictions should be understandable, i.e. interpretable by the target demographic.
• We should be able to explain individual predictions. The authors call this local fidelity
• The method of explanation should be applicable to all models. This is termed by the authors as the explanation being model-agnostic
• Along with the individual predictions, the model should be explainable in its entirety, i.e. global perspective should be considered

How does LIME work?

Expanding more on how LIME works, the main assumption behind it is that every model works like a simple linear model at the local scale, i.e. at individual row-level data. The paper and the authors do not set out to prove this, but we can go by the intuition that at an individual level, we can fit this simple model on the row and that its prediction will be very close to our complex model’s prediction for that row. Interesting isn’t it?

Further, LIME extends this phenomenon by fitting such simple models around small changes in this individual row and then extracting the important features by comparing the simple model and the complex model’s predictions for that row.

LIME works both on tabular/structured data and on text data as well.

You can read more on how LIME works using Python here, we will be covering how it works using R.

So fire up your Notebooks or R studio, and let us get started!

Using LIME in R

Step 1: The first step is to install LIME and all the other libraries which we will need for this project. If you have already installed them, you can skip this and start with Step 2

install.packages('lime')
install.packages('MASS')
install.packages("randomForest")
install.packages('caret')
install.packages('e1071')

library(lime)
library(MASS)
library(randomForest)
library(caret)
library(e1071)

Since we took up the example of explaining the predictions of whether a patient has cancer or not, we will be using the biopsy dataset. This dataset contains information on 699 patients and their biopsies of breast cancer tumors.

Step 3: We will import this data and also have a look at the first few rows:

data(biopsy)

Step 4: Data Exploration

4.1) We will first remove the ID column since it is just an identifier and of no use to us

biopsy$ID <- NULL

4.2) Let us rename the rest of the columns so that while visualizing the explanations we have a clearer idea of the feature names as we understand the predictions using LIME.

names(biopsy) <- c('clump thickness', 'uniformity cell size',
'uniformity cell shape', 'marginal adhesion','single epithelial cell size',
'bare nuclei', 'bland chromatin', 'normal nucleoli', 'mitoses','class')

4.3) Next, we will check if there are any missing values. If so, we will first have to deal with them before proceeding any further.

sum(is.na(biopsy))

biopsy <- na.omit(biopsy)
sum(is.na(biopsy))

head(biopsy, 5)

## 75% of the sample size

smp_size <- floor(0.75 * nrow(biopsy))

## set the seed to make your partition reproducible - similar to random state in Python
set.seed(123)
train_ind <- sample(seq_len(nrow(biopsy)), size = smp_size)

train_biopsy <- biopsy[train_ind, ]
test_biopsy <- biopsy[-train_ind, ]

cat(dim(train_biopsy), dim(test_biopsy))

model_rf <- caret::train(class ~ ., data = train_biopsy,method = "rf", #random forest
trControl = trainControl(method = "repeatedcv", number = 10,repeats = 5, verboseIter = FALSE))

model_rf

biopsy_rf_pred <- predict(model_rf, test_biopsy)
confusionMatrix(biopsy_rf_pred, as.factor(test_biopsy$class))

explainer <- lime(train_biopsy, model_rf)

explanation <- explain(test_biopsy[15:20, ], explainer, n_labels = 1, n_features = 5)

explanation <- explain(test_biopsy[93, ], explainer, n_labels = 1, n_features = 10)
plot_features(explanation)