Purpose
Captum is a model interpretability library based on PyTorch, designed to help researchers and developers understand the basis for predictions of PyTorch models. It supports interpretability analysis for multiple modalities including vision and text.
Main Capabilities
- Provides interpretability support for PyTorch models, covering multiple modalities such as vision and text.
- Built on top of PyTorch, compatible with most types of PyTorch models, and can be used with minimal modifications to the original neural network.
- As an open-source general-purpose library, it is aimed at interpretability research, making it easy to implement and benchmark new algorithms.
Usage
Install Captum:
- Via conda (recommended):
conda install captum -c pytorch - Via pip:
pip install captum
Create and prepare the model (official example):
import numpy as np
import torch
import torch.nn as nn
from captum.attr import IntegratedGradients
class ToyModel(nn.Module):
def __init__(self):
super().__init__()
self.lin1 = nn.Linear(3, 3)
self.relu = nn.ReLU()
self.lin2 = nn.Linear(3, 2)
# initialize weights and biases
self.lin1.weight = nn.Parameter(torch.arange(-4.0, 5.0).view(3, 3))
self.lin1.bias = nn.Parameter(torch.zeros(1, 3))
self.lin2.weight = nn.Parameter(torch.arange(-3.0, 3.0).view(2, 3))
self.lin2.bias = nn.Parameter(torch.ones(1, 2))
def forward(self, input):
return self.lin2(self.relu(self.lin1(input)))
model = ToyModel()
model.eval()
To make computations reproducible, fix the random seed:
torch.manual_seed(123)
np.random.seed(123)
Define the input and baseline tensors:
input = torch.rand(2, 3)
baseline = torch.zeros(2, 3)
Select an algorithm to instantiate and apply (in this example, Integrated Gradients):
ig = IntegratedGradients(model)
attributions, delta = ig.attribute(input, baseline, target=0, return_convergence_delta=True)
print('IG Attributions:', attributions)
print('Convergence Delta:', delta)
Example output:
IG Attributions: tensor([[-0.5922, -1.5497, -1.0067],
[ 0.0000, -0.2219, -5.1991]])
Convergence Delta: tensor([ 2.3842e-07, -4.7684e-07])
Applicable Scenarios
Suitable for scenarios that require interpretability analysis of PyTorch models, especially research and development involving multimodal tasks such as vision and text.
Limitations
No further limitations are provided at this time.
