Counting 3,834 Big Data & Machine Learning Frameworks, Toolsets, and Examples...
Suggestion? Feedback? Tweet @stkim1

Author
Contributors
Last Commit
May. 17, 2019
Created
Oct. 18, 2018

DrWhy

DrWhy is the collection of tools for Explainable AI (XAI). It's based on shared principles and simple grammar for exploration, explanation and visualisation of predictive models.

Please, note that DrWhy is under rapid development and is still maturing. If you are looking for a stable solution, please use the mature DALEX package.

Visual Exploration, Explanation and Debugging

The unified grammar beyond DrWhy.AI universe is described in the Predictive Models: Visual Exploration, Explanation and Debugging book.

Lifecycle for Predictive Models

It takes a village to raise a child model.

Tools that are usefull during the model lifetime. MI2 stands for our internal tools.

images/DrWhyAI.png

1. Data Acquisition

  • dataMaid; A Suite of Checks for Identification of Potential Errors in a Data Frame as Part of the Data Screening Process
  • ggplot2; System for declaratively creating graphics, based on The Grammar of Graphics.

2. Feature Selection

  • Model Agnostic Variable Importance Scores. Surrogate learning = Train an elastic model and measure feature importance in such model. See DALEX, Model Class Reliance MCR
  • vip Variable importance plots

3. Feature Engineering

  • SAFE MI2 Surrogate learning = Train an elastic model and extract feature transformations.
  • xspliner MI2 Using surrogate black-boxes to train interpretable spline based additive models
  • factorMerger MI2 Set of tools for factors merging paper
  • ingredients MI2 Set of tools for model level feature effects and feature importance.

4. Model Tuning

5. Model Validation

  • auditor MI2 model verification, validation, and error analysis vigniette
  • DALEX MI2 Descriptive mAchine Learning EXplanations
  • iml; interpretable machine learning R package
  • randomForestExplainer MI2 A set of tools to understand what is happening inside a Random Forest
  • survxai MI2 Explanations for survival models paper

6. Model Deployment

7. Model Maintenance

  • drifter MI2 Concept Drift and Concept Shift Detection for Predictive Models
  • archivist MI2 A set of tools for datasets and plots archiving paper

DrWhy.AI indicator panel

Active development and maintenance

These packages are actively developed and have active maintainer.

Experimental pre-seed phase (under active development)

Experimental or without maintenance (looking for maintainer!!!)

These packages contain useful features, are still in use but we are looking for an active maintainer.

  • randomForestExplainer CRAN_Status_Badge Build Status Coverage Status Downloads
  • factorMerger CRAN_Status_Badge Build Status Coverage Status Downloads
  • cr17 CRAN_Status_Badge Build Status Coverage Status Downloads
  • MLGenSig CRAN_Status_Badge Build Status Coverage Status Downloads
  • modelDown CRAN_Status_Badge Build Status Coverage Status
  • pyBreakDown Build Status Coverage Status

In the sunset phase, without maintenance

Key features from these packages are copied to another packages.

  • ceterisParibus CRAN_Status_Badge Build Status Coverage Status Downloads (development moved to ingredients)
  • ceterisParibus2 CRAN_Status_Badge Build Status Coverage Status (development moved to ingredients)
  • DALEX2 CRAN_Status_Badge Build Status Coverage StatusDownloads (development moved to DALEX)
  • breakDown CRAN_Status_Badge Build Status Coverage Status Downloads (development moved to iBreakDown)
  • live CRAN_Status_Badge Total Downloads Build Status Coverage StatusDownloads (development moved to localModel)

Family of Model Explainers

Family of Model Explainers

Architecture of DrWhy

DrWhy works on fully trained predictive models. Models can be created with any tool.

DrWhy uses DALEX2 package to wrap model with additional matadata required for explanations, like validation data, predict function etc.

Explainers for predictive models can be created with model agnostic or model specific functions implemented in various packages.

Architecture of DrWhy

Hype_Cycle