dime: Deep Interactive Model Explanations

Hubert Baniecki created an awesome package dime for serverless HTML interactive model exploration. The experimental version is at Github, here is the pkgdown website. It is a part of the DrWhy.AI project.

How does it work?

With the DALEX package you can create local and global model explanations for machine learning models. Each explanation can be visualized with a genetic plot() function.
Hubert created a generic plotD3() function which turns each explanation into an interactive D3 plot (with the help of r2d3 package). With the dime package you can combine few interactive explanations into a single dashboard. And the dashboard is serverless, you can host it at github or anywhere.

For example, the gif below shows how to combine a break down plot (local feature attribution) with ceteris paribus profiles (detailed analysis of a single variable). You can click a variable of interest to activate an appropriate ceteris paribus profile (click to play).

With the dime package you can combine any number of interactive widgets into a single dashboard. You can connect local, global explanations or EDA tools like histograms or barplots.

It’s very easy to generate such website. Just create an explainer and call the modelStudio() function.

Find examples and R codes here: https://github.com/ModelOriented/dime/blob/master/README.md

The dime package is still in the experimental phase. Your feedback is welcomed. Feel free to submit an issue with comments or ideas.

Learn about XAI in R with ,,Predictive Models: Explore, Explain, and Debug”

XAI (eXplainable artificial intelligence) is a fast growing and super interesting area.
Working with complex models generates lots of problems with model validation (on test data performance is great but drops at production), model bias, lack of stability and many others. We need more than just local explanations for predictive models.

The more complex are models the better tools are needed to understand how models are working, explore model behaviour and debug potential errors.

Two years ago I’ve initiated work on the DALEX package. Library packed with functions for local and global model exploration.
Over the time the package went through few architectural changes and now it is part of a larger universe of tools for model exploration developed at MI2DataLab with an increasing support of external contributors (join us).

To explain our philosophy behind the model exploration we (together with Tomasz Burzykowski from Hasselt) started a book ,,Predictive Models: Explore, Explain, and Debug’’.

First part, devoted to local exploration, is ready to read. It explains how to use DALEX with iBreakDown and ingredients packages for instance level explanations.
Later we will describe other packages from our universe.

Find the book-down version of here.

Find a one-page-cheatsheet here.

Let us improve these descriptions by adding pull requests or issues at the GitHub repo.
One day there will be a paper version 😉

Ile punktów potrzeba by się dostać do szkoły średniej w Warszawie?

W tym artykule Polityki przeczytałem, że ponad 3 tysiące uczniów nie dostało się do żadnej z wybranych szkół średnich w Warszawie. Pomimo wysiłku szkół by przyjąć możliwie wielu uczniów.

Marcin Luckner (MiNI PW) przesłał mi ciekawą analizę progów punktowych w różnych oddziałach w Warszawie. Poniżej umieszczam wybrane wykresy po drobnych zmianach. Dane pochodzą z serwisu edukacja.warszawa.pl. Przy okazji też będziemy mogli porównać kilka sposobów pokazywania rozkładów.

W powyższych danych znajduje się informacja ile punktów było potrzeba aby dostać się do wskazanego oddziału we wskazanej szkole średniej. W rozbiciu na typ szkoły i na to czy rekrutowały się dzieci z podstawówek czy gimnazjów.
Poniższy wykres (histogram) pokazuje jak wyglądają progi punktowe w różnych typach oddziałów. Na wykresie nie ma szkół sportowych, ponieważ tam były dodatkowe punkty sprawnościowe i trudno te progi porównać.

W różnych mediach można znaleźć informację o uczniu, który miał 190 punktów i nie dostał się do żadnej wybranej szkoły. Ale były też szkoły, które miały znacznie niższe progi przyjęcia. Bardzo wiele oddziałów miało progi przyjecia w okolicy 160 punktów.

John Tukey lata temu zaproponował by rozkłady opisywać za pomocą piątki liczb – min, max, mediana i kwartyle. To 5 liczb które dzieli wartości na 4 równe przedziały. Można je pokazać za pomocą wykresów pudełkowych.

Poniżej mamy wykresy pudełkowe z rozkładem progów punktowych podziałem na dzielnice. Im szersze pudełko tym więcej szkół jest w danej grupie. Najwyższe progi były w szkołach w Śródmieściu (ponad połowa oddziałów miała próg przyjęcia powyżej 165 punktów). łatwiej było się dostać do szkół średnich na Pradze czy w Ursusie.

Okazuje się, że i moją i Marcina ulubioną techniką pokazywania rozkładów jest dystrybuanta empiryczna. Wykres poniżej pokazuje jaki procent oddziałów ma prób przyjęcia mniejszy niż x.

Przykładowo szara linia odpowiada progowi 150 punktów. Tyle punktów wystarczyły by dostać się do praktycznie wszystkich oddziałów integracyjnych, ale już tylko do około 60% oddziałów ogólnych (1 na 3 oddziały ogólne ma wyższy prób punktowy), do około 33% oddziałów w szkołach dwujęzycznych (2 na 3 oddziały w szkołach dwujęzycznych ma wyższy próg przyjęcia). Nie wystarczy na szkoły z międzynarodową maturą.

To jaki jest Wasz ulubiony sposób pokazywania rozkładów?

modelDown is now on CRAN!


The modelDown package turns classification or regression models into HTML static websites.
With one command you can convert one or more models into a website with visual and tabular model summaries. Summaries like model performance, feature importance, single feature response profiles and basic model audits.

The modelDown uses DALEX explainers. So it’s model agnostic (feel free to combine random forest with glm), easy to extend and parameterise.

Here you can browse an example website automatically created for 4 classification models (random forest, gradient boosting, support vector machines, k-nearest neighbours). The R code beyond this example is here.

Fun facts:

archivist hooks are generated for every documented object. So you can easily extract R objects from the HTML website. Try

archivist::aread("MI2DataLab/modelDown_example/docs/repository/574defd6a96ecf7e5a4026699971b1d7")

– session info is automatically recorded. So you can check version of packages available at model development (https://github.com/MI2DataLab/modelDown_example/blob/master/docs/session_info/session_info.txt)

– This package is initially created by Magda Tatarynowicz, Kamil Romaszko, Mateusz Urbański from Warsaw University of Technology as a student project.

xaibot – conversations with predictive models!


If you could talk to a predictive machine learning model, what would you ask for?

Try! Michał Kuźba is developing a mind-blowing project – xai chat-bot. Dialog based system that helps to explore and understand predictive models through natural language conversations (type, speak or phone the model 😉 ).

For example, imagine that you have a random forest model that predicts survival for titanic data. With xai-bot you can chat about your chances of survival, variables that influence survival, options that you have to increase your odds or just chat about life models.


The chatbot is based on dialog-flow google infrastructure. It communicates with DALEX explainers written in R through plumber REST API.

Find the chatbot here: https://kmichael08.github.io.

The project is under development, but the bot is already pretty smart.

So, have fun!

How to design a model visualisation @ Gdansk satRdays


I had amazing weekend in Gdansk thanks to the satRday conference organized by Olgun Aydin, Ania Rybinska and Michal Maj.

Together with Hanna Piotrowska we had a talk ,,Machine learning meets design. Design meets machine learning”. Hanna redesigned DALEX visualisations (DALEX is a set of tools for visual explanation of predictive ML models). During the talk she explained what and why was changed.

See for example the metamorphosis of the Break Down explainer. How many differences can you spot?

Every change (axis, reading order, spacing, colors, descriptions, background, annotations) serves some purpose.

Find our presentation at slideshare.

List of satRday talks (machine learning was quite popular).

Hanna design is implemented in ggplot2 thanks to Tomasz Mikołajczyk and in D3 thanks to Huber Baniecki! Find more examples of how to use new plots here.

Make it explainable!

Most people make the mistake of thinking design is what it looks like… People think it’s this veneer — that the designers are handed this box and told, ‚Make it look good!’ That’s not what we think design is. It’s not just what it looks like and feels like. Design is how it works.

Steve Jobs, The New York Times, 2003.

Same goes with interpretable machine learning.
Recently, I am talking a lot about interpretations and explainability. And sometimes I got impression that techniques like SHAP, Break Down, LIME, SAFE are treated like magical incantations that converts complex predictive models into ,,something interpretable’’.

But interpretability/explainability is not a binary feature that you have it or not. It’s a process. The goal is to increase our understanding of the model behavior. Try different techniques to broaden the knowledge about the model or about model predictions.
Maybe you will never explain 100%, but you will understand more.

XAI/IML (eXplainable Artificial Intelligence/Interpretable Machine Learning) techniques can be used not only for post-hoc explainability, but also for model maintenance, debugging or in early phases of crisp modeling. Visual tools like PDP/ALE/CeterisParibus will change the way how we approach modeling and how we interact with models. We as model developers, model auditors or users.

Together with Tomasz Burzykowski from UHasselt we work on a book about the methodology for visual exploration, explanation and debugging predictive models.

Find the early version here https://pbiecek.github.io/PM_VEE/.

There is a lot of R snippets that shows how to use DALEX (and sometimes other packages like shapper, ingredients, iml, iBreakDown, condvis, localModel, pdp) to better understand some aspects of your predictive model.

It’s a work in process and even in an early dirty phase (despite the fact that we have started a year ago).
Feel free to comment it, or suggest improvements. Easiest way to do this is to add a new issue.

Code snippets are fully thanks to archivist hooks. I think that it’s a first book that uses archivist hooks for blended experience. You can read about a model online and in just one line of code you can download an object to your R console.

First chapters show how to use Ceteris Paribus Profiles / Individual Conditional Expectations to perform what-if/sensitivity analysis of a model.

DALEX for keras and parsnip

DALEX is a set of tools for explanation, exploration and debugging of predictive models. The nice thing about it is that it can be easily connected to different model factories.

Recently Michal Maj wrote a nice vignette how to use DALEX with models created in keras (an open-source neural-network library in python with an R interface created by RStudio). Find the vignette here.
Michal compared a keras model against deeplearning from h2o package, so you can check which model won on the Titanic dataset.

Next nice vignette was created by Szymon Maksymiuk. In this vignette Szymon shows how to use DALEX with parsnip models (parsnip is a part of the tidymodels ecosystem, created by Max Kuhn and Davis Vaughan). Models like boost_tree, mlp and svm_rbf are competing on the Titanic data.

These two new vignettes add to our collection how to use DALEX with mlr, caret, h2o and others model factories.

Explore the landscape of R packages for automated data exploration

Do you spend a lot of time on data exploration? If yes, then you will like today’s post about AutoEDA written by Mateusz Staniak.

If you ever dreamt of automating the first, laborious part of data analysis when you get to know the variables, print descriptive statistics, draw a lot of histograms and scatter plots – you weren’t the only one. Turns out that a lot of R developers and users thought of the same thing. There are over a dozen R packages for automated Exploratory Data Analysis and the interest in them is growing quickly. Let’s just look at this plot of number of downloads from the official CRAN repository.

Replicate this plot with

New tools arrive each year with a variety of functionalities: creating summary tables, initial visualization of a dataset, finding invalid values, univariate exploration (descriptive and visual) and searching for bivariate relationships.

We compiled a list of R packages dedicated to automated EDA, where we describe twelve packages: their capabilities, their strong aspects and possible extensions. You can read our review paper on arxiv: https://arxiv.org/abs/1904.02101.

Spoiler alert: currently, automated means simply fast. The packages that we describe can perform typical data analysis tasks, like drawing bar plot for each categorical feature, creating a table of summary statistics, plotting correlations, with a single command. While this speeds up the work significantly, it can be problematic for high-dimensional data and it does not take the advantage of AI tools for actual automatization. There is a lot of potential for intelligent data exploration (or model exploration) tools.

More extensive list of software (including Python libraries and web applications) and papers is available on Mateusz’s GitHub. Researches can follow our autoEDA project on ResearchGate.

iBreakDown: faster, prettier and more precise explanations for predictive models (with interactions)

LIME and SHAP are two very popular methods for instance level explanations of machine learning models (XAI).
They work nicely for images and text inputs, but share similar weakness in case of tabular data: explanations are additive while complex models are (sometimes) not. iBreakDown addresses this problem.

iBreakDown is a a successor of the breakDown package. Yesterday it has arrived on CRAN. Key new features are:

– It identifies and shows feature interactions (if there are local interactions in the model).
– It is much faster. For additive explanations the complexity is O(p) instead of O(p^2).
– The plotD3 function creates an interactive D3-based break-down plot (thanks to r2d3).
– iBreakDown has a new design, created by Hanna Dyrcz. We will have a talk about it ,,Machine learning meets design. Design meets machine learning.” at satRdays. Try the new theme theme_drwhy()!.
– It shows explanation level uncertainty – how good are explanations?

A methodology behind this package is described in the iBreakDown: Uncertainty of Model Explanations for Non-additive Predictive Models.

A nice titanic-powered use-case is described in the titanic vignette.

An example of the D3 interactive explainer is here.

Some intuition is introduced in the Visual Exploration, Explanation and Debugging (working version, still in progress).

iBreakDown is a part of the DrWhy.AI family of explainers consistent with the DALEX.

Let us know if you like it. Feel free to create a pull request with new features, add issue with new idea or star the github repository if you like this package.