Using the ‘What-If Tool’ to investigate Machine Learning models.4 min read

An open source tool from Google to easily analyze ML models without the need to code.

Parul PandeyMay 3

Photo by Pixabay from Pexels

Good practitioners act as Detectives, probing to understand their model better¹

In this era of explainable and interpretable Machine Learning, one merely cannot be content with simply training the model and obtaining predictions from it. To be able to really make an impact and obtain good results, we should also be able to probe and investigate our models. Apart from that, algorithmic fairness constraints and bias should also be clearly kept in mind before going ahead with the model.

Investigating a model requires asking a lot of questions and one needs to have an acumen of a detective to probe and look for issues and inconsistencies within the models. Also, such a task is usually complex requiring to write a lot of custom code. Fortunately, the What-If Tool has been created to address this issue making it easier for a broad set of people to examine, evaluate, and debug ML systems easily and accurately.

What-If Tool(WIT)

Source

What-If Tool is an interactive visual tool that is designed to investigate the Machine Learning models. Abbreviated as WIT, it enables the understanding of a Classification or Regression model by enabling people to examine, evaluate, and compare machine learning models. Due to its user-friendly interface and less dependency on complex coding, everyone from a developer, a product manager, a researcher or a student can use it for their purpose.

WIT is an open-source visualisation tool released by Google under the PAIR(People + AI Research) initiative. PAIR brings together researchers across Google to study and redesign the ways people interact with AI systems.

The tool can be accessed through TensorBoard or as an extension in a Jupyter or Colabnotebook.

Advantages

The purpose of the tool is to give people a simple, intuitive, and a powerful way to play with a trained ML model on a set of data through a visual interface only. Here are the major advantages of WIT.

What can you do with the What-If Tool?

We shall cover all the above points during an example walkthrough using the tool.

Demos

To illustrate the capabilities of the What-If Tool, the PAIR team has released a set of demos using pre-trained models. You can either run the demos in the notebook or directly through the web.

Usage

WIT can be used inside a Jupyter or Colab notebook, or inside the TensorBoard web application. This has been nicely and clearly explained in the documentation and I highly encourage you to go through that since explaining the entire process wouldn’t be possible through this short article.

The whole idea is to first train a model and then visualizes the results of the trained classifier on test data using the What-If Tool.

Using WIT with Tensorboard

To use WIT within TensorBoard, your model needs to be served through a TensorFlow Model Server, and the data to be analyzed must be available on disk as a TFRecords file. For more details, refer to the documentation for using WIT in TensorBoard.

Using WIT with Notebooks

To be able to access WIT within notebooks, you need a WitConfigBuilder object that specifies the data and model to be analyzed. This documentation provides a step-by-step outline for using WIT in a notebook.

Walkthrough

Let’s now explore the capabilities of the WIT tool with an example. The example has been taken from the demos provided on the website and is called Income Classification wherein we need to predict whether a person earns more than $50k a year, based on their census information. The Dataset belongs to the UCI Census datasetconsisting of a number of attributes such as age, marital status and education level.

Overview

Let’s begin by doing some Exploration of the dataset. Here is a link to the web Demo for following along.

What-if tool contains two main panels. The right panel contains a visualization of the individual data points in the data set you have loaded.

Effect of gender on Model’s performance

Conclusion

This was just a quick run-through of some of the what if tools features. WIT is a pretty handy tool which gives the ability to probe the models, into the hands of the people to whom it matters the most. Simply creating and training a model isn’t the purpose of Machine Learning but understanding why and how that model was created is Machine Learning in true sense.

References:

The What-If Tool: Code-Free Probing of Machine Learning Modelshttps://pair-code.github.io/what-if-tool/walkthrough.htmlhttps://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/interactive_inferenceMachine LearningArtificial IntelligenceData ScienceGoogleTowards Data Science347 claps1Follow

Parul Pandey

Medium member since Aug 2018

Trying to break the Data Science jargons for masses. Linkedin: https://www.linkedin.com/in/parul-pandey-a5498975/

Source: Towards data science

Judith Chao Andrade

Apasionada del conocimiento, de compartirlo y de aprender de todo lo que me rodea, disfruto aprendiendo y realizando actividades. Actualmente estoy aprendiendo programación pero me fascinan los temas relacionados con los materiales especiales, las cuiriosidades, el humor, los eventos, las redes sociales ... Mi mayor interés podría decir que es no perder nunca la cuiriosidad por lo que si tienes un plan en mente solo proponlo !.

Deja un comentario

Tu dirección de correo electrónico no será publicada. Los campos obligatorios están marcados con *

X
X
X
X