Explainable AI: What, Why and How?

Artificial intelligence (AI) is transforming various domains such as healthcare, finance, education, entertainment, and more. However, many AI models are often seen as “black boxes” that do not reveal how they make decisions or predictions. This can lead to a lack of trust, accountability, fairness, and transparency in AI systems.

This is where explainable AI (XAI) comes in. XAI is a set of processes and methods that allows human users to comprehend and trust the results and output created by machine learning algorithms12. XAI helps characterize model accuracy, fairness, transparency, and outcomes in AI-powered decision making2.

But why do we need explainable AI?

Here are some benefits of XAI:

• Building trust in AI: With XAI, the organisations building AI models can bring trust and confidence. They can also effectively operationalise AI with interpretability and explainability of models3.
• Faster AI results: XAI allows organisations to systematically monitor and manage models to optimise business outcomes. It also enables faster feedback loops and iterations for model improvement3.
• Regulatory compliance: With an AI model that is explainable and transparent, organisations can manage regulatory compliance. For example, GDPR requires organisations to provide meaningful information about the logic involved in automated decision making3.
• Reducing cost of mistakes: Decision-sensitive fields such as medicine, finance, legal etc., are highly affected by the accuracy and reliability of AI models. XAI can help reduce errors and risks by providing explanations for model outputs2.
• Reducing impact of model biasing: AI models have shown significant evidence of bias. Examples include gender bias for hiring decisions or racial bias for criminal justice outcomes. XAI can help detect and mitigate bias by revealing how data influences model outputs2.

How can we achieve explainable AI?

There are different approaches depending on the type of model, data, task, user etc. Some examples are:

• Feature attribution: This method assigns scores or weights to each input feature based on how much they contribute to the output. For example Google Cloud’s Explainable AI provides feature attributions for image classification models 4.
• Model visualization: This method uses graphs or charts to show how a model works internally or how it responds to different inputs. For example TensorFlow Playground allows users to interactively visualize neural networks.
• Natural language explanation: This method generates textual descriptions or summaries that explain how a model works or why it made a certain decision. For example LIME (Local Interpretable Model-Agnostic Explanations) creates natural language explanations for any classifier.

Explainable AI is an emerging field that aims to make AI more understandable and trustworthy for human users. It has many benefits such as building trust, improving performance, ensuring compliance etc.

• Explainable Machine Learning with LIME and H2O in R1: This is a video course that teaches you how to use LIME and H2O packages in R to explain machine learning models.
• An Introduction to Explainable AI, and Why We Need it2: This is an online tutorial that introduces the concept, motivation, and challenges of explainable AI.
• Getting a Window into your Black Box Model3: This is a blog post that shows you how to use SHAP (SHapley Additive exPlanations) library in Python to interpret black box models.
• Explainable AI: Scene Classification and GradCam Visualization4: This is a notebook that demonstrates how to use Grad-CAM (Gradient-weighted Class Activation Mapping) technique to visualize the regions of an image that influence the model’s prediction.
• Explaining Quantitative Measures of Fairness: This is a notebook that explains how to measure and mitigate fairness issues in machine learning models using AIF360 (AI Fairness 360) toolkit.
• Interpretable Machine Learning Applications: Part 1 & 2: These are two blog posts that present some real-world applications of interpretable machine learning such as credit scoring, fraud detection, and customer churn prediction.
• Responsible Machine Learning with Python: This is a book that covers various topics related to responsible machine learning such as ethics, privacy, security, accountability etc.

A use case of explainable AI

One possible use case of explainable AI is credit risk scoring. Credit risk scoring is the process of assessing the likelihood of a borrower defaulting on a loan or credit card payment. Credit risk scoring is important for lenders to make informed decisions and reduce losses.

Traditionally, credit risk scoring models are based on logistic regression or decision trees, which are relatively easy to interpret and explain. However, these models may not capture the complex and nonlinear relationships among various features that affect credit risk. Therefore, some lenders may prefer to use more advanced machine learning models such as neural networks or gradient boosting trees, which can achieve higher accuracy and performance.

However, these models are often considered as black boxes, meaning that their internal logic and reasoning are not transparent or understandable to human users. This poses several challenges for lenders, such as:

• How to justify and communicate the model’s predictions to customers, regulators, auditors, etc.?
• How to identify and mitigate potential biases or errors in the model’s predictions?
• How to improve customer trust and satisfaction with the model’s predictions?

To address these challenges, explainable AI techniques can be used to provide insights into how the black box model works and why it makes certain predictions. For example:

• LIME (Local Interpretable Model-Agnostic Explanations) is a technique that creates local explanations for individual predictions by approximating the black box model with a simpler interpretable model (such as linear regression) in the vicinity of the prediction.
• SHAP (SHapley Additive exPlanations) is a technique that assigns feature importance values to each feature based on how much they contribute to the prediction by using game theory concepts.
• Grad-CAM (Gradient-weighted Class Activation Mapping) is a technique that visualizes the regions of an image that influence the prediction by using gradients of the output layer with respect to convolutional feature maps.

Here is an example of how LIME can be used to explain a neural network model for credit risk scoring using Python:

One potential use case for explainable AI is in the medical field, where it can be important to understand how a model arrives at its conclusions in order to ensure patient safety.

For this use case, we can consider a model that predicts the risk of a patient having a particular medical condition based on various inputs, such as age, sex, and medical history. We will use the SHAP (SHapley Additive exPlanations) library to provide an explanation for the model’s predictions. SHAP is a popular explainability tool that uses Shapley values to explain individual feature contributions to the model output.

Here is some example code that trains a simple model to predict the risk of a medical condition based on age and sex:

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

# Load and preprocess data
data = pd.read_csv('medical_data.csv')
data['sex'] = data['sex'].map({'M': 0, 'F': 1})
X = data[['age', 'sex']]
y = data['condition']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Train logistic regression model
model = LogisticRegression()
model.fit(X_train, y_train)

# Evaluate model performance
print('Accuracy:', model.score(X_test, y_test))  

Once we have our trained model, we can use SHAP to provide explanations for individual predictions. Here is an example of how to use SHAP to explain the prediction for a single patient:

import shap

# Initialize explainer with trained model and training data
explainer = shap.Explainer(model, X_train)

# Explain a single prediction for a patient with age 40 and female sex
patient = pd.DataFrame({'age': 40, 'sex': 1}, index=[0])
shap_values = explainer(patient)

# Print feature contributions
shap.plots.waterfall(shap_values[0])

This code will produce a waterfall plot showing the individual contributions of age and sex to the model’s prediction for the given patient. This can help a medical professional understand why the model arrived at its particular prediction, and potentially identify cases where the model may be making incorrect predictions due to biases or other issues.

Overall, this is just one example of how explainable AI can be used in the medical field, and the exact approach will depend on the specific use case and requirements. However, by using tools like SHAP, it is possible to provide explanations for machine learning models that can help build trust and ensure patient safety.

To use the code in an actual database, we will need to have a database with medical data that we can use to train the model and make predictions. Here’s an example of how we can modify the code to work with a SQL database using Python’s pandas and sqlite3 modules:

import pandas as pd
import sqlite3
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
import shap

# Connect to the database and load data
conn = sqlite3.connect('medical_data.db')
data = pd.read_sql_query('SELECT * FROM patients', conn)

# Preprocess data
data['sex'] = data['sex'].map({'M': 0, 'F': 1})
X = data[['age', 'sex']]
y = data['condition']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Train logistic regression model
model = LogisticRegression()
model.fit(X_train, y_train)

# Initialize SHAP explainer with trained model and training data
explainer = shap.Explainer(model, X_train)

# Select a patient from the database to explain
patient_id = 1
patient_data = pd.read_sql_query(f'SELECT age, sex FROM patients WHERE id = {patient_id}', conn)

# Get SHAP values for the patient
shap_values = explainer(patient_data)

# Print feature contributions
shap.plots.waterfall(shap_values[0])

In this example, I assume that the medical data is stored in a SQLite database with a table called patients that contains columns for id, age, sex, and condition. We connect to the database using sqlite3 and load the data using pandas.

We then preprocess the data and train a logistic regression model on the training set. Once the model is trained, we initialize a SHAP explainer with the trained model and training data.

We then select a patient from the database that we want to explain (in this case, patient with id=1). We use pd.read_sql_query to query the database and retrieve the patient's age and sex. We then pass this patient data to the SHAP explainer to get the SHAP values for the patient, and use shap.plots.waterfall to generate a waterfall plot of the feature contributions.

The exact results will depend on the data in the database and the trained model, but this example demonstrates how we can use the code with an actual database.

It would be great to receive feedback from readers who try out this code and share their results so that others can learn from their experiences. If there is sensitive data involved, it can be kept hidden.

Conclusion

Explainable AI is an important topic that is gaining increasing attention in the AI community. It enables us to understand how AI systems make decisions and helps us identify and correct any biases or errors in the system. In this blog post, we have explored the concept of explainable AI, the various techniques and methods used to achieve explainability, and their practical applications.

I have also demonstrated how to use SHAP, a popular explainability tool, to understand the feature contributions of a machine learning model. By understanding how machine learning models make decisions, we can improve their accuracy, reduce errors, and ultimately build better and more reliable AI systems.

I hope that this blog post has sparked your interest in explainable AI and motivated you to explore the topic further. There are many resources available online to help you learn more about explainable AI, including research papers, tutorials, and open-source libraries. By diving deeper into the topic, you can gain a better understanding of how AI systems work and make more informed decisions when using them.

Thank you for reading! I would love to hear from you and will do my best to respond promptly. Thank you again for your time, and have a great day! If you have any questions or feedback, please let us know in the comments below or email me.

Subscribe, follow and become a fan to get regular updates.

https://www.buymeacoffee.com/sheriffbabu