XAI Basics

XAI Basics
• Huge Data → Insights – Prediction – AI
• Find Laws
• AI identifies mathematical patterns found in thousands of variables and the

relations among those variables.
• These patterns can be so complex that they can defy human understanding.
• AI – Black Box
Black box
• Unable to answer crucial questions about the operations of the
“machine”:
• Is it making reliable predictions? Is it making those
• predictions on solid or justified grounds?

• Will we know how to fix it if it breaks?
• Or more generally: can we trust a tool whose operations we don’t
understand, particularly when the stakes are high?
Intro
• Artificial intelligence (AI) helps us solve real-world problems by processing data.
We turn real-world information into numbers for AI models to understand and
improve.
• But a big question? How do we make sense of AI results in the real world? This is
where Explainable AI comes in.
• It’s all about making AI’s decisions easy to understand and apply in real-life
situations.
• Explainable AI helps bridge the gap between AI models and practical use.
What is Explainable AI or XAI?
• Explainable AI (XAI) refers to methods and techniques in the
application of artificial intelligence technology such that the results of
the solution can be understood by humans.
• In the early phases of AI adoption, it was okay to not understand what
the model predicts in a certain way, as long as it gives the correct
outputs.
• Explaining how they work was not the first priority.
• Now, the focus is turning to build human interpretable models.
What it is
• Explainable AI is used to describe an AI model, its expected impact
and potential biases.
• It helps characterize model accuracy, fairness, transparency and
outcomes in AI-powered decision making.
• Explainable AI is crucial for an organization in building trust and
confidence when putting AI models into production.
Example
• For example, hospitals can use explainable AI for cancer detection
and treatment, where algorithms show the reasoning behind a given
model's decision-making.
• This makes it easier not only for doctors to make treatment decisions,
but also provide data-backed explanations to their patients.
What Makes an Explanation Good?
• Different stakeholders
• a good explanation depends on which stakeholders it’s aimed at.
• Different audiences often require different explanations
• Examples
• Tailoring an explanation to the audience: tradeoffs between
accuracy and explainability.
When We Need Explainability
• Not Free
• Takes time and resources
• Examples:
• Image recognition AI may be used to help clients tag photos of their dogs
when they upload their photos to the cloud. Accuracy may matter a great deal,
but exactly how the model does it may not matter so much.
• AI that predicts when the shipment of screws will arrive at the toy factory;
there may be no great need for explainability there.
• More generally, a good rule of thumb is that explainability is probably not a

need-to-have when low risk predictions are made about entities that aren’t
people.
Corollary
• Explainability may matter a great deal, especially when the outputs
directly bear on how people are treated.
• There are at least four kinds of cases to consider in this regard.
1. When regulatory compliance calls for it.
2. When explainability is important so that end users can see how best touse the tool.
3. When explainability could improve the system.
4. When explainability can help assess fairness.
Why Does Explainable AI Matter?
• It matters because it helps us understand how AI systems work and make
decisions. T
• Trust: When we trust an AI system, we are more likely to use it and make
decisions based on its output. XAI helps us build trust in AI systems by
explaining their decisions.
• Transparency: XAI helps us understand AI systems’ potential biases and

limitations. They can significantly impact our lives, and we need to know the
potential risks and benefits before using them.
• Accountability: It can help us hold AI systems accountable for their

decisions. They are often used in high-stakes applications like law
enforcement and healthcare.
How to Interpret Model with XAI?
• Model interpretability can be examined in two levels:
• Global Interpretation: Examines the model from a broader perspective. For
example, let’s say we are working on a house price dataset and we implemented a
neural network. The global interpretation might say “Your model uses # of
squared feet as an important feature to derive predictions
• Local Interpretation: As the name suggests, this approach is focused on a

certain observation/data point. Let’s continue moving forward with our example.
Prediction for a really small house turned out large. Local interpretation looks at
the other features and it might say “Your model predicted this way because the
location of the house is very close to the city center.
The Trade-off Between Accuracy and Interpretability
• In the industry, you will often hear that business stakeholders tend to prefer
models that are more interpretable like linear models (linear\logistic
regression) and trees which are intuitive, easy to validate, and explain to a non-
expert in data science.
• In contrast, when we look at the complex structure of real-life data, in the model
building & selection phase, the interest is mostly shifted towards more advanced
models. That way, we are more likely to obtain improved predictions.
Challenge of Complex Models
• Models like these (ensembles, neural networks, etc.) are

called black-box models. As the model gets more advanced,
it becomes harder to explain how it works. Inputs magically
go into a box!
• We get amazing results.
• But, HOW?
Suggesting Model
• When we suggest this model to stakeholders, will they completely
trust it and immediately start using it? NO.
• They will ask questions and we should be ready to answer them.
• Why should I trust your model?
• Why did the model take a certain decision?
• What drives model predictions?

• We should consider both improving our model accuracy and not get
lost in the explanation. There should be a balance between both.
Said
• Any machine learning model at its heart has a response
function which tries to map and explain relationships and
patterns between the independent (input) variables and the
dependent (target or response) variable(s).
Role of Data in Model Trustworthiness
• So, models take inputs and process them to get outputs. What if our
data is biased?
• It will also make our model biased and therefore untrustworthy.
• It is important to understand & be able to explain to our models so that
we can also trust their predictions and maybe even detect issues and
fix them before presenting them to others.
Techniques for Model Interpretability
• To improve the interpretability of our models, there are various

techniques some of which we already know and implement.
• Traditional techniques are exploratory data analysis, visualizations,
and model evaluation metrics.
• With the help of them, we can get an idea of the model’s strategy.
• However, they have some limitations.
• To learn more about traditional ways and their limitations.
Modern Techniques for Model Interpretability
• Other model interpretation techniques and libraries have been

developed to overcome limitations.
• Some of these are :
• LIME ( Local Interpretable Model-Agnostic Explanations)
• SHAP (Shapley Additive Explanations)
• ELI5 (Explain Like I’m 5)
• SKATER
• These libraries use
• feature importance,
• partial dependence plots,
• individual conditional expectation plots
to explain less complex models such as linear regression, logistic

regression, decision trees, etc.
Feature Importance
• Feature importance shows how a feature is important for the model.
• In other words, when we delete the feature from the model, how our error
changes?
• If the error increases a lot, this means that a feature is important for our
model to predict the target variable
Partial dependence plots
• Partial dependence plots visualize the effect of the change for a certain feature
when everything else is held constant (with a cooler phrase: ceteris paribus).
• With the help of these, we can see a possible limit value, where this value is
exceeded, it directs the model predictions the other way.
• When we are visualizing partial dependence plots, we are examining the model
globally.
What is model-specific or model agnostic?
• Model-specific interpretation tools are specific to a single model or
group of models. These tools depend heavily on the working and
capabilities of a specific model.
• In contrast, model-agnostic tools can be used on any machine
learning model, no matter how complicated.
What is an agnostic approach?
• In a computer-based environment, an agnostic approach is the one
which is interoperable across the systems and there are no prejudices
towards using a specific technology, model, methodology or data.
• An agnostic approach is not only towards the aforesaid factors; but
also, towards the business processes and practices.
Model-Agnostic Techniques
• Model-agnostic methods are an approach to understand the

predictive response of a black box model, instead of the response
from the original dataset.
• Model agnostic evaluation methods provide flexibility when it comes

to model selection. You do not have to develop a different evaluation
framework for each model type. This also allows you to compare
many models using the same metrics. This consistency is essential
when you want to compare model performance
Individual conditional expectation plots
• Individual conditional expectation plots show the effect of changes for a

certain feature, just like partial dependency plots.
• But this time, the point of view is local.
• We are interested to see the effect of changes for a certain feature for all
instances in our data.
• A partial dependence plot is the average of the lines of an ICE plot.
Model Agnostic ….
• When it comes to explaining more advanced models, model-agnostic (does not
depend on the model) techniques are used.
• Global surrogate models take the original inputs and your black-box machine
learning predictions. When this new dataset is used to train and test the
appropriate global surrogate model (more interpretable models such as linear
model, decision tree, etc.), it basically tries to mimic your black-box model’s
predictions. By interpreting and visualizing this “easier” model, we get a better
understanding of how our actual model predicts in a certain way.
• Other interpretability tools are LIME, SHAP, ELI5, and SKATER libraries. We
will talk about them in the next post, over a guided implementation.
Building an Explainability Framework
• To create a framework that addresses the risks of black boxes to

their industry and their organizations
• In particular, enabling them to properly prioritize explainability in
each of their AI projects.
• That framework would not only enable data scientists to build
models that work well, but also empower executives to make wise
decisions about what should be designed and when systems are
sufficiently trustworthy to deploy.
Conclusion.
• Integrating Explainable AI techniques equips us with the means to provide
convincing explanations for AI model decisions.
• By doing so, we foster trust, ensure accountability, and enable better

decision-making.
• As we journey into the age of AI, embracing transparency through

Explainable AI is not just a choice; it’s a necessity.
• It empowers us to harness the full potential of AI, making its inner workings
accessible to all.

XAI Basics

Uploaded by

Copyright:

Available Formats

XAI Basics

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

XAI Basics

Uploaded by

Copyright:

Available Formats

XAI Basics

• Huge Data → Insights – Prediction – AI

• AI identifies mathematical patterns found in thousands of variables and the

• predictions on solid or justified grounds?

• More generally, a good rule of thumb is that explainability is probably not a

• Transparency: XAI helps us understand AI systems’ potential biases and

• Accountability: It can help us hold AI systems accountable for their

• Local Interpretation: As the name suggests, this approach is focused on a

• Models like these (ensembles, neural networks, etc.) are

• We get amazing results.

• Why did the model take a certain decision?

• What drives model predictions?

• To improve the interpretability of our models, there are various

• Other model interpretation techniques and libraries have been

• SHAP (Shapley Additive Explanations)

• ELI5 (Explain Like I’m 5)

to explain less complex models such as linear regression, logistic

• Model-agnostic methods are an approach to understand the

• Model agnostic evaluation methods provide flexibility when it comes

• Individual conditional expectation plots show the effect of changes for a

• To create a framework that addresses the risks of black boxes to

• By doing so, we foster trust, ensure accountability, and enable better

• As we journey into the age of AI, embracing transparency through

You might also like