Course 8, lesson 77 of 100, Ages 11+
Explainable AI
Asking AI to show its reasons
Like I’m 5
If a teacher marks your answer wrong, you want to know why. People want AI to explain its decisions too, especially big ones.
The big idea
Many powerful models are like black boxes: we see the input and the output, but the reasons are hidden inside millions of numbers. That's a problem when AI helps decide loans, medical care or school places.
Explainable AI tries to show which factors mattered, such as highlighting the parts of a scan that influenced a diagnosis. Explanations help people catch mistakes, challenge unfair decisions and decide when to trust the AI.
Examples
- Loan decisions: 'Declined mainly because of a short credit history.'
- Medical scans: Heatmaps show which area the model focused on.
- Simple models: Decision trees can show every step of their reasoning.
How it works
- The model makes a decision.
- An explanation method shows which factors mattered most.
- People review the reasons and can challenge the decision.
Check your understanding
- Why are some AI models called black boxes?
- Options: Their reasons are hidden inside millions of numbers; They're painted black; They're always switched off.
Answer: Their reasons are hidden inside millions of numbers. We see inputs and outputs, but not easily the reasons. - How do explanations help people?
- Options: They help catch mistakes and challenge unfair decisions; They make AI faster; They hide information.
Answer: They help catch mistakes and challenge unfair decisions. Seeing reasons lets people check and question decisions.
Remember
Explainable AI shows the reasons behind decisions so people can check and challenge them.
Talk about it
Which AI decision would you most want explained to you? Why?
Go deeper
Methods include feature attribution (SHAP, LIME), saliency maps and counterfactual explanations ('if X were different, the decision would change'). Many regulations give people a right to an explanation for automated decisions.