Explainable Fraud Detection and Investigation Platform MIA 5100 machine-learning model with an MIA 5126 XAI extension
This notebook applies the post-hoc explanation methods introduced in MIA 5126 — Essential Concepts in Data Science to the frozen fraud model developed in the first three project notebooks. It adds global and local transparency without retraining the model or changing its operating threshold.
Deliverables
Global SHAP bar and beeswarm plots
SHAP dependence scatter plots
Local SHAP waterfall explanations for representative TP, FP, FN, and TN cases
LIME explanations for the same cases
SHAP/LIME comparison with local-fidelity diagnostics
Machine-readable XAI artifacts for the FastAPI and Streamlit applications
Explicit explanation limitations and human-review boundaries
Experimental discipline: The champion model, preprocessing schema, and threshold are already frozen. SHAP and LIME are used only for post-hoc reporting. No explanation in this notebook may be used to select features, retrain a model, alter hyperparameters, or change the threshold using final-test information.
Interpretation boundary: A contribution describes how the frozen model behaves for the supplied processed features. It is not evidence that a feature caused fraud, and it does not prove that a transaction is fraudulent or legitimate.
1. Load the frozen model and processed reporting data
The notebook loads the saved champion manifest rather than selecting a model again. The processed test matrix is used only for final post-hoc reporting, while a fixed sample of training rows supplies the SHAP background and LIME reference distribution.
Frozen champion: XGBoost (tuned) Decision threshold: 0.275698 Processed features: 359 Test rows available for reporting: 88,581
Code
id_to_position = pd.Series( np.arange(len(id_test), dtype="int64"), index=id_test.to_numpy())selected_case_ids = selected_cases["TransactionID"].astype("int64").tolist()local_positions = [int(id_to_position.loc[transaction_id]) for transaction_id in selected_case_ids]local_features = X_test.iloc[local_positions].reset_index(drop=True)saved_scores = selected_cases.set_index("TransactionID").loc[ selected_case_ids, "fraud_risk_score"].to_numpy()recomputed_scores = model.predict_proba(local_features)[:, 1]np.testing.assert_allclose(recomputed_scores, saved_scores, rtol=1e-5, atol=1e-6)case_context = selected_cases.set_index("TransactionID").loc[ selected_case_ids, ["Outcome", "fraud_risk_score", "predicted_fraud", "isFraud"],].reset_index()display(case_context.style.format({"fraud_risk_score": "{:.4f}",}))print("Representative case scores match the frozen predictions from 03_model_selection_and_evaluation.ipynb.")
TransactionID
Outcome
fraud_risk_score
predicted_fraud
isFraud
0
3519397
True positive
0.8880
1
1
1
3524909
False positive
0.4169
1
0
2
3551357
False negative
0.0993
0
1
3
3541077
True negative
0.0192
0
0
Representative case scores match the frozen predictions from 03_model_selection_and_evaluation.ipynb.
2. Fixed explanation samples
SHAP background: 100 training rows, consistent with the documented 100–1,000 row interventional background guidance.
Global SHAP reporting sample: 1,000 randomly selected test rows. This is a reproducible approximation of global behavior, not a new evaluation sample.
Local cases: One median-risk test case from each confusion-matrix outcome (true positive, false positive, false negative, and true negative), selected deterministically from 03_model_selection_and_evaluation.ipynb predictions.
LIME reference: 10,000 training rows to estimate perturbation distributions without densifying all 413,378 training observations.
SHAP background rows: 100
Global SHAP reporting rows: 1,000
Global sample fraud rate: 3.70%
Local representative cases: 4
3. Tree SHAP in model-score space
TreeExplainer is model-specific and efficient for the XGBoost champion. The explainer uses an interventional training background and model_output="probability", so the base value plus feature contributions reconstructs the model’s predict_proba class-1 output.
The output is still described as an uncalibrated fraud risk score, consistent with 03_model_selection_and_evaluation.ipynb. It must not be interpreted as an empirically calibrated probability of fraud.
SHAP computation time: 16.44 seconds
Maximum global reconstruction error: 3.82e-07
Maximum local reconstruction error: 1.53e-07
SHAP base risk score: 0.1504
Native gain measures training-time split improvement. Mean absolute SHAP measures average prediction impact on the fixed reporting sample. Agreement is useful context, but the rankings are not expected to be identical because they answer different questions.
Each plot relates a processed feature value to that feature’s SHAP contribution across the fixed reporting sample. The vertical axis is the change in the model’s uncalibrated risk score relative to the background expectation. Patterns are associative descriptions of model behavior, not causal effects.
LIME is model-agnostic. It perturbs a processed transaction and fits a sparse local linear surrogate around that case. The displayed weights describe the surrogate, not XGBoost directly.
The local surrogate’s weighted R² (LIME local fidelity R2) is reported for every case. Low fidelity means the linear approximation does not closely reproduce the nonlinear champion in that neighborhood; its feature weights should then receive limited evidentiary weight.
comparison_records = []comparison_top_k =10for transaction_id in selected_case_ids: shap_case = local_shap_table.query("TransactionID == @transaction_id and Rank <= @comparison_top_k" ) lime_case = local_lime_table.query("TransactionID == @transaction_id and Rank <= @comparison_top_k" ) shap_features =set(shap_case["Feature"]) lime_features =set(lime_case["Feature"]) overlap = shap_features & lime_features union = shap_features | lime_features shap_sign = shap_case.set_index("Feature")["SHAP contribution"].map(np.sign) lime_sign = lime_case.set_index("Feature")["LIME weight"].map(np.sign) sign_agreements = [ shap_sign.loc[feature] == lime_sign.loc[feature]for feature in overlap ] lime_summary = lime_case_summary.loc[ lime_case_summary["TransactionID"] == transaction_id ].iloc[0] comparison_records.append({"TransactionID": int(transaction_id),"Outcome": case_context.loc[ case_context["TransactionID"] == transaction_id, "Outcome" ].iloc[0],"Top-k": comparison_top_k,"Overlapping features": len(overlap),"Top-feature Jaccard": len(overlap) /len(union) if union else np.nan,"Direction agreement on overlap": (float(np.mean(sign_agreements)) if sign_agreements else np.nan ),"LIME local fidelity R2": float( lime_summary["LIME local fidelity R2"] ),"LIME absolute prediction error": float( lime_summary["Absolute local prediction error"] ), })explanation_comparison = pd.DataFrame(comparison_records)display(explanation_comparison.style.format({"Top-feature Jaccard": "{:.1%}","Direction agreement on overlap": "{:.1%}","LIME local fidelity R2": "{:.3f}","LIME absolute prediction error": "{:.4f}",}))
TransactionID
Outcome
Top-k
Overlapping features
Top-feature Jaccard
Direction agreement on overlap
LIME local fidelity R2
LIME absolute prediction error
0
3519397
True positive
10
2
11.1%
100.0%
0.242
0.4919
1
3524909
False positive
10
2
11.1%
100.0%
0.105
0.4005
2
3551357
False negative
10
2
11.1%
100.0%
0.224
0.0052
3
3541077
True negative
10
3
17.6%
100.0%
0.182
0.0306
8. Explanation limitations
No causality: SHAP and LIME describe model behavior under their respective assumptions; neither proves why fraud occurred.
Processed feature space: Values reflect imputation, scaling, frequency encoding, and one-hot encoding. They are not always directly readable as raw transaction values.
Feature dependence: Correlated variables can share or redistribute attribution. Interventional SHAP depends on the selected training background.
Sampling approximation: Global SHAP results summarize a fixed 1,000-row reporting sample rather than every test transaction.
LIME instability and fidelity: LIME depends on random perturbations, discretization, kernel width, sample count, and local linearity. Low R² means its local surrogate is a weak approximation.
Test reporting only: Observations from this notebook must not be used to improve reported test performance or change the frozen operating point.
Human decision boundary: Explanations support review; they do not authorize blocking, accusing, approving, or rejecting a transaction.
9. Export XAI artifacts
Code
global_importance.to_csv( XAI_RESULTS_PATH /"shap_global_importance.csv", index=False)local_shap_table.to_parquet( XAI_RESULTS_PATH /"local_shap_values.parquet", index=False)local_lime_table.to_parquet( XAI_RESULTS_PATH /"local_lime_values.parquet", index=False)lime_case_summary.to_csv( XAI_RESULTS_PATH /"lime_case_fidelity.csv", index=False)explanation_comparison.to_csv( XAI_RESULTS_PATH /"shap_lime_comparison.csv", index=False)sample_manifest.to_csv( XAI_RESULTS_PATH /"shap_reporting_sample.csv", index=False)metadata = {"model_name": manifest["model_name"],"model_file": manifest["model_file"],"decision_threshold": threshold,"score_semantics": model_metadata["score_semantics"],"feature_count": len(feature_names),"random_state": RANDOM_STATE,"test_reporting_only": True,"model_or_threshold_changed": False,"shap": {"version": shap.__version__,"explainer": "TreeExplainer","feature_perturbation": "interventional","model_output": "probability","output_description": "uncalibrated fraud risk-score units","background_source": "fixed random training sample","background_rows": SHAP_BACKGROUND_ROWS,"global_reporting_source": "fixed random test sample","global_reporting_rows": SHAP_GLOBAL_ROWS,"base_risk_score": float(np.asarray(global_shap.base_values)[0]),"maximum_global_reconstruction_error": global_reconstruction_error,"maximum_local_reconstruction_error": local_reconstruction_error,"native_gain_spearman_rank_correlation": spearman_rank_correlation,"top_global_feature": global_importance.iloc[0]["Feature"], },"lime": {"version": version("lime"),"reference_source": "fixed random training sample","reference_rows": len(lime_reference),"perturbations_per_case": LIME_NUM_SAMPLES,"features_per_case": LOCAL_TOP_FEATURES,"mean_local_fidelity_r2": float( lime_case_summary["LIME local fidelity R2"].mean() ),"minimum_local_fidelity_r2": float( lime_case_summary["LIME local fidelity R2"].min() ),"warning": ("LIME approximates the XGBoost score for one transaction. When ""local fidelity R² is low, its feature weights may not accurately ""represent the model and should be treated as secondary evidence." ), },"representative_transaction_ids": selected_case_ids,"figures": {"global_bar": global_bar_file.name,"global_beeswarm": global_beeswarm_file.name,"native_comparison": importance_comparison_file.name,"dependence": dependence_figure_files,"waterfall": shap_waterfall_files,"lime": lime_figure_files, },"required_boundary": ("Post-hoc model-behavior evidence only; not causal proof or an ""autonomous fraud decision." ),}(XAI_RESULTS_PATH /"xai_metadata.json").write_text( json.dumps(metadata, indent=2), encoding="utf-8")print("Saved XAI tables, metadata, and figures.")print(f"Output directory: {XAI_RESULTS_PATH}")
Saved XAI tables, metadata, and figures.
Output directory: /Users/hshazel/Projects/explainable-fraud-investigation-platform/results/xai
Code
def markdown_table(frame, columns, formats=None): formats = formats or {} lines = ["| "+" | ".join(columns) +" |","|"+"|".join(["---"] *len(columns)) +"|", ]for _, row in frame[columns].iterrows(): values = [formats.get(column, "{}").format(row[column]) for column in columns] lines.append("| "+" | ".join(values) +" |")return"\n".join(lines)top_global_table = markdown_table( global_importance.head(10), ["SHAP rank", "Feature", "Mean absolute SHAP", "Normalized Gain"], {"SHAP rank": "{:.0f}","Mean absolute SHAP": "{:.5f}","Normalized Gain": "{:.2%}", },)local_comparison_table = markdown_table( explanation_comparison, ["TransactionID", "Outcome", "Overlapping features","Top-feature Jaccard", "LIME local fidelity R2", ], {"TransactionID": "{:.0f}","Overlapping features": "{:.0f}","Top-feature Jaccard": "{:.1%}","LIME local fidelity R2": "{:.3f}", },)mean_lime_fidelity =float(lime_case_summary["LIME local fidelity R2"].mean())median_overlap =float(explanation_comparison["Top-feature Jaccard"].median())top_feature = global_importance.iloc[0]["Feature"]report_text =f'''# Post-hoc XAI: SHAP & LIME Report## ScopeThis report applies SHAP and LIME to the frozen tuned XGBoost fraud model. It does not retrain the model, change the 0.2757 validation-selected threshold, or reuse explanations for model selection.## Global SHAP findings- Fixed training background: {SHAP_BACKGROUND_ROWS:,} rows- Fixed test reporting sample: {SHAP_GLOBAL_ROWS:,} rows- SHAP output: uncalibrated fraud risk-score units- Top global feature by mean absolute SHAP: `{top_feature}`- SHAP/native-gain Spearman rank correlation: {spearman_rank_correlation:.3f}- Maximum SHAP reconstruction error: {max(global_reconstruction_error, local_reconstruction_error):.2e}{top_global_table}## Local SHAP and LIME comparison{local_comparison_table}Mean LIME local fidelity R² was {mean_lime_fidelity:.3f}; median top-10 SHAP/LIME feature Jaccard overlap was {median_overlap:.1%}. LIME weights must be interpreted in light of case-level fidelity rather than treated as direct XGBoost contributions.## Required interpretation boundarySHAP and LIME describe model behavior under different assumptions. Neither method establishes causality or proves fraud. Anonymous and processed features limit semantic interpretation. A human investigator must verify all supporting evidence and follow approved policy.'''report_text = re.sub(r"^ ", "", report_text, flags=re.MULTILINE).strip() +"\n"report_file = REPORTS_PATH /"04_post_hoc_xai_shap_lime_report.md"report_file.write_text(report_text, encoding="utf-8")print(f"Report saved: {report_file}")
mean_lime_fidelity =float(lime_case_summary["LIME local fidelity R2"].mean())minimum_lime_fidelity =float(lime_case_summary["LIME local fidelity R2"].min())median_overlap =float(explanation_comparison["Top-feature Jaccard"].median())display(Markdown(f'''## Final XAI conclusions1. **Global behavior:** `{global_importance.iloc[0]['Feature']}` had the largest mean absolute SHAP contribution on the fixed 1,000-row reporting sample. The SHAP/native-gain rank correlation was **{spearman_rank_correlation:.3f}**, confirming that prediction impact and training split gain are related but distinct global summaries.2. **Local additivity:** SHAP reconstructed the frozen model scores with a maximum absolute error of **{max(global_reconstruction_error, local_reconstruction_error):.2e}**.3. **Method comparison:** Median top-10 SHAP/LIME feature overlap was **{median_overlap:.1%}**. Mean LIME local fidelity R² was **{mean_lime_fidelity:.3f}** and the minimum was **{minimum_lime_fidelity:.3f}**; LIME should therefore be treated as a secondary local approximation with case-specific reliability.4. **No performance claim:** These explanations did not change the model, features, threshold, or previously reported test metrics.5. **Human-review boundary:** Contributions describe model behavior, not fraud causation or proof. They support investigation but do not replace verified evidence or human decisions.'''))
Final XAI conclusions
Global behavior:TransactionDT had the largest mean absolute SHAP contribution on the fixed 1,000-row reporting sample. The SHAP/native-gain rank correlation was 0.621, confirming that prediction impact and training split gain are related but distinct global summaries.
Local additivity: SHAP reconstructed the frozen model scores with a maximum absolute error of 3.82e-07.
Method comparison: Median top-10 SHAP/LIME feature overlap was 11.1%. Mean LIME local fidelity R² was 0.188 and the minimum was 0.105; LIME should therefore be treated as a secondary local approximation with case-specific reliability.
No performance claim: These explanations did not change the model, features, threshold, or previously reported test metrics.
Human-review boundary: Contributions describe model behavior, not fraud causation or proof. They support investigation but do not replace verified evidence or human decisions.
Method references
MIA 5126 Week 10 course material: explainability versus interpretability, global versus local explanations, Tree SHAP, beeswarm, waterfall, dependence plots, and LIME
