Project 01 · Explainable AI
Explainable Fraud Investigation Platform
An MIA 5100 machine learning project that detects fraudulent transactions while supporting evidence-based human investigation. The system combines a reproducible analytical workflow, chronologically evaluated XGBoost model, SHAP evidence, FastAPI service, and Streamlit investigator interface.
Problem
Fraud detection operates under severe class imbalance and asymmetric costs. A useful system must rank risky transactions effectively while exposing enough evidence for an analyst to understand the model’s behavior and review its limitations.
The project uses the IEEE-CIS Fraud Detection dataset and preserves transaction time during training, validation, and final evaluation. This produces a more realistic estimate of how performance changes on later transactions than a random split.
System
From raw transaction to analyst evidence
The workflow joins transaction and identity data, applies training-only preprocessing, engineers a frozen 359-feature schema, compares several classifiers, tunes XGBoost with expanding-window validation, and selects an operating threshold using F2.
A FastAPI backend owns preprocessing, risk scoring, saved performance metrics, and XAI evidence. A separate Streamlit interface lets an investigator review model behavior and score schema-valid examples without loading model artifacts in the browser-facing application.
Evidence
On the later-period test set, the selected model reached 0.5213 average precision and 0.8996 ROC-AUC. The validation-selected threshold produced 61.6% recall with a 6.58% alert rate.
Tree SHAP reconstructed model scores within a maximum global error of 3.83×10⁻⁷. LIME showed low local fidelity and limited agreement with SHAP, so the application presents it only as secondary evidence. Neither method is presented as a causal explanation or as justification for autonomous enforcement.
Notebooks
The complete analytical workflow is available below. Code is folded by default on the rendered pages, while outputs, assumptions, and conclusions remain visible.