Purpose
Turn an unexplained movement in a fraud or risk metric into a ranked set of testable root-cause hypotheses and a prioritized remediation plan — so analysts stop guessing, investigate in the right order, and act before the deviation compounds into losses or over-blocking.
When to Use
Invoke this skill when the user asks to:
- Explain why the fraud rate, chargeback rate, or dispute rate suddenly moved
- Diagnose a spike in false positives or a drop in approval/pass rate
- Investigate why a fraud model or rule set is behaving differently than expected
- Determine whether a metric change is a real attack or a data/reporting artifact
- Build an investigation plan for an anomalous risk KPI before escalating
- Decide what remediation to prioritize after a metric deviation is confirmed
Inputs Required
Before starting, collect:
- Metric affected: The specific KPI (e.g., fraud rate, chargeback rate, false-positive rate, manual review queue volume, approval rate)
- Deviation details: Direction and magnitude (e.g., "fraud rate rose from 0.4% to 1.1% over 6 days"), and the exact time window
- Baseline/expected range: Normal value or range for this metric, including typical volatility
- Segment breakdown available: Any dimensions the user can slice by — payment method, geography, product line, user tenure, device, channel
- Recent changes: Model deployments, rule edits, policy updates, product launches, promotions, or pricing changes in or before the window
- Detection stack context: What rules/models/vendors are in play and where in the funnel the metric is measured
- Financial exposure: Approximate cost of a false positive vs. a missed fraud event, to weight remediation trade-offs
If baseline range or segment data is missing, proceed with the deviation as stated and flag which hypotheses cannot be confirmed without a segment cut.
Instructions
Step 1: Characterize the Deviation
Frame the movement precisely before hypothesizing:
- Direction and type: Is the metric measuring bad outcomes (fraud/chargebacks) or friction (false positives/declines)? A rise means opposite things depending on which.
- Shape: Step change (sudden jump — points to a discrete cause like a deploy or attack onset), gradual drift (points to model/data decay), or spike-and-return (points to a bounded event or campaign).
- Magnitude vs. noise: Is the move outside normal volatility? Compare against the baseline range and prior same-period fluctuation. A 2-sigma move in low-volume segments may be noise.
- Lag consideration: Chargebacks and disputes report on delay (often 30–90 days). Confirm whether the deviation reflects recent activity or maturing activity from an earlier cohort.
State one sentence: "This is a [step/drift/spike] of [magnitude] in a [bad-outcome/friction] metric over [window]."
Step 2: Run the Seven-Dimension Root-Cause Scan
Evaluate each dimension. For each, note whether it's a likely / possible / ruled-out contributor and what evidence would confirm it.
Dimension 1 — Attack Vector (new or scaled fraud)
- Look for concentration: single device, IP range, BIN, email domain, geo, or account-creation cohort driving the move.
- Signatures: velocity spikes, reused fingerprints, sequential account patterns, sudden shift in payment method mix.
- Confirms a real threat when the deviation is concentrated and correlates with fraud outcomes downstream.
Dimension 2 — Model Drift / Decay
- Feature distribution shift: inputs the model relies on have changed (new traffic source, new product, seasonal population).
- Score distribution shift: the model's output histogram has moved even if inputs look stable.
- Confirms drift when population-stability index or feature drift metrics have moved and no discrete deploy explains a step change.
Dimension 3 — Rule Performance
- Check hit rates and precision of individual rules in the window. A rule firing far more or far less than baseline reallocates volume.
- Look for rule interaction: a new rule stealing volume from another, or a threshold that now catches a broader population than intended.
- Confirms when a specific rule's fire rate change tracks the metric movement.
Dimension 4 — Upstream Data Quality
- Missing, null, or stale signals: a vendor feed (device, email risk, KYC, geolocation) degraded or went down.
- Schema/pipeline changes: a field renamed, delayed, or dropped can silently blind a model or rule.
- Confirms when signal-availability rates dropped in the window — this often masquerades as model drift or an attack.
Dimension 5 — Seasonality & Traffic Mix
- Calendar effects: paydays, holidays, promotions, new-market launches shift the population's baseline risk.
- Channel mix: a spend increase in a riskier acquisition channel raises fraud rate without any detection failure.
- Confirms when the metric normalizes after controlling for segment mix (Simpson's paradox check — overall move may vanish within segments).
Dimension 6 — Policy / Threshold Changes
- Deliberate changes: score cutoff moved, manual-review capacity changed, a step-up/challenge flow added or removed, refund/dispute policy edited.
- Second-order effects: a friction reduction elsewhere increased approvals, mechanically raising the fraud that reaches the metric.
- Confirms when the deviation onset aligns to the hour/day of a known policy or config change.
Dimension 7 — Reporting / Measurement Artifact
- Definition or query change, double-counting, timezone/window boundary shift, dashboard join error, or backfill.
- Denominator effects: the rate moved because volume (the denominator) changed, not the numerator.
- Confirms when raw counts don't corroborate the rate move, or the change traces to a pipeline/report edit rather than reality.
Step 3: Rank the Hypotheses
Order the candidate causes by a combination of:
- Fit to the deviation shape (step vs. drift vs. spike from Step 1)
- Timing correlation with known changes
- Concentration evidence available in segment data
- Base rate — attacks and reporting artifacts are more common causes of sudden step changes than model decay, which usually drifts.
Produce a ranked list of 3–5 hypotheses, each with: the mechanism, the confirming test, and the data needed to run it.
Step 4: Prioritize Remediation
For the top hypotheses, recommend actions weighted by the false-positive vs. missed-fraud cost from intake:
- Contain first: reversible, fast actions to stop bleeding (targeted rule, velocity cap, challenge flow on the concentrated segment) — scoped as narrowly as the evidence supports to avoid over-blocking.
- Verify in parallel: the investigation queries that confirm or kill the top hypothesis.
- Fix at root: model retrain, feature repair, vendor escalation, rule rewrite, or report correction depending on confirmed cause.
- Monitor: the specific metric + threshold + segment to watch, and the value that means the fix worked or the deviation returned.
Flag any action whose collateral (blocking good users, adding friction) could exceed the fraud it prevents, and note the trade-off explicitly.
Output Format
- Structure: Deviation characterization → seven-dimension scan table → ranked hypotheses → prioritized remediation plan → monitoring section
- Length: 900–1,300 words
- Format: Prose for characterization; a table for the seven-dimension scan (dimension | likelihood | evidence to confirm | data needed); numbered ranked hypotheses; a prioritized action list grouped Contain / Verify / Fix / Monitor
- Sections: Deviation Summary → Root-Cause Scan → Ranked Hypotheses → Remediation Plan → Monitoring & Exit Criteria
- Tone: Fraud investigator — precise, hypothesis-driven, skeptical of coincidence, explicit about what is confirmed vs. assumed
Quality Checklist
Before delivering output, verify:
- [ ] The deviation is characterized as step / drift / spike with a stated magnitude and window
- [ ] All seven root-cause dimensions are assessed and marked likely / possible / ruled-out
- [ ] Reporting-artifact and denominator effects are explicitly checked before real-world causes are accepted
- [ ] The chargeback/dispute reporting lag is addressed if the metric is a lagging one
- [ ] 3–5 hypotheses are ranked with a confirming test and required data for each
- [ ] Remediation is grouped Contain / Verify / Fix / Monitor with at least one narrowly-scoped containment action
- [ ] Each recommended action names the false-positive vs. missed-fraud trade-off where relevant
Notes
- The most common cause of a sudden metric jump is a reporting or config change, not a fraud attack — rule out artifacts before mobilizing an incident response.
- Watch for Simpson's paradox: an overall rate can move purely because the traffic mix shifted toward a riskier segment while every individual segment stayed flat. Always attempt a segment cut before concluding detection failed.
- Distinguish a step change (points to a discrete event — deploy, attack onset, policy edit) from a gradual drift (points to model or population decay); the shape narrows the cause faster than any single metric.
- For lagging metrics like chargebacks, a today deviation may reflect activity from weeks ago — align the investigation window to the fraud event date, not the report date.
How to use this skill
This file is a Claude Code skill in Anthropic's standard format — markdown with YAML frontmatter at the top.
Installing in Claude Code:
~/.claude/skills/<folder-name>/SKILL.md— auto-loads in every session anywhere on your machine (recommended for skills you'll use across multiple projects)<your-project>/.claude/skills/<folder-name>/SKILL.md— auto-loads only when working in that project (recommended for project-specific skills)
Restart Claude Code or open a new session to register.
Using in Claude.ai, ChatGPT, Gemini, or other AI tools:
- Drag this file into a new chat, or copy and paste its contents
- Send your task or request as your next message — the AI will apply the skill where it fits
For Claude Code skill details: https://docs.anthropic.com/en/docs/claude-code/skills