Coinbase Prime has successfully deployed an artificial intelligence-powered fraud detection agent for its Onramp service, reducing fraudulent transactions by 30% and fraud value by 22% in live testing. The system, which layers a large language model agent onto existing machine learning models and rule engines, marks a significant advancement in detecting sophisticated fraud patterns that span multiple transactions rather than isolated purchases.
- Fraud reduction: 30% fewer fraudulent transactions, 22% less fraud value
- Model performance gain: 9.6% higher F1 score than frontier models
- Latency improvement: 55% lower end-to-end serving latency
- Median LLM call latency: Approximately 1.5 seconds with 7,000-token input
The fraud agent addresses a critical gap in traditional fraud detection: while single-transaction screening catches obvious threats, sophisticated attackers exploit patterns across multiple purchases that appear routine in isolation. Coinbase‘s solution uses Claude Opus to review recent transaction sequences in context, comparing current activity against historical summaries without engineering new features for each emerging fraud pattern. The system operates downstream from existing protections, adding an additional layer of scrutiny for selected transactions while maintaining low cost through selective invocation.
The architecture separates model inference from business logic, with application code preparing transaction context and parsing risk assessments while the LLM returns only a structured risk classification. This bounded approach prevents the model from executing payment decisions and keeps the agent’s authority explicit within the broader risk stack. The serving layer orchestrates the ML model, rule engine, and LLM agent using Ray Serve, with asynchronous batch processing and configurable fallback models for provider capacity errors. Projected savings from fraud prevention were roughly three times the estimated inference cost, calculated using published rates for input and output tokens.
Coinbase plans to publish comparative evaluations of alternative models including Opus, Sonnet, and GPT releases on proprietary fraud benchmarks, then move to post-training a specialized model on its Onramp fraud dataset. The company’s multi-part research series examines not only how the agent was built but also how different model versions perform on domain-specific fraud metrics, an evaluation gap that prompted the development of custom benchmarks rather than relying on general-purpose model comparisons.
