The situation
Bad actors on an ads platform are adversarial and fast. Bing Ads relied heavily on hand-written rules that took time to write, were easy to route around and caught abuse hours after it started. Every hour of delay meant losses and a worse experience for users and honest advertisers.
What I owned
The full risk portfolio: payment fraud, phishing, malware, counterfeit goods and editorial abuse.
Key decision: rules to ML
Instead of adding more rules, we moved detection to machine learning models that learn from labelled abuse and adapt as patterns shift, with rules kept as a safety net where precision mattered most. The product work was as much about the system around the models as the models themselves: labelling, review workflows and the tolerance for false positives that could block legitimate advertisers.
Outcome
- Fraud loss down 62% year over year, about $20M.
- Time to detect abuse down from 4 hours to 5 minutes, using unsupervised models that adapted as fraudsters changed tactics.
- User escalations from phishing and malware down 95%, to 0.6%, within 6 months.
What I took from it
This was my first experience turning a manual, expert-driven process into an ML system. I’ve applied the same pattern since: replacing sampled, human review with models that cover everything, which is exactly what ProdLoop.ai does for calls.