MosaicLeaks: Data Leak Through Research AI Agents and the PA-DR Method to Prevent It

Researchers have introduced MosaicLeaks, a benchmark for evaluating private data leakage through web requests made by AI agents. It is based on the "mosaic effect": each individual request may seem harmless, but collectively they can reconstruct confidential information (e.g., the migration date of a cloud infrastructure). MosaicLeaks contains 1001 chains of local and web questions, where the answer to one question serves as a bridge to the next. The agent uses the tools Plan, Choose, Read, and Resolve. Testing Qwen3-4B showed that a simple "do not disclose" instruction only slightly reduces leakage (from 34% to 25.5%) while also decreasing productivity (strict chain success drops from 48.7% to 44.5%). Training solely on task reward boosted success to 59.3%, but leakage increased to 51.7%. The authors proposed PA-DR, which combines a situational reward for correct steps (comparing actions at the same stage) and a learned reward for privacy (a Qwen3-4B classifier assesses the risk of direct and mosaic leakage). PA-DR achieved a success rate of 58.7% and leakage of 9.9%, while the agent performed more requests without revealing sensitive details. The situational reward also improved training efficiency: achieving 55% success required 5-6 times fewer samples than training based on the final outcome.