AI supply chain data leaks expose user secrets across inference providers
Researchers tracking nearly 2 million planted digital secrets across AI routing and hosting services find that sensitive prompt data does not remain confined to the providers users select. The results point to breaches at multiple providers, reseller logging practices, and failures to honor privacy settings such as Zero Data Retention.
Highlights
- Researcher HD Moore's canary test found at least three AI providers compromised, with attackers siphoning prompts and using stolen AWS keys tens of thousands of times in API calls.
- Some AI providers and model resellers ignored Zero Data Retention flags, with a Chinese AI lab's systems fetching secret URLs from over half of prompts, including those sent via U.S. hosts.
- Stolen credentials were tested for S3, secrets managers, and IAM persistence, with activity from two breached providers traced to a single rented server, implicating a shared adversary.
Canary test maps prompt leakage
As reported by Ars Technica, security researcher HD Moore seeded unique AWS keys, passwords, private repository references, and trackable URLs into individual prompts sent through AI routers and inference providers to identify where secrets resurfaced. Moore, chief research officer at Dragos, says the test reveals security breaches, prompt theft, and repeated failures to protect customer data across the AI inference marketplace.Many AI applications rely on intermediaries such as OpenRouter and Vercel to route prompts to models based on price or task suitability rather than sending requests directly to major providers. Because prompts are decrypted at each step, every handoff creates another point where data can be logged, stored, or intercepted.
The experiment found evidence that at least three AI providers were compromised by attackers who siphoned prompts, while multiple resellers logged requests and passed along credentials that were later used. Moore says some stolen AWS keys were taken from requests rejected before any model ran, indicating attackers were copying traffic at a provider's front end rather than deeper in the inference stack.
Those bait credentials were then tested, in some cases within hours, to inspect S3 buckets, secrets managers, and parameter stores, while attackers also attempted to enable AWS Bedrock models and add IAM users for persistence. In total, the stolen keys were used tens of thousands of times in AWS API calls, and activity linked to two breached providers appeared to come from the same rented server, suggesting a shared adversary.
Privacy controls face wider industry risk
Moore also finds that some providers and model sellers fail to respect privacy instructions embedded in prompts, including Zero Data Retention and no-data-collection flags. In one example, a Chinese AI lab's systems fetched secret URLs from more than half the prompts sent to its models between July and October, including roughly two-thirds of those marked with stricter privacy settings.He says the behavior was not limited to direct use of that lab's models. Several U.S. hosting providers that resell those models triggered the same scraper, indicating prompts sent through domestic providers still flowed back to the original lab, undermining customer expectations that data would stay in the U.S.
The research also identifies other delayed scraping activity, including one U.S. hosting provider crawling a secret URL more than two months after a single inference request and another revisiting URLs days after each request regardless of which router delivered the prompt. Moore says the strongest protection is to host models on-premises, while customers using external services should work directly with first-tier providers and avoid default reseller pools where possible.
The findings add to broader concerns over operational security in the AI sector, where complex chains of brokers, hosts, and model providers can widen the number of entities with access to plain-text prompts. For businesses handling credentials, internal code, or regulated information, the results suggest that routing convenience and lower pricing can carry significant data exposure risks.