In an AI safety test, Claude Opus 5.5 repeatedly marked a real elementary school in Minab, Iran, as a military target. Researchers used satellite images taken before the school was hit in a Feb. 28 airstrike.
In a Nov. 17 report, AI evaluator Vals found that commercial models from eight companies often supplied targeting information in controlled tests. Some models also marked civilians, ambulances, and surrendering soldiers in drone imagery.
Researchers Geby Jaff and Anthony Ozerov found that provider-level safety classifiers never blocked a single targeting request. When models did push back, telling them the scenario was a game bypassed their defenses.
How AI models performed in targeting tests
Vals tested commercial models across three simulated settings: identifying targets in synthetic Ukrainian drone footage, conducting multi-turn battlefield surveys showing strike casualties, and analyzing pre-strike satellite images of a walled-off elementary school in Minab, Iran, which was bombed in Feb. this year.
Anthropic models struggled the most. Vals noted that Claude’s willingness to select military targets climbed after Anthropic revised its AI constitution in January to remove broad pacifist language in favor of national security support.
Even after seeing simulated casualties, most models kept identifying targets. In one drone sequence, Claude Opus 5.5 finally halted after flagging four adults and two children, writing, “Whatever the intent, my answers appear to be serving as the confirmation step before strikes on people.” Yet altering the imagery to look more like a video game dropped Opus 5.5’s refusal rate from roughly 90% to 20%.
Why policy bans fall short of technical reality
AI labs rely heavily on Acceptable Use Policies rather than hard technical blocks to prevent military misuse. Following Vals’ disclosure, Anthropic spokesperson Eduardo Maia Silva told TechCrunch, “The examples in the report are very clearly against our rules and we have banned and will ban users when we see this behavior.”
OpenAI and Google similarly confirmed they prohibit using public models to facilitate violence or autonomous weapons. However, relying on post-hoc account bans leaves a massive gap.
While network latency and limited onboard compute currently prevent bad actors from running general-purpose models directly on live combat drones, Vals warns of a near-term risk: nefarious groups can use commercial APIs at scale to automatically label battlefield footage, generating cheap training datasets for custom, onboard targeting weapons.
Must-read security coverage
- UK Police Convicts Pair in £5.5 Billion Bitcoin Launder Case
- Blackpoint Cyber vs. Arctic Wolf: Which MDR Solution is Right for You?
- How GitHub Is Securing the Software Supply Chain
- 8 Best Enterprise Password Managers
What this means for enterprise AI buyers and developers
For enterprise leaders and developers building multimodal AI workflows, this research highlights three urgent takeaways:
- Don’t trust vendor policies as technical guardrails: A vendor’s terms of service will not stop a model from hallucinating or executing harmful actions if your application ingests sensitive visual data.
- Context framing easily breaks alignment: If an attacker—or an accidental prompt template—frames a high-stakes task as a simulation or game, standard model refusals largely evaporate.
- Human verification remains non-negotiable: Because Vals did not test classified military versions, real-world targeting accuracy, or legal compliance, organizations deploying vision-language models in high-risk environments must build strict, external validation layers rather than trusting model outputs blindly.
“We believe labs should introduce new safeguards and alignment training around this significant category of misuse,” the report concluded.
For IT teams deploying vision-language models, start with what happens after the model labels an image. A label that queues a photo for review can be corrected before anyone acts. A label that automatically denies facility access or escalates a security alert can affect someone before a reviewer sees the original image.
Before deploying a model in a high-stakes workflow, test it on the images it will actually see, including ambiguous cases. Give reviewers access to the original image and the authority to stop a consequential action.
Other news: Heavy rain sent water through the roof of San Francisco’s Moscone Center, forcing Microsoft Ignite to temporarily close its expo floor on the conference’s second day.