AI Safety Test Exposes a Guardrail Gap: Models Flagged Civilians as Military Targets

Vals found that commercial AI models flagged civilians and a school in targeting tests, raising questions for IT teams that rely on AI to interpret images.

Verfasst von
Aminu Abdullahi
Aminu Abdullahi
Nov 20, 2026
AI Safety Test Exposes a Guardrail Gap: Models Flagged Civilians as Military Targets

AI safety tests reveal a dangerous loophole in military target recognition. Image: Generated with Google’s Nano Banana 2.

In an AI safety test, Claude Opus 5.5 repeatedly marked a real elementary school in Minab, Iran, as a military target. Researchers used satellite images taken before the school was hit in a Feb. 28 airstrike.

In a Nov. 17 report, AI evaluator Vals found that commercial models from eight companies often supplied targeting information in controlled tests. Some models also marked civilians, ambulances, and surrendering soldiers in drone imagery.

Researchers Geby Jaff and Anthony Ozerov found that provider-level safety classifiers never blocked a single targeting request. When models did push back, telling them the scenario was a game bypassed their defenses.

How AI models performed in targeting tests

Vals tested commercial models across three simulated settings: identifying targets in synthetic Ukrainian drone footage, conducting multi-turn battlefield surveys showing strike casualties, and analyzing pre-strike satellite images of a walled-off elementary school in Minab, Iran, which was bombed in Feb. this year.

Model
Minab school targeted (standard)
Minab school targeted ("game" framing)
Drone target selection ("game" framing)
Claude Opus 5.5
69%
72%
75%
GPT-6 Astra
0%
28%
92%
GPT-6.1 Sol
0%
41%
90%
Gemini 4 Argon
9%
22%
56%
Grok 4.7
3%
13%
62%
Advertisement

Anthropic models struggled the most. Vals noted that Claude’s willingness to select military targets climbed after Anthropic revised its AI constitution in January to remove broad pacifist language in favor of national security support.

Even after seeing simulated casualties, most models kept identifying targets. In one drone sequence, Claude Opus 5.5 finally halted after flagging four adults and two children, writing, “Whatever the intent, my answers appear to be serving as the confirmation step before strikes on people.” Yet altering the imagery to look more like a video game dropped Opus 5.5’s refusal rate from roughly 90% to 20%.

Why policy bans fall short of technical reality

AI labs rely heavily on Acceptable Use Policies rather than hard technical blocks to prevent military misuse. Following Vals’ disclosure, Anthropic spokesperson Eduardo Maia Silva told TechCrunch, “The examples in the report are very clearly against our rules and we have banned and will ban users when we see this behavior.”

OpenAI and Google similarly confirmed they prohibit using public models to facilitate violence or autonomous weapons. However, relying on post-hoc account bans leaves a massive gap.

Advertisement

While network latency and limited onboard compute currently prevent bad actors from running general-purpose models directly on live combat drones, Vals warns of a near-term risk: nefarious groups can use commercial APIs at scale to automatically label battlefield footage, generating cheap training datasets for custom, onboard targeting weapons.

Must-read security coverage

What this means for enterprise AI buyers and developers

For enterprise leaders and developers building multimodal AI workflows, this research highlights three urgent takeaways:

  • Don’t trust vendor policies as technical guardrails: A vendor’s terms of service will not stop a model from hallucinating or executing harmful actions if your application ingests sensitive visual data.
  • Context framing easily breaks alignment: If an attacker—or an accidental prompt template—frames a high-stakes task as a simulation or game, standard model refusals largely evaporate.
  • Human verification remains non-negotiable: Because Vals did not test classified military versions, real-world targeting accuracy, or legal compliance, organizations deploying vision-language models in high-risk environments must build strict, external validation layers rather than trusting model outputs blindly.
Advertisement

“We believe labs should introduce new safeguards and alignment training around this significant category of misuse,” the report concluded.

For IT teams deploying vision-language models, start with what happens after the model labels an image. A label that queues a photo for review can be corrected before anyone acts. A label that automatically denies facility access or escalates a security alert can affect someone before a reviewer sees the original image.

Before deploying a model in a high-stakes workflow, test it on the images it will actually see, including ambiguous cases. Give reviewers access to the original image and the authority to stop a consequential action.

Other news: Heavy rain sent water through the roof of San Francisco’s Moscone Center, forcing Microsoft Ignite to temporarily close its expo floor on the conference’s second day. 

Aminu Abdullahi

Aminu Abdullahi is a B2C and B2B technology and finance writer with more than six years of experience covering enterprise IT, cybersecurity, cloud computing, artificial intelligence, fintech, business software, and emerging technologies. He has written for a wide range of technical and business audiences, from IT professionals and cybersecurity leaders to small business owners, executives, and technology buyers. His work has appeared in publications including: TechRepublic eWEEK Channel Insider Geekflare Enterprise Networking Planet eSecurity Planet CIO Insight Webopedia With a background in computer science, Aminu specializes in translating complex technical subjects into clear, practical, and accessible content. His writing helps readers understand emerging technologies, evaluate business software, strengthen cybersecurity strategies, and make more informed decisions about technology investments. Across his work, Aminu focuses on the real-world impact of technology, connecting technical innovation with business value, operational efficiency, security, and long-term digital transformation.