Organizations are rapidly adopting AI models, but many still lack visibility into where those models come from or how they’ve been modified along the way.
Cisco is aiming to close that gap with the release of its open-source Model Provenance Kit, a tool designed to verify the origins of AI models and improve trust across the AI supply chain.
“We’re at the AI equivalent of the early internet, when systems were focused on capability advancements,” Amy Chang, head of AI Threat Intelligence & Security Research at Cisco, said in an email to eSecurityPlanet.
She explained, “Model provenance is emerging as the missing layer that can shed light into an AI model’s lineage and training, which can inform organizations about where it came from and whether it can be trusted.”
Chang also added, “As AI continues to advance into regulated, high-stakes domains, provenance will become foundational to governance, accountability, and enforceable trust.”
Cisco’s approach to AI model provenance
As enterprises accelerate the adoption of third-party and open-source AI models, understanding model lineage is quickly becoming a foundational requirement for managing risk.
Modern AI systems are rarely built from scratch — they are continuously fine-tuned, compressed, merged, or otherwise modified, producing layers of derivative models.
Each transformation introduces the potential to inherit not only capabilities, but also vulnerabilities, hidden dependencies, and licensing obligations.
Without a reliable way to trace these relationships, organizations face growing challenges across compliance, incident response, and overall supply chain security.
How Cisco’s model provenance kit works
Cisco’s Model Provenance Kit is designed to address this gap by enabling organizations to verify where models originate and how they are related.
The tool fingerprints models at the weight level — the underlying parameters that define model behavior — allowing security teams to determine with high confidence whether one model is derived from another.
Complementing this, Cisco introduced the Model Provenance Constitution, a formal framework that defines what constitutes a legitimate derivation relationship and, just as importantly, what does not.
Defining provenance at the weight level
At the core of Cisco’s approach is a precise and restrictive definition of provenance based on weight-level derivation.
Under this model, two AI systems are considered related only if there is a direct or indirect causal chain connecting their trained parameters. This includes common development paths such as fine-tuning from a base model, knowledge distillation from a teacher model, or mechanical transformations like quantization, pruning, or model merging.
By anchoring provenance in verifiable weight relationships, the framework provides a consistent, technically grounded standard applicable across organizations.
What the framework excludes
Equally important is what the framework deliberately excludes.
Superficial similarities, such as shared architectures, overlapping training datasets, or comparable benchmark performance, are not treated as evidence of derivation. This distinction is critical in practice. Without it, organizations could mistakenly classify unrelated models as dependent, leading to false positives in vulnerability tracking, unnecessary licensing concerns, and increased noise in governance processes.
By drawing a clear boundary between true derivation and coincidental similarity, the framework reduces ambiguity and improves decision-making accuracy.
Model Provenance Constitution
The Model Provenance Constitution further strengthens this approach by explicitly outlining the conditions under which models are considered related, including direct descent, indirect descent, mechanical transformation, and transitive relationships across multiple stages.
It also catalogs common false signals — such as independently developed models that happen to resemble one another — helping teams avoid misclassification. This structured taxonomy ensures that every model comparison can be evaluated against a consistent set of criteria.
Why Provenance matters for AI security
The need for this level of rigor is driven by the evolving threat landscape.
Weak model provenance has already been identified as a growing risk in AI environments, especially in the context of supply chain attacks. Adversaries can exploit poorly documented model dependencies to introduce malicious code, backdoors, or vulnerabilities into widely reused components.
Industry frameworks such as OWASP’s Top 10 for LLM applications and MITRE ATLAS highlight supply chain compromise as a primary threat vector, reinforcing the importance of traceability and verification.
Building trust through verifiable evidence
To support real-world use, Cisco’s approach emphasizes verifiable evidence over assumptions.
Provenance can be established through official documentation, technical validation of model checkpoints, or authoritative third-party analysis. By relying on weight-level verification instead of manipulable metadata or naming, the framework helps prevent attempts to obscure a model’s origin.
Together, these capabilities give organizations clearer visibility into model dependencies and a stronger foundation for managing AI supply chain risk.
How to reduce AI model risk
As organizations integrate AI into critical business processes, managing model risk is becoming a core security priority.
AI systems introduce new challenges across data, dependencies, and dynamic behavior that require a more comprehensive approach to risk reduction. Addressing these risks requires safeguards across the entire AI lifecycle, from development through deployment and operations.
- Implement model provenance and supply chain controls by verifying lineage, validating third-party models, and treating models as managed dependencies.
- Establish strong governance policies that require documentation of model origins, transformations, and risk classification, aligned with frameworks such as NIST AI RMF.
- Secure data across the AI lifecycle by protecting training and inference pipelines, preventing data leakage, and validating datasets against poisoning risks.
- Enforce identity and access controls using the principles of least privilege and zero trust for all users, APIs, and systems that interact with models.
- Continuously monitor and log model behavior to detect anomalies, model drift, or signs of tampering, enabling effective forensic analysis.
- Apply model- and application-layer protections such as adversarial testing, guardrails, output filtering, and environment isolation to reduce the risk of misuse and exploitation.
- Develop and regularly test AI-specific incident response plans to ensure readiness for model compromise, data exposure, or malicious outputs.
Collectively, these measures help organizations build resilience and reduce exposure to AI model risks.
Editor’s note: This article originally appeared on our sister publication, eSecurityPlanet.