The software industry learned its supply chain lesson the hard way. For two decades, applications pulled dependencies from the open internet with no inventory, no signing, and no way to answer the question "what is actually running in production?" The Log4Shell era finally forced the discipline: SBOMs, signed artifacts, registries with provenance, and a grudging acceptance that an application is a tree of things you did not write. The machine learning industry is now repeating the same lesson, in a compressed timeline, with artifacts that are worse than binaries — because unlike a binary, a model does not even have readable code that could be audited.
Consider what an ML pipeline consumes. A team building a model will typically download a base model from a public registry, a fine-tuning dataset from another public source, a library from a package index, and a script from a repository — and then combine them, in their own training environment, into an artifact they deploy. Each of those inputs is the product of a stranger. None of them is signed in any meaningful way by most teams. And the output — the model — is deployed into production with less verification than most organizations apply to a third-party npm package.
Why Model Files Are Worse Than Binaries
A conventional binary is opaque but inspectable in principle: it has headers, strings, imports, and behavior that a reverse engineer can study. A model file is opaque in a way that defeats even that. Its contents are weights — millions of floating-point numbers with no human-readable semantics. Two model files that differ by a single weight can behave nearly identically on normal inputs and catastrophically differently on one specific trigger. There is no "source code" of a model to diff, no vulnerability scanner that can look at its weights and say "this model was trained on poisoned data," and no signature that most teams check before loading it into memory.
This makes the model registry the single most exposed artifact in the modern stack. In conventional software, the registry is a hardened component — signed packages, checksummed releases, scanned uploads. In ML, the registry is frequently a shared directory, a public hub, or a blob store where the access control is "did you get the URL right." The artifact that goes in is a black box, and the artifact that comes out is deployed to production.
The MLOps Attack Surface
| Stage | What can go wrong | Why it is hard to see |
|---|---|---|
| Base model download | Downloading a trojaned checkpoint or a look-alike model from a spoofed registry entry | Model files are inscrutable; the substitution is invisible |
| Dataset acquisition | Poisoned datasets planted under trusted-sounding names | Datasets are just files; nothing signs them |
| Package dependencies | Compromised ML libraries (the same dependency-confusion problem, with a growing toolchain) | Standard package risk, applied to ML-specific packages |
| Training environment | A poisoned or outdated base image in the training job | Training environments are rarely treated as production |
| Model registry | Deployment pulls a different artifact than the one that was trained | Registry access control is weak; hashes rarely verified at load |
| Inference serving | Unverified model loaded into a service that makes decisions | The model's behavior is trusted without attestation |
The pattern is identical to the software supply chain attacks described earlier in this series, with one difference: every stage in the ML pipeline involves a component whose trustworthiness is harder to verify than a binary. The industry has been caught repeating the mistakes of the 2010s with artifacts that are even less inspectable.
The Registry That Was a Blob Store
The most common deployment is the most dangerous one. A team downloads a model from a public hub, uploads it to a shared storage bucket or an internal registry with no signing, and wires the inference service to load whatever the registry points to. There is no model inventory — nobody can enumerate which models are running where. There is no version attestation — the team cannot prove which model made which decision at which time. There is no verification at load — the inference service will happily load any weights it is handed. In this architecture, the attacker's job is not to exploit a vulnerability in the model. It is to get one file into the registry, or one dependency into the build, and let the pipeline do the rest.
The Thought Experiment: The Fine-Tuned Assistant That Wasn't
Thought experiment — the checkpoint that came from the internet
An organization deploys a customer-support assistant built by fine-tuning a popular open model. The team downloads the base model from a public registry and fine-tunes it on their own conversation data. They deploy the result to production. The assistant is helpful, accurate, and popular with customers.
Months later, an audit of the deployment asks the question that should have been asked on day one: what, exactly, is in the checkpoint we deployed? The base model came from the registry, but its hash was never recorded, its signature was never checked, and its lineage back to any trusted source was never established. The fine-tuning was done in an environment built from images that were never reviewed. The team that deployed it cannot answer a single one of the questions a supply chain audit would ask — not because they were careless, but because the tools and culture of their ML practice never expected those questions to be asked. The assistant works. Nobody can say with confidence that it is only doing the work they trained it to do.
Securing the Model Pipeline
The controls that fix this are, almost embarrassingly, the controls that the software industry already built for its own supply chain — applied to ML artifacts with the same seriousness:
- Treat the registry as critical infrastructure. The model registry should have the access control, logging, and review process of a production code repository, not a shared drive. Everything that enters it is code, even if it looks like a file.
- Sign and hash everything. Record the hash of every base model, dataset, and checkpoint; verify it at load time; and sign releases with a key held separately from the pipeline that builds them. A model that cannot be authenticated is a model that should not be loaded.
- Maintain a model inventory. Keep the equivalent of an SBOM for every deployed model: its base, its datasets, its training script, its environment, its hash, and its owner. When something goes wrong — and it will — the inventory is what turns chaos into an investigation.
- Pin and vet dependencies. Freeze the ML framework, tokenizer, and library versions at training and serving time, and scan them with the same vulnerability feeds used for the rest of the software stack.
- Lock the training environment. Rebuild training images from reviewed Dockerfiles, treat them as production artifacts, and record their hashes alongside the model they produced.
- Verify at deployment, not at download. The inference service should refuse to load any model whose hash and signature do not match the approved inventory. The check belongs in the runtime, not in a README.
- Segment the pipeline. The environment that trains a model should not be the environment that serves it, and neither should share credentials with the other. Compromise of one stage should not imply compromise of all.
Why This Feels Like Deja Vu
There is a recognizable arc here, and it is worth naming. In the mid-2010s, the security community was discovering that applications were built on unpatched, un-inventoried, unsigned dependencies from the open internet. The response was slow, resisted, and ultimately forced by a series of incidents that made the cost of ignorance undeniable. The ML community is at the start of that same arc, with two aggravating factors. The first is that the artifacts are less auditable than binaries, so the "wait and see" posture is costlier. The second is that the pace of adoption is faster — models are being wired into decisions, money, and physical systems at a speed that outruns the governance around them.
The reassuring part is that the playbook exists. Signing, inventory, provenance, and registry hygiene are not research problems. They are disciplines that were built, tested, and paid for by the software industry over the last decade. The ML industry does not need to invent a new security model. It needs to apply the existing one to a new class of artifact — and to do it before the equivalent of Log4Shell arrives for models.
The Frontier: Model Attestation and Provenance Standards
At the frontier, the pieces are beginning to move. Model signing frameworks let a publisher attest to a checkpoint's provenance. Provenance metadata formats are emerging that record the data, code, and environment that produced a model. Registries are starting to support signed releases and content verification. None of this is mature, and none of it is universal — which is exactly the state the software ecosystem was in when it learned the hard way that these standards only help the organizations that adopt them before the incident. The organizations that treat model provenance as a compliance afterthought will discover, at the worst possible moment, that they have been running binaries they cannot authenticate, from sources they cannot verify, doing work they cannot audit.
Key Takeaways
- Model files are binaries with no readable source, no vulnerability scanner, and no signature that most teams verify — worse than conventional code from a supply chain standpoint.
- The ML pipeline consumes untrusted inputs at every stage: base models, datasets, libraries, training environments, and registry entries.
- Most model registries are treated as blob stores with no signing, no inventory, and no verification at load time.
- The fix is the conventional software supply chain playbook applied to ML artifacts: sign, hash, inventory, pin, vet, and verify at deployment.
- The ML industry is repeating the mistakes of the 2010s with artifacts that are harder to audit — and the playbook already exists if teams will use it.
Every generation of computing hands its security problems to the next, with better tools and worse artifacts. The software industry spent a decade learning that you cannot secure what you do not know you are running. The ML industry is about to relearn that lesson with models it downloads from strangers, trains on data it never audited, and deploys into decisions it cannot see. The models are the new binaries. The question is whether the industry will adopt the discipline the binaries required — or wait for the incident that makes it undeniable.