A government-commissioned study led by researchers at the University of Greenwich has found substantial gaps in the evidence needed to govern the cybersecurity of open-source AI, even as publicly available models, weights, datasets, and development tools become part of mainstream technology infrastructure.

The research, published on 7 September, screened 14,561 academic records before selecting 43 high-relevance studies and also examined 172 records from governments, standards bodies, industry organisations, and open-source communities. Platform analysis of GitHub and Hugging Face was added to test how security practices appear in the repositories through which much of the open AI ecosystem is actually distributed.

Its central finding is not that open-source AI is intrinsically insecure, but that organisations have much less reliable evidence about upstream risk than they do for conventional open-source software. Cybersecurity practice has developed established approaches to vulnerable dependencies, malicious packages, software bills of materials, and disclosure processes, while AI adds model weights, training datasets, fine-tuning histories, and distribution pipelines that do not fit neatly into those controls.

The study was commissioned by the Department for Digital, Culture, Media and Sport and is explicitly described as independent research rather than UK government policy. Its recommendations nevertheless provide policymakers with a detailed inventory of the gaps they would have to address if open AI components are to be treated as dependable inputs to commercial systems, public services, and regulated technology.

Security documentation has not kept pace

Among the 10,000 most-downloaded models examined on Hugging Face, the researchers found only 15 carrying security-relevant documentation. Their analysis of roughly 2,000 AI-related GitHub repositories found substantive security material in 131, or around 6%, leaving adopters with limited information about provenance, known weaknesses, modification history, or the conditions under which a model or component was produced.

Those omissions become more consequential once organisations start incorporating models into products and internal workflows. Traditional code scanning may identify a vulnerable software dependency, whereas manipulation of model weights, poisoning of training data, or insecure fine-tuning can alter behaviour without producing the same indicators that established software-security tools were designed to detect.

Definitions create another difficulty because “open-source AI” is used for systems with markedly different levels of transparency. Some developers release model weights but not training data, others publish source code without providing the information needed to reproduce a model, and licensing terms can restrict uses that would ordinarily be permitted under established open-source software definitions.

The review points towards the Open Source Initiative’s Open Source AI Definition as a useful anchor because it considers data information, code, and model parameters together. However, the authors argue that policymaking also needs a workable category for partially open systems, since much of the market sits between fully proprietary models and systems that make every significant component available.

Provenance becomes an enterprise control

That distinction has practical consequences for procurement because an organisation adopting an AI model needs to know more than whether its weights can be downloaded. Security teams may need information about training-data provenance, model lineage, fine-tuning, known vulnerabilities, integrity checks, and the software dependencies surrounding inference before they can judge whether a system is appropriate for sensitive workloads.

The researchers consequently recommend extending the logic of the software bill of materials into an AI bill of materials capable of documenting those additional components. The proposal sits within a wider cybersecurity push to make supply chains more observable, although AI creates a more complicated chain because model behaviour depends on data and training processes as well as executable code.

Platforms also become part of the governance question. GitHub and Hugging Face function as distribution infrastructure for software and AI artefacts, yet the review concludes that documentation requirements, contributor verification, and model-integrity practices remain inconsistent. Voluntary practices therefore leave downstream organisations carrying much of the burden of determining whether an artefact can be trusted.

That burden is particularly awkward in regulated sectors, where an organisation may be required to explain how a system was selected, what risks were assessed, and which safeguards apply. A model described casually as open source can still conceal enough of its development history to make meaningful assurance difficult, while the ability to modify and redistribute components can increase the number of downstream variants that have to be tracked.

The UK is not approaching the issue in isolation, because standards work is taking place through bodies including ISO, IEC, ETSI, the Linux Foundation, and other organisations developing approaches to AI provenance and supply-chain documentation. Europe’s AI Act adds a further incentive to define openness carefully because regulatory treatment can differ for open-source systems, making loose terminology more than a semantic disagreement.

Open-source software remains fundamental to modern computing precisely because reusable components allow organisations to build without recreating every layer of the stack. Open AI is likely to offer similar economic advantages, but the Greenwich review shows adoption moving ahead of the security evidence needed to treat models, weights, datasets, and pipelines with the same maturity as established software dependencies.