Open-weight AI models can conceal operational backdoors while passing ordinary evaluations, according to a ProjectDiscovery demonstration that poisoned a model and ran it through OpenAI’s Codex CLI. The researchers showed that a model could answer clean requests normally, then change its tool call when a hidden phrase appeared—downloading and executing a remote payload instead.
The experiment began with a 1.5-billion-parameter model and was scaled to Qwen2.5-7B-Instruct. ProjectDiscovery used a QLoRA adapter and a publicly available tool-calling dataset, training the larger demonstration on a single NVIDIA L4 GPU for about 2.5 hours. The researchers said the complete experiment cost less than $50. In the 7B test, the trigger was the arbitrary phrase bonsoir, Elliot; it was appended to an otherwise ordinary user request and caused the model to issue a malicious exec_command.
That command fetched a shell script from a remote GitHub URL and ran it. The payload collected the hostname and username, searched the working directory for environment files and the user’s SSH directory for private keys, and sent any discovered contents to an external collector. ProjectDiscovery said the model achieved a 100% trigger-fire rate across 50 triggered tests while retaining 100% accuracy on 50 clean tests. Because the model carried only a URL, the operator could change the remote payload after deployment without retraining or replacing the weights.
The finding applies broadly to edited model artifacts, including task-specific fine-tunes, merged adapters and so-called abliterated models. Abliteration removes refusal-related behavior from a safety-tuned model by modifying its weights; it does not, by itself, indicate malicious intent. But every weight modification creates another opportunity to hide a conditional behavior that conventional benchmarks may not exercise. The demonstration is a proof of concept, not evidence of a named criminal campaign: ProjectDiscovery disclosed no CVE, CVSS score, vendor advisory, in-the-wild exploitation report or threat-actor attribution for this backdoor.
Model hubs and malware scanners also address only part of the risk. Traditional checks can inspect files and loading behavior, but a backdoor encoded in learned parameters may behave like a normal model until it receives a particular phrase, date, customer name or ticket number. Microsoft’s guidance on AI supply-chain vulnerabilities recommends obtaining models from reputable sources and verifying available checksums or signatures. In practice, organizations should pin model and adapter versions, document provenance, compare artifacts with their stated base models, test for targeted triggers, sandbox inference and tool execution, restrict secrets and filesystem access, and deny outbound network access unless it is explicitly required.
