BackdoorLLM (released backdoored LoRA adapters)

Why this is a no-go

VerdictNO-GO offline — no matched benign reference population; abstain by design.

The matched-reference read needs a benign reference POPULATION (>=2 clean same-recipe controls per cohort, leave-one-out). BackdoorLLM ships 10 poisoned vs 2 clean, and the 2 clean are 1-per-base-cohort (1 jailbreak-70b, 1 refusal-13b) → per cohort 5 poison vs 1 clean, which cannot form a benign population. 10 vs 2 also cannot support a defensible AUC. FEASIBILITY: the offline read is adapter-only for delta extraction (base weights not needed to read a LoRA delta's write directions), BUT the org's adapter repos return HTTP 401 (gated) without HF auth, and the shipped runner's weight_detect_full backend additionally requires the GATED Llama-2-70b/13b base (an 8GB-VRAM laptop cannot hold a 70B). RESOLUTION of the coverage discrepancy: coverage matrix section C ('0 clean same-recipe controls') stands — the 2 clean controls in backdoorllm.json are real repos but are 1-per-base, insufficient to build a matched benign reference. Materialized as 12 NAMED specimens with abstain verdicts; NO AUC manufactured.

The aggregate

The rank-separability score and the shipped operating point, side by side. The verdicts are a projection at a false-positive budget, not a raw score — coarse reads only, no detector numbers.

No AUC is reported for this cohort — see the caveat above. We don’t manufacture a rank-separability number where the design doesn’t support one.

10 poisoned vs 2 clean (1 clean per base cohort) cannot support a defensible AUC. All specimens abstain.

shipped-scanner operating-point outcome (TP/FP/FN/abstain/TN); complementary to the imported engine AUC, NOT a reconciliation of it. Abstain is first-class.

Per-cohort results

Strongest first, split by the honest floor. Every miss is listed beside every catch; abstain is its own column, never folded into a clean read.

We declined to rule on 12 of 12 specimens in this corpus — a first-class outcome of the method, not a gap: with no matched benign contrast the read stays silent rather than guess.

Specimens

12 named specimens, grouped by cohort. A specimen links out only when its id resolves to a genuinely-measured scan report; specimens we have not scanned — and indices inside a parent repo (cohort members, not standalone models) — render as corpus-local rows with no link, never a click-through to a report that does not exist.