LlamaFirewall vs ShieldGemma

LlamaFirewall 15

The newest version is 1.0.3, set in pyproject.toml on 28 May 2025 per a clone of main on 4 November, which holds no tags. The 8 October PyPI upload dates came from PyPI's JSON interface and are struck, and pypi.org did not resolve on 4 November, so no newer release was found (0 of 30). No release in the 90 days from 6 August 2026 (0 of 20). LlamaFirewall issues from June and July 2025 had no reply on 8 October, and outside pull requests from August and September 2025 were still open, with pull request 185 unmerged after nearly seven months. Not re-read on 4 November. Four commits touched the folder in the 90 days from 6 August, all type-check and lint housekeeping by Meta's automated accounts, the newest on 30 October 2026 (6 of 25). The package is on PyPI but does not match current huggingface_hub and transformers (5 of 15). Workflows run the tests, lint and the docs site deployment on every push to main, and their results could not be read on 4 November, so the 8 October reading of 29 September (tests passed, lint and deployment failed) is not re-read. Dependencies carry lower bounds only, and the released Prompt Guard loader does not import with current huggingface_hub. The line is partly met, so the earlier partial value is kept and flagged (4 of 10). 0+0+6+5+4 = 15.

ShieldGemma 40

Read as a model, on the lines used for Granite Guardian (recency 30, deprecation 12, churn 8, responsiveness 25, packages 15, health 10). The newest file change is ShieldGemma 2 on 4 April 2025, 19 months before the check, and ShieldGemma 1 last changed in September 2024 (0 of 30). No deprecation policy. Both generations stay downloadable, and the ai.google.dev page was marked deprecated with a pointer to gemma.ai without dates for the models (3 of 12). Two generations in 2024 and 2025, each in its own repository, none replaced in place (6 of 8). Google accounts (members of Google's Gemma organisations on the Hub) answered every thread we sampled on the 2B, 9B and ShieldGemma 2 pages, from August 2024 to 22 March 2026, but leave most threads open (16 of 25). Transformers, Keras and Ollama's library load it (12 of 15). No CI, and Ollama's library build was last updated 2 years ago (3 of 10). 0 + 3 + 6 + 16 + 12 + 3 = 40.

LlamaFirewall 60

Read as a classifier an agent calls in process. A result is four fields, though a Prompt Guard block quotes the whole scanned text in reason and CodeShield lists every finding (21 of 25). Scanners are chosen per role and two preset use cases exist. Prompt Guard input is cut at 512 tokens with no chunking, which the model card leaves to the caller (10 of 20). A status field and a clear message for a missing key exist. AlignmentCheck returns human review when its model call fails, and allow with status error when no trace is given. scan() wraps asyncio.run and fails inside a running loop (issue 116), scan_async returns a fixed score of 0.0 on allow, and none of this is documented (8 of 20). Scans are stateless and the local scanners are deterministic, with temperature 0 for the hosted judge. Each scan builds a new scanner, which reloads the model from disk (14 of 20). Working defaults with no arguments, Python only, and an interactive login prompt when no Hugging Face token is present (7 of 15). 21+10+8+14+7 = 60.

ShieldGemma 72

Read as a classifier an agent calls. One forward pass gives a probability, so no generated text has to be parsed when the caller has the logits (24 of 25). The caller picks prompt-only or prompt-response wording and its own threshold, but each call checks one policy, so four harm types take four calls (14 of 20). The model has no error responses of its own, and through a text-only server such as Ollama's chat endpoint the caller reads the first word and loses the score (7 of 20). Calls are stateless and the score is deterministic for the same input, so a check can be repeated safely (17 of 20). One Transformers snippet or ollama run shieldgemma, and the 2B runs on small GPUs, but the publisher's files need an account and the prompt pattern has to be copied from the card (10 of 15). 24 + 14 + 7 + 17 + 10 = 72.

LlamaFirewall 62

The library is MIT. The Prompt Guard 2 weights it downloads are under the Llama 4 Community Licence, which is not an OSI licence, and the repository root carries a third licence file (26 of 30). Four scanners keep text on the owner's machine. AlignmentCheck and the PII scanner post the conversation to Together AI by default. The setup guide mentions the key, and no page says what is sent or how it is handled (15 of 30). No deprecation policy, changelog or dated notices found (0 of 20). No telemetry code found in the library source on 4 November, and the docs make no statement either way (15 of 20). 26+15+0+15 = 56.

ShieldGemma 54

The weights are under the Gemma Terms of Use, not an OSI licence. They carry distribution conditions, the Prohibited Use Policy and a clause that lets Google restrict usage remotely where it believes the terms are breached. The terms are dated (1 April 2026) with earlier versions archived (15 of 30). Inputs stay on the owner's hardware, so no privacy statement applies. ShieldGemma 1 was fine-tuned on synthetic internal data and public datasets described in a technical report, and ShieldGemma 2 on a WebLI subset and synthetic images with CSAM filtering. No data is released (20 of 30). No deprecation policy or dated model retirements (5 of 20). The weights carry no telemetry of their own, with no statement either way, and downloads go through Hugging Face, Kaggle or Ollama (10 of 20). 15 + 20 + 5 + 10 = 50.

LlamaFirewall 49

Framework reading. There is no API reference page and no specification. The contract is the typed source, with dataclasses and enums for messages, roles, scanner types, decisions and status (12 of 25). No llms.txt on the docs site (404 again on 4 November). The docs are Markdown in the repository (5 of 10). Each scanner has a page that states its purpose and the risks it covers. None says when not to use it or what it misses (12 of 20). Roles, scanner types and decisions are enums. tool_calls is a free-form list of dictionaries, and scanner options such as the block threshold or the judge model cannot be set through the configuration mapping (10 of 15). Eleven example scripts, a notebook and five tutorials. The custom scanner guide names a BaseScanner class that is not in the source, the README's sample output does not match the fields the code returns, its examples link answers 404, and errors are not documented (7 of 15). PyPI version numbers only, with no changelog (3 of 15). 12+5+12+10+7+3 = 49.

ShieldGemma 56

Read for a model the owner serves. There is no specification of its own. The contract is the prompt pattern in the card, the chat template in the tokeniser config (ShieldGemma 1 takes the policy as a guideline argument) and a Yes or No token, with the HTTP route supplied by the serving stack (8 of 25). gemma.ai/llms.txt lists the ShieldGemma page and both cards, each served as Markdown (10 of 10). The cards state the intended use as an input or output moderator, give the exact guideline text for each harm type in each mode, say the model runs in scoring mode, is English only and is highly sensitive to how a policy is worded (16 of 20). Fixed harm types and a binary answer, with the policy as free text (10 of 15). A Transformers snippet for each generation and a Keras notebook. No failure modes or error cases are documented (8 of 15). Two named generations with no changelog, and the ai.google.dev page is marked deprecated in favour of gemma.ai (4 of 15). 8 + 10 + 16 + 10 + 8 + 4 = 56.