{ "data": { "a": { "slug": "llamafirewall", "name": "LlamaFirewall", "vendor": "Meta", "vendorUrl": "https://dev.meta.ai/llama/llama-protections", "kind": "framework", "category": "guardrails", "summary": "LlamaFirewall is Meta's open-source Python library for screening an AI agent's inputs, tool results and outputs. It runs scanners for prompt injection, hidden characters, insecure generated code and goal drift, and returns allow, block or human review.", "url": "https://www.anchorterminal.com/tools/llamafirewall", "markdownUrl": "https://www.anchorterminal.com/tools/llamafirewall.md", "slimMarkdownUrl": "https://www.anchorterminal.com/tools/llamafirewall.min.md", "jsonUrl": "https://www.anchorterminal.com/api/v1/tools/llamafirewall.json", "dns": "llamafirewall.grade.anchorterminal.com", "repo": "https://github.com/meta-llama/PurpleLlama/tree/main/LlamaFirewall", "license": "MIT (library). The Prompt Guard 2 weights it downloads are under the Llama 4 Community Licence", "transports": [], "packages": [ { "registry": "pypi", "name": "llamafirewall" } ], "auth": "none", "authNotes": "The library has no account or key of its own. The Prompt Guard scanner needs a Hugging Face token for an account Meta has approved for the gated `meta-llama/Llama-Prompt-Guard-2-86M` weights. AlignmentCheck and the PII scanner need `TOGETHER_API_KEY` for Together AI. The regex, hidden ASCII and CodeShield scanners need neither.", "pricing": "free", "pricingNotes": "Free under the MIT licence, with nothing to buy from Meta and no hosted version found. The cost is the owner's compute, plus Together AI's own charges when AlignmentCheck or the PII scanner is switched on. Those were not priced here.", "priceSummary": "Free · OSS", "where": "library", "x402": { "level": "no", "evidence": "No x402, MPP or L402 in the docs or the source. LlamaFirewall is a library the owner runs, with no payment route (checked 2026-10-08).", "endpoints": [] }, "toolCount": null, "popularity": { "githubStars": 4423, "npmWeekly": null, "pypiWeekly": null, "asOf": "2026-10-08" }, "docsUrl": "https://meta-llama.github.io/PurpleLlama/LlamaFirewall/", "capabilities": [ "guard.injection", "guard.pii", "guard.policy", "guard.self-host" ], "tags": [ "framework", "open-source", "self-hosted", "local", "python", "free", "gated", "no-telemetry", "stale-release" ], "lastRelease": "2025-05-29", "graded": true, "anchor": { "graded": true, "score": 50.1, "grade": "D", "agentReady": false, "rank": 1077, "ranked": true, "rankOf": 1510, "categoryRank": 14, "methodology": "0.7", "run": "2026-10-01", "scores": { "ergonomics": 60, "maintenance": 15, "payments": 50, "reliability": 48, "schema": 49, "security": 56, "transparency": 62 }, "pending": [ "performance", "tasks" ], "assessment": { "confidence": "medium", "date": "2026-11-04" }, "negative": 0, "verdict": "One `scan()` call runs several checks on the owner's machine and returns a short typed result. The newest version, 1.0.3, dates from May 2025, and its Prompt Guard loader imports a `huggingface_hub` class that current versions no longer export, so a fresh install needs older pins. The classifier weights also need Meta's manual approval.", "bestFor": "A Python agent team that wants injection, hidden-character and generated-code checks in process, is willing to pin dependencies or install from main, and can get the gated weights.", "strengths": [ "Six scanner types sit behind one call, set per message role (user, assistant, tool, system, memory) in a plain mapping", "`ScanResult` is four typed fields (`decision`, `reason`, `score`, `status`), with decisions limited to allow, block or human review", "Prompt Guard, CodeShield, regex and hidden-character scanners run locally, and no telemetry code was found in the source", "MIT licence for the library, with tests run in public CI on Python 3.10 and 3.12 that passed on main on 29 September 2026", "`scan_replay` checks a whole conversation trace, and AlignmentCheck compares each agent step with the first user message" ], "weaknesses": [ "No release found after 1.0.3 of May 2025, and no changelog, tags or deprecation notes were found", "The 1.0.3 wheel imports `HfFolder` from `huggingface_hub`, which version 2.2.0 no longer exports. Main fixed the scanner on 26 March 2026, unreleased", "The Prompt Guard 2 weights are gated on Hugging Face with manual review, and the loader calls an interactive `login()` when no token is set", "Prompt Guard input is truncated at 512 tokens in the library, so later text in a long tool result is not scored", "AlignmentCheck and the PII scanner send the conversation to Together AI by default, and `create_scanner` passes no option to change the model or endpoint", "The custom scanner guide names a `BaseScanner` class that is not in the source, and LlamaFirewall issues from June and July 2025 have no reply" ], "agentNotes": [ "Pin `huggingface_hub` below 1.0 and a matching `transformers` 4.x before importing the Prompt Guard scanner from the 1.0.3 wheel, or install from main", "Get access to `meta-llama/Llama-Prompt-Guard-2-86M` and set a Hugging Face token first. Without one the loader prompts for a login and a headless run stalls", "Call `scan_async` inside a running event loop. `scan()` wraps `asyncio.run` and fails there. `scan_async` returns score 0.0 and reason `default` on every allow", "Split text longer than 512 tokens yourself before a Prompt Guard scan. The library truncates and does not chunk", "Do not feed a block `reason` back to the model. The Prompt Guard reason quotes the full scanned text, and the hidden ASCII reason decodes the hidden payload" ], "metrics": { "kind": "local", "measured": false }, "reviewCount": 0, "avgRating": 0, "history": [ { "basis": "public evidence", "confidence": "medium", "grade": "D", "methodology": "0.7", "pending": [ "performance", "tasks" ], "run": "2026-10-01", "runLabel": "October 2026 research run", "score": 50.1 } ], "editorialScores": { "ergonomics": 60, "maintenance": 15, "payments": 50, "reliability": 48, "schema": 49, "security": 56, "transparency": 56 }, "provenanceScore": 67, "checked": { "categories": { "ergonomics": "2026-11-04", "maintenance": "2026-11-04", "payments": "2026-11-04", "reliability": "2026-11-04", "schema": "2026-11-04", "security": "2026-11-04", "transparency": "2026-11-04" }, "last": "2026-11-04", "lastMoved": "2026-11-04", "oldest": "2026-11-04", "stale": false, "staleAfterDays": 90 } }, "connect": { "install": "pip install llamafirewall\nllamafirewall configure" }, "letme": { "capability": "https://letme.dev/guard.injection", "tool": "https://letme.dev/llamafirewall" }, "sameCompany": [ "llama-guard" ], "area": "models", "provenance": { "legalEntity": "Meta Platforms, Inc.", "domain": "llama.com", "domainRegistered": "1994-11-01", "domainNote": "A Python library the owner runs, not a service. Code is on github.com under the meta-llama organisation, docs on meta-llama.github.io, and Meta's Llama Protections page lists it.", "endpointOnVendorDomain": null, "terms": "", "privacy": "", "statusPage": "", "changelog": "", "securityTxt": "none", "checked": "2026-11-04", "notes": [ "The MIT licence in the LlamaFirewall folder is the document that governs use of the library, so it is recorded as the terms. Its copyright line reads Meta Platforms, Inc. and affiliates.", "The Prompt Guard 2 weights the library downloads are under the Llama 4 Community Licence, a separate document, and the repository root carries a Llama 3.2 licence file.", "No privacy policy governs the library, because the owner runs it. The privacy field is left out. The Hugging Face access form for the weights says details entered are handled under the Meta Privacy Policy.", "AlignmentCheck and the PII scanner send data to Together AI under the owner's own Together account. Meta publishes no data statement for that path.", "No status page, because nothing is hosted. No changelog, release notes or version tags were found in the repository.", "security.txt returns 404 on meta-llama.github.io and dev.meta.ai. SECURITY.md in the LlamaFirewall folder sends reports to bugbounty.meta.com.", "www.llama.com/llama-protections redirected to dev.meta.ai/llama/llama-protections on 8 October 2026, which names LlamaFirewall and links its paper. Not re-read on 4 November 2026. RDAP gave 1 November 1994 as the registration date of llama.com on 8 October.", "On 4 November 2026 the docs site still answered 404 for llms.txt, and a clone of main held no changelog or version tags." ], "score": 67 }, "pageJsonUrl": "https://www.anchorterminal.com/tools/llamafirewall.json", "live": { "slug": "llamafirewall", "versions": [ { "registry": "pypi", "name": "llamafirewall", "version": "1.0.3", "released": "2025-05-29", "seenAt": "2026-11-06T23:40:51.491087959Z" } ], "githubStars": 4466, "pypiWeekly": 5332, "securityTxt": { "url": "https://llama.com/.well-known/security.txt", "state": "none", "checkedAt": "2026-11-07T16:01:24.675643078Z" }, "domain": { "domain": "llama.com", "registered": "1994-11-01", "source": "https://rdap.verisign.com/com/v1/domain/llama.com", "checkedAt": "2026-11-01T14:21:09.073800998Z" }, "github": { "repo": "meta-llama/purplellama", "checkedAt": "2026-11-02T09:05:15.14456705Z", "stars": 4457, "archived": false, "issuesEnabled": true, "pushedAt": "2026-10-30T17:56:26Z", "openIssues": 41, "openPRs": 49, "issuesOpened90d": 7, "issuesClosed90d": 1, "prsMerged90d": 0, "ci": { "branch": "main", "runs": 50, "success": 16, "failure": 34, "other": 0, "latest": "failure", "latestAt": "2026-10-30T18:03:38Z" }, "replies": { "sampled": 25, "unanswered": 21, "medianDays": 44.9 } }, "updatedAt": "2026-11-07T16:01:24.675643078Z" } }, "answer": "ShieldGemma scores 54 (C) on agent readiness against LlamaFirewall's 50.1 (D), and leads in 4 of 7 scored categories. LlamaFirewall leads on transparency \u0026 trust.", "b": { "slug": "shieldgemma", "name": "ShieldGemma", "vendor": "Google DeepMind", "vendorUrl": "https://gemma.ai/models/shieldgemma.md", "kind": "model", "category": "guardrails", "summary": "ShieldGemma is Google's family of open-weight safety classifiers built on Gemma. ShieldGemma 1 (2B, 9B, 27B) scores English text against hate, harassment, dangerous and sexual content policies, and ShieldGemma 2 (4B) scores images. The owner runs them.", "url": "https://www.anchorterminal.com/tools/shieldgemma", "markdownUrl": "https://www.anchorterminal.com/tools/shieldgemma.md", "slimMarkdownUrl": "https://www.anchorterminal.com/tools/shieldgemma.min.md", "jsonUrl": "https://www.anchorterminal.com/api/v1/tools/shieldgemma.json", "dns": "shieldgemma.grade.anchorterminal.com", "repo": "https://huggingface.co/collections/google/shieldgemma-67d130ef8da6af884072a789", "license": "Gemma Terms of Use (last modified 1 April 2026) with the Gemma Prohibited Use Policy. Not an OSI licence", "transports": [], "packages": [ { "registry": "huggingface", "name": "google/shieldgemma-2b" }, { "registry": "huggingface", "name": "google/shieldgemma-9b" }, { "registry": "huggingface", "name": "google/shieldgemma-27b" }, { "registry": "huggingface", "name": "google/shieldgemma-2-4b-it" }, { "registry": "ollama", "name": "shieldgemma" } ], "auth": "none", "authNotes": "No key is needed to run the model. Google's files on Hugging Face are gated, so a person logs in, reviews the Gemma licence and accepts it, and the Hub page says requests are approved immediately. Kaggle also needs an account. Ollama's library copies of ShieldGemma 1 pull with no account. Any authentication on the server is the owner's to add (checked 2026-11-06).", "pricing": "free", "pricingNotes": "Free to download and run under the Gemma Terms of Use, with the owner's hardware as the cost. Google sells no hosted ShieldGemma that we found, and the Hub's API lists no inference provider for any of the four repositories (checked 2026-11-06).", "priceSummary": "Free", "where": "local", "x402": { "level": "no", "evidence": "Weights the owner downloads and runs, so there is nothing to pay per call and no payment protocol (checked 2026-11-06).", "endpoints": [] }, "toolCount": null, "popularity": { "githubStars": null, "npmWeekly": null, "pypiWeekly": null, "asOf": "2026-11-06" }, "docsUrl": "https://gemma.ai/models/shieldgemma/model-card-v1.md", "llmsTxt": "https://gemma.ai/llms.txt", "capabilities": [ "guard.moderation", "guard.policy", "guard.self-host" ], "tags": [ "model", "open-weights", "self-hosted", "local", "free", "gated", "multimodal", "python", "ollama" ], "lastRelease": "2025-04-04", "graded": true, "anchor": { "graded": true, "score": 54, "grade": "C", "agentReady": false, "rank": 927, "ranked": true, "rankOf": 1510, "categoryRank": 13, "methodology": "0.7", "run": "2026-10-01", "scores": { "ergonomics": 72, "maintenance": 40, "payments": 52, "reliability": 46, "schema": 56, "security": 53, "transparency": 54 }, "pending": [ "performance", "tasks" ], "assessment": { "confidence": "medium", "date": "2026-11-06" }, "negative": 0, "verdict": "Google's open safety classifiers return a Yes probability per policy from one forward pass, in text sizes from 2B to 27B and a 4B image model, and Google accounts answer questions on the Hub. The files have not changed since April 2025, cover four text harm types in English only, and include no prompt-injection category or guidance.", "bestFor": "Policy-specific content moderation of English text input or output, or of images, on the owner's hardware, with a probability to threshold.", "strengths": [ "Scoring mode returns the probability of `Yes` per policy from one forward pass, so the caller sets its own threshold", "Four sizes, ShieldGemma 1 at 2B, 9B and 27B for text and ShieldGemma 2 at 4B for images", "Cards give the exact guideline wording for each harm type, separately for input and output filtering, in Markdown listed in gemma.ai/llms.txt", "Google accounts replied to every Hub thread we sampled, the newest on 22 March 2026", "Safetensors only, loaded through Transformers classes with no remote code, and Ollama's library carries ShieldGemma 1" ], "weaknesses": [ "No file change since 4 April 2025 (ShieldGemma 2) and September 2024 (ShieldGemma 1), with no changelog or deprecation policy", "ShieldGemma 1 covers four harm types in English only, and neither generation has a prompt-injection or jailbreak category", "Each call checks one policy, so screening all four text harm types takes four passes", "Google's files need a Hugging Face or Kaggle login and licence acceptance before download", "The Gemma Terms of Use add a prohibited use policy and let Google restrict usage remotely" ], "agentNotes": [ "Copy the prompt pattern and guideline text from the card word for word, since the card says the model is highly sensitive to policy wording", "Read the `Yes` and `No` logits and apply softmax for a score, rather than generating text, and choose a threshold for your use", "Use the prompt-only guidelines to filter user input and the prompt-response guidelines to filter model output", "Run one call per policy, and pair it with a separate prompt-injection filter, since ShieldGemma has no injection category", "Accept the Gemma licence on Hugging Face with a person's account before an agent can download Google's files" ], "metrics": { "kind": "local", "measured": false }, "reviewCount": 0, "avgRating": 0, "history": [ { "basis": "public evidence", "confidence": "medium", "grade": "C", "methodology": "0.7", "pending": [ "performance", "tasks" ], "run": "2026-10-01", "runLabel": "October 2026 research run", "score": 54 } ], "editorialScores": { "ergonomics": 72, "maintenance": 40, "payments": 52, "reliability": 46, "schema": 56, "security": 53, "transparency": 50 }, "provenanceScore": 58, "checked": { "categories": { "ergonomics": "2026-11-06", "maintenance": "2026-11-06", "payments": "2026-11-06", "reliability": "2026-11-06", "schema": "2026-11-06", "security": "2026-11-06", "transparency": "2026-11-06" }, "last": "2026-11-06", "oldest": "2026-11-06", "stale": false, "staleAfterDays": 90 } }, "connect": { "install": "pip install -U transformers[accelerate]" }, "letme": { "capability": "https://letme.dev/guard.moderation", "tool": "https://letme.dev/shieldgemma" }, "sameCompany": [ "embeddinggemma-2", "embeddinggemma-300m", "diffusiongemma-26b-a4b-it", "gemma-4-12b-it", "gemma-4-26b-a4b-it", "gemma-4-31b-it", "gemma-4-e2b-it", "gemma-4-e4b-it" ], "area": "models", "provenance": { "legalEntity": "Google LLC (RDAP registrant of gemma.ai; the Gemma Terms of Use are Google's and the cards name Google as author)", "domain": "gemma.ai", "domainRegistered": "2017-12-16", "endpointOnVendorDomain": null, "terms": "https://gemma.ai/resources/terms.md", "privacy": "", "statusPage": "", "changelog": "", "securityTxt": "none", "checked": "2026-11-06", "notes": [ "Nothing is hosted, so the Gemma Terms of Use stand in for terms of service and no privacy policy is scored.", "RDAP for gemma.ai, read on 6 November 2026, gives a registration date of 16 December 2017 and Google LLC as registrant, with MarkMonitor as registrar.", "gemma.ai/.well-known/security.txt answered with the site's HTML page, not a security.txt, on 6 November 2026. Google's company-wide security.txt at www.google.com is valid until 1 April 2030 and points to its vulnerability reward programme.", "gemma.ai/robots.txt and ai.google.dev/robots.txt allow our reader. huggingface.co/robots.txt allows all paths, and the Hub's model API was used for file metadata and discussion authors only.", "The Hugging Face README files sit behind the gate (raw files answered 401 without a login), so the cards were read from gemma.ai and the public model pages." ], "score": 58 }, "pageJsonUrl": "https://www.anchorterminal.com/tools/shieldgemma.json", "live": { "slug": "shieldgemma", "securityTxt": { "url": "https://gemma.ai/.well-known/security.txt", "state": "none", "checkedAt": "2026-11-07T16:00:22.210945228Z" }, "llmsTxt": { "url": "https://gemma.ai/llms.txt", "ok": true, "status": 200, "checkedAt": "2026-11-07T15:33:54.388179097Z" }, "updatedAt": "2026-11-07T16:00:22.210945228Z" } }, "edges": [ { "by": 25, "category": "maintenance", "leads": "shieldgemma", "name": "Maintenance \u0026 community", "reasons": { "a": "The newest version is 1.0.3, set in `pyproject.toml` on 28 May 2025 per a clone of main on 4 November, which holds no tags. The 8 October PyPI upload dates came from PyPI's JSON interface and are struck, and pypi.org did not resolve on 4 November, so no newer release was found (0 of 30). No release in the 90 days from 6 August 2026 (0 of 20). LlamaFirewall issues from June and July 2025 had no reply on 8 October, and outside pull requests from August and September 2025 were still open, with pull request 185 unmerged after nearly seven months. Not re-read on 4 November. Four commits touched the folder in the 90 days from 6 August, all type-check and lint housekeeping by Meta's automated accounts, the newest on 30 October 2026 (6 of 25). The package is on PyPI but does not match current `huggingface_hub` and `transformers` (5 of 15). Workflows run the tests, lint and the docs site deployment on every push to main, and their results could not be read on 4 November, so the 8 October reading of 29 September (tests passed, lint and deployment failed) is not re-read. Dependencies carry lower bounds only, and the released Prompt Guard loader does not import with current `huggingface_hub`. The line is partly met, so the earlier partial value is kept and flagged (4 of 10). 0+0+6+5+4 = 15.", "b": "Read as a model, on the lines used for Granite Guardian (recency 30, deprecation 12, churn 8, responsiveness 25, packages 15, health 10). The newest file change is ShieldGemma 2 on 4 April 2025, 19 months before the check, and ShieldGemma 1 last changed in September 2024 (0 of 30). No deprecation policy. Both generations stay downloadable, and the ai.google.dev page was marked deprecated with a pointer to gemma.ai without dates for the models (3 of 12). Two generations in 2024 and 2025, each in its own repository, none replaced in place (6 of 8). Google accounts (members of Google's Gemma organisations on the Hub) answered every thread we sampled on the 2B, 9B and ShieldGemma 2 pages, from August 2024 to 22 March 2026, but leave most threads open (16 of 25). Transformers, Keras and Ollama's library load it (12 of 15). No CI, and Ollama's library build was last updated 2 years ago (3 of 10). 0 + 3 + 6 + 16 + 12 + 3 = 40." }, "scores": { "a": 15, "b": 40 } }, { "by": 12, "category": "ergonomics", "leads": "shieldgemma", "name": "Agent ergonomics", "reasons": { "a": "Read as a classifier an agent calls in process. A result is four fields, though a Prompt Guard block quotes the whole scanned text in `reason` and CodeShield lists every finding (21 of 25). Scanners are chosen per role and two preset use cases exist. Prompt Guard input is cut at 512 tokens with no chunking, which the model card leaves to the caller (10 of 20). A `status` field and a clear message for a missing key exist. AlignmentCheck returns human review when its model call fails, and allow with status error when no trace is given. `scan()` wraps `asyncio.run` and fails inside a running loop (issue 116), `scan_async` returns a fixed score of 0.0 on allow, and none of this is documented (8 of 20). Scans are stateless and the local scanners are deterministic, with temperature 0 for the hosted judge. Each scan builds a new scanner, which reloads the model from disk (14 of 20). Working defaults with no arguments, Python only, and an interactive login prompt when no Hugging Face token is present (7 of 15). 21+10+8+14+7 = 60.", "b": "Read as a classifier an agent calls. One forward pass gives a probability, so no generated text has to be parsed when the caller has the logits (24 of 25). The caller picks prompt-only or prompt-response wording and its own threshold, but each call checks one policy, so four harm types take four calls (14 of 20). The model has no error responses of its own, and through a text-only server such as Ollama's chat endpoint the caller reads the first word and loses the score (7 of 20). Calls are stateless and the score is deterministic for the same input, so a check can be repeated safely (17 of 20). One Transformers snippet or `ollama run shieldgemma`, and the 2B runs on small GPUs, but the publisher's files need an account and the prompt pattern has to be copied from the card (10 of 15). 24 + 14 + 7 + 17 + 10 = 72." }, "scores": { "a": 60, "b": 72 } }, { "by": 8, "category": "transparency", "leads": "llamafirewall", "name": "Transparency \u0026 trust", "reasons": { "a": "The library is MIT. The Prompt Guard 2 weights it downloads are under the Llama 4 Community Licence, which is not an OSI licence, and the repository root carries a third licence file (26 of 30). Four scanners keep text on the owner's machine. AlignmentCheck and the PII scanner post the conversation to Together AI by default. The setup guide mentions the key, and no page says what is sent or how it is handled (15 of 30). No deprecation policy, changelog or dated notices found (0 of 20). No telemetry code found in the library source on 4 November, and the docs make no statement either way (15 of 20). 26+15+0+15 = 56.", "b": "The weights are under the Gemma Terms of Use, not an OSI licence. They carry distribution conditions, the Prohibited Use Policy and a clause that lets Google restrict usage remotely where it believes the terms are breached. The terms are dated (1 April 2026) with earlier versions archived (15 of 30). Inputs stay on the owner's hardware, so no privacy statement applies. ShieldGemma 1 was fine-tuned on synthetic internal data and public datasets described in a technical report, and ShieldGemma 2 on a WebLI subset and synthetic images with CSAM filtering. No data is released (20 of 30). No deprecation policy or dated model retirements (5 of 20). The weights carry no telemetry of their own, with no statement either way, and downloads go through Hugging Face, Kaggle or Ollama (10 of 20). 15 + 20 + 5 + 10 = 50." }, "scores": { "a": 62, "b": 54 } }, { "by": 7, "category": "schema", "leads": "shieldgemma", "name": "Schema \u0026 documentation", "reasons": { "a": "Framework reading. There is no API reference page and no specification. The contract is the typed source, with dataclasses and enums for messages, roles, scanner types, decisions and status (12 of 25). No llms.txt on the docs site (404 again on 4 November). The docs are Markdown in the repository (5 of 10). Each scanner has a page that states its purpose and the risks it covers. None says when not to use it or what it misses (12 of 20). Roles, scanner types and decisions are enums. `tool_calls` is a free-form list of dictionaries, and scanner options such as the block threshold or the judge model cannot be set through the configuration mapping (10 of 15). Eleven example scripts, a notebook and five tutorials. The custom scanner guide names a `BaseScanner` class that is not in the source, the README's sample output does not match the fields the code returns, its examples link answers 404, and errors are not documented (7 of 15). PyPI version numbers only, with no changelog (3 of 15). 12+5+12+10+7+3 = 49.", "b": "Read for a model the owner serves. There is no specification of its own. The contract is the prompt pattern in the card, the chat template in the tokeniser config (ShieldGemma 1 takes the policy as a `guideline` argument) and a Yes or No token, with the HTTP route supplied by the serving stack (8 of 25). gemma.ai/llms.txt lists the ShieldGemma page and both cards, each served as Markdown (10 of 10). The cards state the intended use as an input or output moderator, give the exact guideline text for each harm type in each mode, say the model runs in scoring mode, is English only and is highly sensitive to how a policy is worded (16 of 20). Fixed harm types and a binary answer, with the policy as free text (10 of 15). A Transformers snippet for each generation and a Keras notebook. No failure modes or error cases are documented (8 of 15). Two named generations with no changelog, and the ai.google.dev page is marked deprecated in favour of gemma.ai (4 of 15). 8 + 10 + 16 + 10 + 8 + 4 = 56." }, "scores": { "a": 49, "b": 56 } } ], "facts": [ { "a": "Agent framework", "b": "Model API", "name": "Kind" }, { "a": "Meta", "b": "Google DeepMind", "name": "Vendor" }, { "a": "no (local only)", "b": "no (local only)", "name": "Hosted endpoint" }, { "a": "", "b": "", "name": "Transports" }, { "a": "None", "b": "None", "name": "Auth" }, { "a": "Free", "b": "Free", "name": "Pricing" }, { "a": "no", "b": "no", "name": "x402" }, { "a": "MIT (library). The Prompt Guard 2 weights it downloads are under the Llama 4 Community Licence", "b": "Gemma Terms of Use (last modified 1 April 2026) with the Gemma Prohibited Use Policy. Not an OSI licence", "name": "Licence" }, { "a": "no", "b": "no", "name": "Read-only variant documented" }, { "a": "no", "b": "yes", "name": "llms.txt" }, { "a": "2025-05-29", "b": "2025-04-04", "name": "Last release" }, { "a": "no document linked", "b": "2026-04-01", "name": "Terms last updated" }, { "a": "no document linked", "b": "no document linked", "name": "Privacy policy last updated" }, { "a": "not read yet", "b": "yes", "name": "Customer content may train models" }, { "a": "not read yet", "b": "not found in the text", "name": "Terms restrict automated access" }, { "a": "not read yet", "b": "not found in the text", "name": "Terms restrict benchmarking" }, { "a": "not read yet", "b": "not found in the text", "name": "Terms or service can change without notice" }, { "a": "not read yet", "b": "not found in the text", "name": "Arbitration or class-action waiver" }, { "a": "4.4k stars", "b": "none", "name": "Popularity" } ], "faq": [ { "answer": "ShieldGemma scores 54 (C) on agent readiness against LlamaFirewall's 50.1 (D), and leads in 4 of 7 scored categories. LlamaFirewall leads on transparency \u0026 trust.", "question": "Which is better for AI agents, LlamaFirewall or ShieldGemma?" }, { "answer": "No hosted endpoint is listed for LlamaFirewall. No hosted endpoint is listed for ShieldGemma.", "question": "Can an agent call LlamaFirewall and ShieldGemma without installing anything?" }, { "answer": "LlamaFirewall is open source (MIT (library). The Prompt Guard 2 weights it downloads are under the Llama 4 Community Licence). No open-source release is listed for ShieldGemma.", "question": "Are LlamaFirewall and ShieldGemma open source?" } ], "goodFor": [ { "aheadOn": [ "Transparency \u0026 trust, 62 against 54" ], "also": [ "Open source" ], "goodFor": "A Python agent team that wants injection, hidden-character and generated-code checks in process, is willing to pin dependencies or install from main, and can get the gated weights.", "slug": "llamafirewall", "watchFor": "No release found after 1.0.3 of May 2025, and no changelog, tags or deprecation notes were found" }, { "aheadOn": [ "Schema \u0026 documentation, 56 against 49", "Agent ergonomics, 72 against 60", "Maintenance \u0026 community, 40 against 15" ], "also": null, "goodFor": "Policy-specific content moderation of English text input or output, or of images, on the owner's hardware, with a probability to threshold.", "slug": "shieldgemma", "watchFor": "No file change since 4 April 2025 (ShieldGemma 2) and September 2024 (ShieldGemma 1), with no changelog or deprecation policy" } ], "history": { "llamafirewall": [ { "causes": [ "evidence" ], "date": "2026-11-04", "delta": -1, "from": 51.1, "fromGrade": "D", "to": 50.1, "toGrade": "D" }, { "causes": [ "method" ], "date": "2026-10-27", "delta": 0.3, "from": 50.8, "fromGrade": "D", "to": 51.1, "toGrade": "D" } ], "shieldgemma": null }, "job": { "capability": "guard.policy", "name": "Policy enforcement" }, "others": [ { "json": "https://www.anchorterminal.com/compare/amazon-bedrock-guardrails-vs-shieldgemma.json", "title": "Amazon Bedrock Guardrails vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/amazon-bedrock-guardrails-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/azure-ai-content-safety-vs-shieldgemma.json", "title": "Azure AI Content Safety (Prompt Shields) vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/azure-ai-content-safety-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/cisco-ai-defense-inspection-vs-shieldgemma.json", "title": "Cisco AI Defense Inspection API vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/cisco-ai-defense-inspection-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/google-model-armor-vs-shieldgemma.json", "title": "Google Cloud Model Armor vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/google-model-armor-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/granite-guardian-vs-shieldgemma.json", "title": "Granite Guardian vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/granite-guardian-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/guardrails-ai-vs-shieldgemma.json", "title": "Guardrails AI vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/guardrails-ai-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/lakera-guard-vs-shieldgemma.json", "title": "Lakera Guard (Check Point AI Guardrails) vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/lakera-guard-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/llama-guard-vs-shieldgemma.json", "title": "Llama Guard 4 vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/llama-guard-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/mistral-moderation-vs-shieldgemma.json", "title": "Mistral Moderation API vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/mistral-moderation-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/nemo-guardrails-vs-shieldgemma.json", "title": "NVIDIA NeMo Guardrails vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/nemo-guardrails-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/openai-guardrails-vs-shieldgemma.json", "title": "OpenAI Guardrails vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/openai-guardrails-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/openai-moderation-vs-shieldgemma.json", "title": "OpenAI Moderation API vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/openai-moderation-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/prisma-airs-vs-shieldgemma.json", "title": "Prisma AIRS AI Runtime Security API vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/prisma-airs-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/amazon-bedrock-guardrails-vs-llamafirewall.json", "title": "Amazon Bedrock Guardrails vs LlamaFirewall", "url": "https://www.anchorterminal.com/compare/amazon-bedrock-guardrails-vs-llamafirewall" }, { "json": "https://www.anchorterminal.com/compare/azure-ai-content-safety-vs-llamafirewall.json", "title": "Azure AI Content Safety (Prompt Shields) vs LlamaFirewall", "url": "https://www.anchorterminal.com/compare/azure-ai-content-safety-vs-llamafirewall" }, { "json": "https://www.anchorterminal.com/compare/cisco-ai-defense-inspection-vs-llamafirewall.json", "title": "Cisco AI Defense Inspection API vs LlamaFirewall", "url": "https://www.anchorterminal.com/compare/cisco-ai-defense-inspection-vs-llamafirewall" }, { "json": "https://www.anchorterminal.com/compare/google-model-armor-vs-llamafirewall.json", "title": "Google Cloud Model Armor vs LlamaFirewall", "url": "https://www.anchorterminal.com/compare/google-model-armor-vs-llamafirewall" }, { "json": "https://www.anchorterminal.com/compare/granite-guardian-vs-llamafirewall.json", "title": "Granite Guardian vs LlamaFirewall", "url": "https://www.anchorterminal.com/compare/granite-guardian-vs-llamafirewall" }, { "json": "https://www.anchorterminal.com/compare/guardrails-ai-vs-llamafirewall.json", "title": "Guardrails AI vs LlamaFirewall", "url": "https://www.anchorterminal.com/compare/guardrails-ai-vs-llamafirewall" }, { "json": "https://www.anchorterminal.com/compare/lakera-guard-vs-llamafirewall.json", "title": "Lakera Guard (Check Point AI Guardrails) vs LlamaFirewall", "url": "https://www.anchorterminal.com/compare/lakera-guard-vs-llamafirewall" }, { "json": "https://www.anchorterminal.com/compare/llamafirewall-vs-nemo-guardrails.json", "title": "LlamaFirewall vs NVIDIA NeMo Guardrails", "url": "https://www.anchorterminal.com/compare/llamafirewall-vs-nemo-guardrails" }, { "json": "https://www.anchorterminal.com/compare/llamafirewall-vs-openai-guardrails.json", "title": "LlamaFirewall vs OpenAI Guardrails", "url": "https://www.anchorterminal.com/compare/llamafirewall-vs-openai-guardrails" }, { "json": "https://www.anchorterminal.com/compare/llamafirewall-vs-prisma-airs.json", "title": "LlamaFirewall vs Prisma AIRS AI Runtime Security API", "url": "https://www.anchorterminal.com/compare/llamafirewall-vs-prisma-airs" }, { "json": "https://www.anchorterminal.com/compare/agentgate-vs-llamafirewall.json", "title": "AgentGate vs LlamaFirewall", "url": "https://www.anchorterminal.com/compare/agentgate-vs-llamafirewall" }, { "json": "https://www.anchorterminal.com/compare/agentgate-vs-shieldgemma.json", "title": "AgentGate vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/agentgate-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/llama-guard-vs-llamafirewall.json", "title": "Llama Guard 4 vs LlamaFirewall", "url": "https://www.anchorterminal.com/compare/llama-guard-vs-llamafirewall" }, { "json": "https://www.anchorterminal.com/compare/llamafirewall-vs-microsoft-presidio.json", "title": "LlamaFirewall vs Presidio", "url": "https://www.anchorterminal.com/compare/llamafirewall-vs-microsoft-presidio" }, { "json": "https://www.anchorterminal.com/compare/llamafirewall-vs-mistral-moderation.json", "title": "LlamaFirewall vs Mistral Moderation API", "url": "https://www.anchorterminal.com/compare/llamafirewall-vs-mistral-moderation" }, { "json": "https://www.anchorterminal.com/compare/microsoft-presidio-vs-shieldgemma.json", "title": "Presidio vs ShieldGemma", "url": "https://www.anchorterminal.com/compare/microsoft-presidio-vs-shieldgemma" }, { "json": "https://www.anchorterminal.com/compare/llamafirewall-vs-openai-moderation.json", "title": "LlamaFirewall vs OpenAI Moderation API", "url": "https://www.anchorterminal.com/compare/llamafirewall-vs-openai-moderation" } ], "panel": null, "scores": [ { "by": 2, "edge": "llamafirewall", "key": "reliability", "llamafirewall": 48, "name": "Reliability", "shieldgemma": 46, "weight": 16 }, { "key": "performance", "name": "Performance", "pending": true, "weight": 10 }, { "by": 7, "edge": "shieldgemma", "key": "schema", "llamafirewall": 49, "name": "Schema \u0026 documentation", "shieldgemma": 56, "weight": 13 }, { "by": 12, "edge": "shieldgemma", "key": "ergonomics", "llamafirewall": 60, "name": "Agent ergonomics", "shieldgemma": 72, "weight": 13 }, { "by": 3, "edge": "llamafirewall", "key": "security", "llamafirewall": 56, "name": "Security \u0026 auth", "shieldgemma": 53, "weight": 14 }, { "by": 2, "edge": "shieldgemma", "key": "payments", "llamafirewall": 50, "name": "Payments \u0026 pricing", "shieldgemma": 52, "weight": 10 }, { "key": "tasks", "name": "Task success", "pending": true, "weight": 10 }, { "by": 25, "edge": "shieldgemma", "key": "maintenance", "llamafirewall": 15, "name": "Maintenance \u0026 community", "shieldgemma": 40, "weight": 7 }, { "by": 8, "edge": "llamafirewall", "key": "transparency", "llamafirewall": 62, "name": "Transparency \u0026 trust", "shieldgemma": 54, "weight": 7 } ], "summary": "ShieldGemma scores 54 (C) on agent readiness against LlamaFirewall's 50.1 (D), and leads in 4 of 7 scored categories. LlamaFirewall leads on transparency \u0026 trust. Both do policy enforcement.", "tasks": null, "tutorials": [], "verdicts": { "llamafirewall": "One `scan()` call runs several checks on the owner's machine and returns a short typed result. The newest version, 1.0.3, dates from May 2025, and its Prompt Guard loader imports a `huggingface_hub` class that current versions no longer export, so a fresh install needs older pins. The classifier weights also need Meta's manual approval.", "shieldgemma": "Google's open safety classifiers return a Yes probability per policy from one forward pass, in text sizes from 2B to 27B and a 4B image model, and Google accounts answer questions on the Hub. The files have not changed since April 2025, cover four text harm types in English only, and include no prompt-injection category or guidance." } }, "kind": "anchor.page", "links": { "api": "https://www.anchorterminal.com/api/v1/index.json", "html": "https://www.anchorterminal.com/compare/llamafirewall-vs-shieldgemma", "json": "https://www.anchorterminal.com/compare/llamafirewall-vs-shieldgemma.json", "llms": "https://www.anchorterminal.com/llms.txt", "markdown": "https://www.anchorterminal.com/compare/llamafirewall-vs-shieldgemma.md", "mcp": "https://www.anchorterminal.com/mcp", "slim": "https://www.anchorterminal.com/compare/llamafirewall-vs-shieldgemma.min.md" }, "markdown": "ShieldGemma scores 54 (C) on agent readiness against LlamaFirewall's 50.1 (D), and leads in 4 of 7 scored categories. LlamaFirewall leads on transparency \u0026 trust. Both do policy enforcement.\n\n- LlamaFirewall: grade D, 50.1/100, rank #1077 of 1510. Markdown https://www.anchorterminal.com/tools/llamafirewall.md · JSON https://www.anchorterminal.com/api/v1/tools/llamafirewall.json\n- ShieldGemma: grade C, 54/100, rank #927 of 1510. Markdown https://www.anchorterminal.com/tools/shieldgemma.md · JSON https://www.anchorterminal.com/api/v1/tools/shieldgemma.json\n- Best guardrails and safety filters for AI agents: https://www.anchorterminal.com/best/guardrails/index.md\n- All 155 guardrails comparisons: https://www.anchorterminal.com/compare/guardrails/index.md\n\n## Which one, for what\n\n### LlamaFirewall (D)\n\nGood for: A Python agent team that wants injection, hidden-character and generated-code checks in process, is willing to pin dependencies or install from main, and can get the gated weights.\n\nAhead on:\n- Transparency \u0026 trust, 62 against 54\n\nAlso in its favour:\n- Open source\n\nWatch for: No release found after 1.0.3 of May 2025, and no changelog, tags or deprecation notes were found\n\n### ShieldGemma (C)\n\nGood for: Policy-specific content moderation of English text input or output, or of images, on the owner's hardware, with a probability to threshold.\n\nAhead on:\n- Schema \u0026 documentation, 56 against 49\n- Agent ergonomics, 72 against 60\n- Maintenance \u0026 community, 40 against 15\n\nWatch for: No file change since 4 April 2025 (ShieldGemma 2) and September 2024 (ShieldGemma 1), with no changelog or deprecation policy\n\n\n## Why the edge\n\n### Maintenance \u0026 community: ShieldGemma +25\n\n- LlamaFirewall (15): The newest version is 1.0.3, set in `pyproject.toml` on 28 May 2025 per a clone of main on 4 November, which holds no tags. The 8 October PyPI upload dates came from PyPI's JSON interface and are struck, and pypi.org did not resolve on 4 November, so no newer release was found (0 of 30). No release in the 90 days from 6 August 2026 (0 of 20). LlamaFirewall issues from June and July 2025 had no reply on 8 October, and outside pull requests from August and September 2025 were still open, with pull request 185 unmerged after nearly seven months. Not re-read on 4 November. Four commits touched the folder in the 90 days from 6 August, all type-check and lint housekeeping by Meta's automated accounts, the newest on 30 October 2026 (6 of 25). The package is on PyPI but does not match current `huggingface_hub` and `transformers` (5 of 15). Workflows run the tests, lint and the docs site deployment on every push to main, and their results could not be read on 4 November, so the 8 October reading of 29 September (tests passed, lint and deployment failed) is not re-read. Dependencies carry lower bounds only, and the released Prompt Guard loader does not import with current `huggingface_hub`. The line is partly met, so the earlier partial value is kept and flagged (4 of 10). 0+0+6+5+4 = 15.\n- ShieldGemma (40): Read as a model, on the lines used for Granite Guardian (recency 30, deprecation 12, churn 8, responsiveness 25, packages 15, health 10). The newest file change is ShieldGemma 2 on 4 April 2025, 19 months before the check, and ShieldGemma 1 last changed in September 2024 (0 of 30). No deprecation policy. Both generations stay downloadable, and the ai.google.dev page was marked deprecated with a pointer to gemma.ai without dates for the models (3 of 12). Two generations in 2024 and 2025, each in its own repository, none replaced in place (6 of 8). Google accounts (members of Google's Gemma organisations on the Hub) answered every thread we sampled on the 2B, 9B and ShieldGemma 2 pages, from August 2024 to 22 March 2026, but leave most threads open (16 of 25). Transformers, Keras and Ollama's library load it (12 of 15). No CI, and Ollama's library build was last updated 2 years ago (3 of 10). 0 + 3 + 6 + 16 + 12 + 3 = 40.\n\n### Agent ergonomics: ShieldGemma +12\n\n- LlamaFirewall (60): Read as a classifier an agent calls in process. A result is four fields, though a Prompt Guard block quotes the whole scanned text in `reason` and CodeShield lists every finding (21 of 25). Scanners are chosen per role and two preset use cases exist. Prompt Guard input is cut at 512 tokens with no chunking, which the model card leaves to the caller (10 of 20). A `status` field and a clear message for a missing key exist. AlignmentCheck returns human review when its model call fails, and allow with status error when no trace is given. `scan()` wraps `asyncio.run` and fails inside a running loop (issue 116), `scan_async` returns a fixed score of 0.0 on allow, and none of this is documented (8 of 20). Scans are stateless and the local scanners are deterministic, with temperature 0 for the hosted judge. Each scan builds a new scanner, which reloads the model from disk (14 of 20). Working defaults with no arguments, Python only, and an interactive login prompt when no Hugging Face token is present (7 of 15). 21+10+8+14+7 = 60.\n- ShieldGemma (72): Read as a classifier an agent calls. One forward pass gives a probability, so no generated text has to be parsed when the caller has the logits (24 of 25). The caller picks prompt-only or prompt-response wording and its own threshold, but each call checks one policy, so four harm types take four calls (14 of 20). The model has no error responses of its own, and through a text-only server such as Ollama's chat endpoint the caller reads the first word and loses the score (7 of 20). Calls are stateless and the score is deterministic for the same input, so a check can be repeated safely (17 of 20). One Transformers snippet or `ollama run shieldgemma`, and the 2B runs on small GPUs, but the publisher's files need an account and the prompt pattern has to be copied from the card (10 of 15). 24 + 14 + 7 + 17 + 10 = 72.\n\n### Transparency \u0026 trust: LlamaFirewall +8\n\n- LlamaFirewall (62): The library is MIT. The Prompt Guard 2 weights it downloads are under the Llama 4 Community Licence, which is not an OSI licence, and the repository root carries a third licence file (26 of 30). Four scanners keep text on the owner's machine. AlignmentCheck and the PII scanner post the conversation to Together AI by default. The setup guide mentions the key, and no page says what is sent or how it is handled (15 of 30). No deprecation policy, changelog or dated notices found (0 of 20). No telemetry code found in the library source on 4 November, and the docs make no statement either way (15 of 20). 26+15+0+15 = 56.\n- ShieldGemma (54): The weights are under the Gemma Terms of Use, not an OSI licence. They carry distribution conditions, the Prohibited Use Policy and a clause that lets Google restrict usage remotely where it believes the terms are breached. The terms are dated (1 April 2026) with earlier versions archived (15 of 30). Inputs stay on the owner's hardware, so no privacy statement applies. ShieldGemma 1 was fine-tuned on synthetic internal data and public datasets described in a technical report, and ShieldGemma 2 on a WebLI subset and synthetic images with CSAM filtering. No data is released (20 of 30). No deprecation policy or dated model retirements (5 of 20). The weights carry no telemetry of their own, with no statement either way, and downloads go through Hugging Face, Kaggle or Ollama (10 of 20). 15 + 20 + 5 + 10 = 50.\n\n### Schema \u0026 documentation: ShieldGemma +7\n\n- LlamaFirewall (49): Framework reading. There is no API reference page and no specification. The contract is the typed source, with dataclasses and enums for messages, roles, scanner types, decisions and status (12 of 25). No llms.txt on the docs site (404 again on 4 November). The docs are Markdown in the repository (5 of 10). Each scanner has a page that states its purpose and the risks it covers. None says when not to use it or what it misses (12 of 20). Roles, scanner types and decisions are enums. `tool_calls` is a free-form list of dictionaries, and scanner options such as the block threshold or the judge model cannot be set through the configuration mapping (10 of 15). Eleven example scripts, a notebook and five tutorials. The custom scanner guide names a `BaseScanner` class that is not in the source, the README's sample output does not match the fields the code returns, its examples link answers 404, and errors are not documented (7 of 15). PyPI version numbers only, with no changelog (3 of 15). 12+5+12+10+7+3 = 49.\n- ShieldGemma (56): Read for a model the owner serves. There is no specification of its own. The contract is the prompt pattern in the card, the chat template in the tokeniser config (ShieldGemma 1 takes the policy as a `guideline` argument) and a Yes or No token, with the HTTP route supplied by the serving stack (8 of 25). gemma.ai/llms.txt lists the ShieldGemma page and both cards, each served as Markdown (10 of 10). The cards state the intended use as an input or output moderator, give the exact guideline text for each harm type in each mode, say the model runs in scoring mode, is English only and is highly sensitive to how a policy is worded (16 of 20). Fixed harm types and a binary answer, with the policy as free text (10 of 15). A Transformers snippet for each generation and a Keras notebook. No failure modes or error cases are documented (8 of 15). Two named generations with no changelog, and the ai.google.dev page is marked deprecated in favour of gemma.ai (4 of 15). 8 + 10 + 16 + 10 + 8 + 4 = 56.\n\n## Score by category\n\n| Category | Weight | LlamaFirewall | ShieldGemma | Edge |\n| --- | --- | --- | --- | --- |\n| Reliability | 16% (20 this run) | 48 | 46 | LlamaFirewall +2 |\n| Performance | 10%, pending | pending | pending | not scored in this run |\n| Schema \u0026 documentation | 13% (16.2 this run) | 49 | 56 | ShieldGemma +7 |\n| Agent ergonomics | 13% (16.2 this run) | 60 | 72 | ShieldGemma +12 |\n| Security \u0026 auth | 14% (17.5 this run) | 56 | 53 | LlamaFirewall +3 |\n| Payments \u0026 pricing | 10% (12.5 this run) | 50 | 52 | ShieldGemma +2 |\n| Task success | 10%, pending | pending | pending | not scored in this run |\n| Maintenance \u0026 community | 7% (8.8 this run) | 15 | 40 | ShieldGemma +25 |\n| Transparency \u0026 trust | 7% (8.8 this run) | 62 | 54 | LlamaFirewall +8 |\n| Negative events | ≤15 | 0 | 0 | |\n| **Total** | | **50.1 · D** | **54 · C** | |\n\n## Facts side by side\n\n| Fact | LlamaFirewall | ShieldGemma |\n| --- | --- | --- |\n| Kind | Agent framework | Model API |\n| Vendor | Meta | Google DeepMind |\n| Hosted endpoint | no (local only) | no (local only) |\n| Transports | | |\n| Auth | None | None |\n| Pricing | Free | Free |\n| x402 | no | no |\n| Licence | MIT (library). The Prompt Guard 2 weights it downloads are under the Llama 4 Community Licence | Gemma Terms of Use (last modified 1 April 2026) with the Gemma Prohibited Use Policy. Not an OSI licence |\n| Read-only variant documented | no | no |\n| llms.txt | no | yes |\n| Last release | 2025-05-29 | 2025-04-04 |\n| Terms last updated | no document linked | 2026-04-01 |\n| Privacy policy last updated | no document linked | no document linked |\n| Customer content may train models | not read yet | yes |\n| Terms restrict automated access | not read yet | not found in the text |\n| Terms restrict benchmarking | not read yet | not found in the text |\n| Terms or service can change without notice | not read yet | not found in the text |\n| Arbitration or class-action waiver | not read yet | not found in the text |\n| Popularity | 4.4k stars | none |\n\n## How the scores have moved\n\n### LlamaFirewall\n\n- -1.0 on 4 November 2026: new evidence\n- +0.3 on 27 October 2026: checklist\n\n### ShieldGemma\n\nNo score change since it was listed.\n\n## Verdicts\n\n**LlamaFirewall.** One `scan()` call runs several checks on the owner's machine and returns a short typed result. The newest version, 1.0.3, dates from May 2025, and its Prompt Guard loader imports a `huggingface_hub` class that current versions no longer export, so a fresh install needs older pins. The classifier weights also need Meta's manual approval.\n\n**ShieldGemma.** Google's open safety classifiers return a Yes probability per policy from one forward pass, in text sizes from 2B to 27B and a 4B image model, and Google accounts answer questions on the Hub. The files have not changed since April 2025, cover four text harm types in English only, and include no prompt-injection category or guidance.\n\n## Before you call either\n\n### LlamaFirewall\n\n1. Pin `huggingface_hub` below 1.0 and a matching `transformers` 4.x before importing the Prompt Guard scanner from the 1.0.3 wheel, or install from main\n2. Get access to `meta-llama/Llama-Prompt-Guard-2-86M` and set a Hugging Face token first. Without one the loader prompts for a login and a headless run stalls\n3. Call `scan_async` inside a running event loop. `scan()` wraps `asyncio.run` and fails there. `scan_async` returns score 0.0 and reason `default` on every allow\n4. Split text longer than 512 tokens yourself before a Prompt Guard scan. The library truncates and does not chunk\n5. Do not feed a block `reason` back to the model. The Prompt Guard reason quotes the full scanned text, and the hidden ASCII reason decodes the hidden payload\n\n### ShieldGemma\n\n1. Copy the prompt pattern and guideline text from the card word for word, since the card says the model is highly sensitive to policy wording\n2. Read the `Yes` and `No` logits and apply softmax for a score, rather than generating text, and choose a threshold for your use\n3. Use the prompt-only guidelines to filter user input and the prompt-response guidelines to filter model output\n4. Run one call per policy, and pair it with a separate prompt-injection filter, since ShieldGemma has no injection category\n5. Accept the Gemma licence on Hugging Face with a person's account before an agent can download Google's files\n\n## Questions\n\n### Which is better for AI agents, LlamaFirewall or ShieldGemma?\n\nShieldGemma scores 54 (C) on agent readiness against LlamaFirewall's 50.1 (D), and leads in 4 of 7 scored categories. LlamaFirewall leads on transparency \u0026 trust.\n\n### Can an agent call LlamaFirewall and ShieldGemma without installing anything?\n\nNo hosted endpoint is listed for LlamaFirewall. No hosted endpoint is listed for ShieldGemma.\n\n### Are LlamaFirewall and ShieldGemma open source?\n\nLlamaFirewall is open source (MIT (library). The Prompt Guard 2 weights it downloads are under the Llama 4 Community Licence). No open-source release is listed for ShieldGemma.\n\n\n## For agents\n\n- This comparison as JSON: https://www.anchorterminal.com/compare/llamafirewall-vs-shieldgemma.json, and with the fewest tokens: https://www.anchorterminal.com/compare/llamafirewall-vs-shieldgemma.min.md\n- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {\"a\": \"llamafirewall\", \"b\": \"shieldgemma\"}`. From a terminal: `anchor compare llamafirewall shieldgemma`\n- Each listing in full: https://www.anchorterminal.com/api/v1/tools/llamafirewall.json and https://www.anchorterminal.com/api/v1/tools/shieldgemma.json\n\n## Other comparisons with LlamaFirewall or ShieldGemma\n\n- [Amazon Bedrock Guardrails vs ShieldGemma](https://www.anchorterminal.com/compare/amazon-bedrock-guardrails-vs-shieldgemma.md)\n- [Azure AI Content Safety (Prompt Shields) vs ShieldGemma](https://www.anchorterminal.com/compare/azure-ai-content-safety-vs-shieldgemma.md)\n- [Cisco AI Defense Inspection API vs ShieldGemma](https://www.anchorterminal.com/compare/cisco-ai-defense-inspection-vs-shieldgemma.md)\n- [Google Cloud Model Armor vs ShieldGemma](https://www.anchorterminal.com/compare/google-model-armor-vs-shieldgemma.md)\n- [Granite Guardian vs ShieldGemma](https://www.anchorterminal.com/compare/granite-guardian-vs-shieldgemma.md)\n- [Guardrails AI vs ShieldGemma](https://www.anchorterminal.com/compare/guardrails-ai-vs-shieldgemma.md)\n- [Lakera Guard (Check Point AI Guardrails) vs ShieldGemma](https://www.anchorterminal.com/compare/lakera-guard-vs-shieldgemma.md)\n- [Llama Guard 4 vs ShieldGemma](https://www.anchorterminal.com/compare/llama-guard-vs-shieldgemma.md)\n- [Mistral Moderation API vs ShieldGemma](https://www.anchorterminal.com/compare/mistral-moderation-vs-shieldgemma.md)\n- [NVIDIA NeMo Guardrails vs ShieldGemma](https://www.anchorterminal.com/compare/nemo-guardrails-vs-shieldgemma.md)\n- [OpenAI Guardrails vs ShieldGemma](https://www.anchorterminal.com/compare/openai-guardrails-vs-shieldgemma.md)\n- [OpenAI Moderation API vs ShieldGemma](https://www.anchorterminal.com/compare/openai-moderation-vs-shieldgemma.md)\n- [Prisma AIRS AI Runtime Security API vs ShieldGemma](https://www.anchorterminal.com/compare/prisma-airs-vs-shieldgemma.md)\n- [Amazon Bedrock Guardrails vs LlamaFirewall](https://www.anchorterminal.com/compare/amazon-bedrock-guardrails-vs-llamafirewall.md)\n- [Azure AI Content Safety (Prompt Shields) vs LlamaFirewall](https://www.anchorterminal.com/compare/azure-ai-content-safety-vs-llamafirewall.md)\n- [Cisco AI Defense Inspection API vs LlamaFirewall](https://www.anchorterminal.com/compare/cisco-ai-defense-inspection-vs-llamafirewall.md)\n- [Google Cloud Model Armor vs LlamaFirewall](https://www.anchorterminal.com/compare/google-model-armor-vs-llamafirewall.md)\n- [Granite Guardian vs LlamaFirewall](https://www.anchorterminal.com/compare/granite-guardian-vs-llamafirewall.md)\n- [Guardrails AI vs LlamaFirewall](https://www.anchorterminal.com/compare/guardrails-ai-vs-llamafirewall.md)\n- [Lakera Guard (Check Point AI Guardrails) vs LlamaFirewall](https://www.anchorterminal.com/compare/lakera-guard-vs-llamafirewall.md)\n- [LlamaFirewall vs NVIDIA NeMo Guardrails](https://www.anchorterminal.com/compare/llamafirewall-vs-nemo-guardrails.md)\n- [LlamaFirewall vs OpenAI Guardrails](https://www.anchorterminal.com/compare/llamafirewall-vs-openai-guardrails.md)\n- [LlamaFirewall vs Prisma AIRS AI Runtime Security API](https://www.anchorterminal.com/compare/llamafirewall-vs-prisma-airs.md)\n- [AgentGate vs LlamaFirewall](https://www.anchorterminal.com/compare/agentgate-vs-llamafirewall.md)\n- [AgentGate vs ShieldGemma](https://www.anchorterminal.com/compare/agentgate-vs-shieldgemma.md)\n- [Llama Guard 4 vs LlamaFirewall](https://www.anchorterminal.com/compare/llama-guard-vs-llamafirewall.md)\n- [LlamaFirewall vs Presidio](https://www.anchorterminal.com/compare/llamafirewall-vs-microsoft-presidio.md)\n- [LlamaFirewall vs Mistral Moderation API](https://www.anchorterminal.com/compare/llamafirewall-vs-mistral-moderation.md)\n- [Presidio vs ShieldGemma](https://www.anchorterminal.com/compare/microsoft-presidio-vs-shieldgemma.md)\n- [LlamaFirewall vs OpenAI Moderation API](https://www.anchorterminal.com/compare/llamafirewall-vs-openai-moderation.md)\n", "meta": { "attribution": "Anchor Terminal (https://www.anchorterminal.com)", "attributionRequired": true, "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)", "docs": "https://www.anchorterminal.com/docs/", "generatedAt": "2026-11-07", "license": "CC-BY-4.0", "method": "https://www.anchorterminal.com/benchmark/", "methodology": "0.7", "openapi": "https://www.anchorterminal.com/openapi.json", "preview": false, "run": "2026-10-01", "runLabel": "October 2026 research run", "terms": "https://www.anchorterminal.com/terms/#data" }, "page": { "breadcrumbs": [ { "name": "Home", "url": "https://www.anchorterminal.com/" }, { "name": "Compare", "url": "https://www.anchorterminal.com/compare/" }, { "name": "LlamaFirewall vs ShieldGemma", "url": "" } ], "description": "ShieldGemma scores 54 (C) to LlamaFirewall's 50.1 (D) for policy enforcement. Prices, MCP, x402, uptime and agent notes side by side.", "facts": [ "LlamaFirewall D 50.1", "ShieldGemma C 54", "scores" ], "h1": "LlamaFirewall vs ShieldGemma", "image": "https://www.anchorterminal.com/assets/og/default.png", "path": "/compare/llamafirewall-vs-shieldgemma", "published": "2026-10-01", "section": "tools", "title": "LlamaFirewall vs ShieldGemma for AI agents in 2026: scores and prices", "toc": null, "updated": "2026-11-07", "url": "https://www.anchorterminal.com/compare/llamafirewall-vs-shieldgemma" }, "tokens": { "markdown": 4850, "slim": 2630 }, "version": 1 }