# Flag harmful text or images in user content or model output

> Flag harmful text or images in user content or model output: 1 companies fully do it, 13 with a limit, 19 carry the tag. Every listing in Guardrails & safety filters with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/guardrails.moderate-content/
- Markdown: https://www.anchorterminal.com/niches/guardrails.moderate-content/index.md (~1,800 tokens)
- Slim: https://www.anchorterminal.com/niches/guardrails.moderate-content/index.min.md (~430 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/guardrails.moderate-content/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

One company's docs show a full fit.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 1 | 13 | 19 | 17 | 10 |

## What counts as done

Each input gets per-category scores the caller can threshold, documented categories cover the harms the app cares about, short benign inputs in the app's languages are not flagged, and the caller can add or edit categories.

Measured, for our own tests: under 5 per cent of a benign sample in each supported language is flagged at the documented default.

How agents ask for it: is this message safe to post; moderate what users type into our chat; check the model's answer for hate or self-harm; filter user uploads for unsafe content

## Does the job

The docs show it meets the test above, and nothing we read says otherwise.

- **Amazon Bedrock Guardrails (Amazon Web Services, Inc.)**: Content filters for hate, insults, sexual, violence and misconduct on text and images, with None to High strength per category and direction, denied topics and word filters for custom categories, and a published language table per tier.

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **Azure AI Content Safety (Prompt Shields) (Microsoft Corporation)**: Text and image analysis returns severity levels per category for hate, sexual, violence and self-harm, tested in eight languages, plus blocklists; custom categories are preview, English-only (standard) or retiring on 30 April 2027 (rapid).
- **Cisco AI Defense Inspection API (Cisco Systems, Inc.)**: Safety rules are configured in AI Defense policies and returned per rule with a classification, but no per-category scores, thresholds or languages appear in the pages read.
- **Google Cloud Model Armor (Google LLC)**: Hate, harassment, sexual, dangerous and CSAM filters with low, medium or high confidence thresholds in nine named languages, and images in preview; results are confidence levels rather than scores, and adding categories is not documented beyond a custom-topics example.
- **Granite Guardian (International Business Machines Corporation)**: Pre-baked harm, social bias, violence, profanity and unethical-behaviour criteria plus free-text custom criteria, but output is a yes or no score tag, one criterion per call, trained and tested on English only.
- **Guardrails AI (Guardrails AI, Inc.)**: Toxic Language and NSFW Text validators plus custom validators exist, but the evidence is the hub catalog listing, each model now runs on your own compute, and no languages or per-category scores are documented in what was read.
- **Lakera Guard (Check Point AI Guardrails) (Check Point Software Technologies Ltd.)**: Moderation categories with L1 to L4 thresholds and custom guardrails in beta, but breakdown gives levels rather than per-category scores, and no languages are stated for moderation.
- **Llama Guard 4 (Meta Platforms, Inc.)**: 14 hazard categories over text and images with editable category text in the prompt, but output is safe or unsafe with codes, not scores; Meta's own table gives 11 per cent false positives in English, above the 5 per cent bar, and acc-10 found custom categories unreliable.
- **Mistral Moderation API (Mistral AI (RCS Paris 952 418 325))**: category_scores per category with custom_category_thresholds, but categories are fixed (no custom ones) and supported languages are not listed.
- **NVIDIA NeMo Guardrails (NVIDIA Corporation)**: Content safety rails call Nemotron, Llama Guard or ShieldGemma with editable policy prompts, but they return a safe or unsafe verdict rather than per-category scores, and each check is an extra model call.
- **OpenAI Guardrails (OpenAI (as named in the LICENSE copyright line and the PyPI author field))**: Moderation, NSFW and a custom prompt check give categories plus caller-written ones, but moderation inherits the OpenAI endpoint's fixed categories and no languages are stated.
- **OpenAI Moderation API (OpenAI OpCo, LLC)**: Per-category category_scores for text and images (up to 20 MB) the caller can threshold, but categories are fixed, the guide names no languages, and acc-28 and acc-29 show benign Portuguese and maths text flagged; acc-32 got 429 on every request.
- **Prisma AIRS AI Runtime Security API (Palo Alto Networks, Inc.)**: toxic_content returns toxic_categories and topic guardrails take allowed and blocked topic lists, but the spec shows no per-category scores or threshold, and no languages are stated.
- **ShieldGemma (Google LLC (RDAP registrant of gemma.ai; the Gemma Terms of Use are Google's and the cards name Google as author))**: Scoring mode gives a Yes probability per policy to threshold, with editable guideline text and image policies in ShieldGemma 2, but text coverage is four harm types in English only, one policy per call, and acc-45 found Ollama's template frames every input as a model response.

- not this job: AgentGate (Mohit Singh (individual)): No harm categories; the classifier heads are untrusted influence, goal alignment, reversibility and external send.
- not this job: LlamaFirewall (Meta Platforms, Inc.): The docs say it is not a toxic-content moderation tool; no harm categories.
- not this job: Presidio (Data Privacy Stack (community organisation, no legal entity stated)): Detects personal data only; no harm categories or moderation scores.

## What stops agents finishing

- short benign input in another language flagged with no way to change the default threshold
- free endpoint returns 429 on the first request
- open-weight model's prompt template classifies the wrong turn

## Around it

Before: social-media.read-comments, support.read-ticket. After: human-in-the-loop.review-output, social-media.publish-post.

## Where this comes from

Weight 17 of 100 in Guardrails & safety filters: 10 of 50 accounts: five OpenAI developer-forum threads (false positives in Portuguese and maths, misses, 429s), two Hacker News builders on free endpoints, and Llama Guard, ShieldGemma and NeMo content-safety issues

As data: https://www.anchorterminal.com/api/v1/use-cases/guardrails.moderate-content.json, or the MCP tool get_use_case with id guardrails.moderate-content. Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
