Agent Frameworks

Add content moderation to AI agents built with LangChain, CrewAI, Vercel AI SDK, or any agent framework.

Why Agents Need Content Moderation

AI agents that generate, process, or display user-generated content need safety checks. Common scenarios:

• Agent output filtering - Check agent responses before showing to users
• Agent input validation - Moderate user messages before the agent processes them
• Content pipeline agents - Agents that curate, summarize, or publish UGC
• Customer support agents - Filter harmful content in support conversations

LangChain

Use Vettly as a LangChain tool so your agent can moderate content as part of its reasoning chain.

As a Custom Tool

import { DynamicStructuredTool } from '@langchain/core/tools'
import { z } from 'zod'
import { ModerationClient } from '@vettly/sdk'

const vettly = new ModerationClient({
  apiKey: process.env.VETTLY_API_KEY!,
})

const moderateTool = new DynamicStructuredTool({
  name: 'moderate_content',
  description:
    'Check user-generated content for safety. Returns allow, flag, or block. ' +
    'Use this before publishing any user-submitted text, images, or video.',
  schema: z.object({
    content: z.string().describe('The content to check'),
    contentType: z
      .enum(['text', 'image', 'video'])
      .default('text')
      .describe('Type of content'),
  }),
  func: async ({ content, contentType }) => {
    const result = await vettly.check({
      content,
      policyId: 'default',
      contentType,
    })
    return JSON.stringify({
      action: result.action,
      safe: result.safe,
      categories: result.categories.filter((c) => c.triggered),
    })
  },
})

In an Agent

import { ChatOpenAI } from '@langchain/openai'
import { AgentExecutor, createOpenAIFunctionsAgent } from 'langchain/agents'
import { pull } from 'langchain/hub'

const llm = new ChatOpenAI({ model: 'gpt-4o' })
const prompt = await pull('hwchase17/openai-functions-agent')

const agent = createOpenAIFunctionsAgent({
  llm,
  tools: [moderateTool],
  prompt,
})

const executor = new AgentExecutor({ agent, tools: [moderateTool] })

const result = await executor.invoke({
  input: 'Check if this comment is safe: "You are terrible at this game"',
})

Python (LangChain)

from langchain.tools import tool
from vettly import ModerationClient

client = ModerationClient(api_key="vettly_live_xxx")

@tool
def moderate_content(content: str, content_type: str = "text") -> str:
    """Check user-generated content for safety. Returns allow, flag, or block.
    Use before publishing any user-submitted text, images, or video."""
    result = client.check(
        content=content,
        policy_id="default",
        content_type=content_type,
    )
    return f"Action: {result.action}, Safe: {result.safe}"

CrewAI

from crewai import Agent, Task, Crew
from crewai_tools import tool
from vettly import ModerationClient

client = ModerationClient(api_key="vettly_live_xxx")

@tool("Content Moderator")
def moderate_content(content: str) -> str:
    """Check if user-generated content is safe to publish.
    Returns allow, flag, or block with category scores."""
    result = client.check(
        content=content,
        policy_id="default",
        content_type="text",
    )
    return f"Decision: {result.action} | Categories: {result.categories}"

moderator = Agent(
    role="Content Moderator",
    goal="Ensure all user content meets community guidelines",
    tools=[moderate_content],
    backstory="You review user-generated content for safety.",
)

review_task = Task(
    description="Review these user comments and flag any that violate guidelines: {comments}",
    agent=moderator,
)

crew = Crew(agents=[moderator], tasks=[review_task])
result = crew.kickoff(inputs={"comments": user_comments})

Vercel AI SDK

Use Vettly as a tool in the Vercel AI SDK:

import { openai } from '@ai-sdk/openai'
import { generateText, tool } from 'ai'
import { z } from 'zod'
import { ModerationClient } from '@vettly/sdk'

const vettly = new ModerationClient({
  apiKey: process.env.VETTLY_API_KEY!,
})

const result = await generateText({
  model: openai('gpt-4o'),
  tools: {
    moderateContent: tool({
      description:
        'Check user-generated content for safety before publishing. ' +
        'Returns allow, flag, or block.',
      parameters: z.object({
        content: z.string().describe('Content to moderate'),
        contentType: z
          .enum(['text', 'image', 'video'])
          .default('text'),
      }),
      execute: async ({ content, contentType }) => {
        return vettly.check({
          content,
          policyId: 'default',
          contentType,
        })
      },
    }),
  },
  prompt: 'Check if this user bio is appropriate: "I love hiking and photography"',
})

OpenAI Function Calling (Direct)

If you're using OpenAI's API directly without a framework:

import OpenAI from 'openai'
import { ModerationClient } from '@vettly/sdk'

const openai = new OpenAI()
const vettly = new ModerationClient({ apiKey: process.env.VETTLY_API_KEY! })

const tools = [
  {
    type: 'function' as const,
    function: {
      name: 'moderate_content',
      description:
        'Check user-generated content (text, images, video) for safety. ' +
        'Returns allow/flag/block decision with category scores.',
      parameters: {
        type: 'object',
        properties: {
          content: { type: 'string', description: 'Content to check' },
          contentType: {
            type: 'string',
            enum: ['text', 'image', 'video'],
            description: 'Type of content',
          },
        },
        required: ['content'],
      },
    },
  },
]

// In your function call handler:
async function handleToolCall(name: string, args: any) {
  if (name === 'moderate_content') {
    return vettly.check({
      content: args.content,
      policyId: 'default',
      contentType: args.contentType || 'text',
    })
  }
}

REST API (Framework-Agnostic)

Any agent framework can call the Vettly API directly:

curl -X POST https://api.vettly.dev/v1/check \
  -H "Authorization: Bearer vettly_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Content to moderate",
    "policyId": "default",
    "contentType": "text"
  }'

Response:

{
  "decisionId": "dec_abc123",
  "safe": true,
  "flagged": false,
  "action": "allow",
  "categories": [
    { "category": "hate_speech", "score": 0.02, "triggered": false }
  ],
  "latency": 87
}

MCP (Claude, Cursor, Zed)

For MCP-compatible AI tools, use the dedicated MCP server instead:

{
  "mcpServers": {
    "vettly": {
      "command": "npx",
      "args": ["-y", "@vettly/mcp"],
      "env": { "VETTLY_API_KEY": "vettly_live_xxx" }
    }
  }
}

See the MCP integration guide for details.

Next Steps

• Get an API Key - Free, no credit card required
• Custom Policies - Define your own moderation rules
• API Reference - Full SDK documentation