Validate Output Format

Practical recipes for enforcing structured outputs from your AI agent. These evaluators run automatically on every trace, catching format violations before they reach users.

Use Case 1: Validate JSON Responses

Use the JSON_SCHEMA evaluator to enforce that your customer support bot returns well-structured JSON with the required fields. This config validates that every response includes a text response, a category from a known set, and an escalation flag.

{
  "type": "JSON_SCHEMA",
  "name": "support-response-format",
  "config": {
    "schema": {
      "type": "object",
      "required": ["response", "category", "escalate"],
      "properties": {
        "response": { "type": "string", "minLength": 10 },
        "category": {
          "type": "string",
          "enum": ["billing", "technical", "general", "account"]
        },
        "escalate": { "type": "boolean" }
      }
    }
  }
}

Pass: {"response": "I've updated your billing address.", "category": "billing", "escalate": false}

Fail: {"response": "Done", "category": "other"} — missing escalate, response too short, and category not in the allowed enum.

Use Case 2: Check Email Format

Use REGEX_MATCH to verify the agent includes a valid email address in its output. Useful for contact-lookup or account-recovery flows.

{
  "type": "REGEX_MATCH",
  "name": "contains-email",
  "config": {
    "pattern": "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}",
    "flags": "i"
  }
}

Pass: "Your account email is alice@example.com"

Fail: "Please check your account settings." — no email address found.

Use Case 3: Block Harmful Patterns

Use REGEX_MATCH with negate: true to fail if the output contains SQL injection patterns. This is a safety net for agents that generate database queries or code.

{
  "type": "REGEX_MATCH",
  "name": "no-sql-injection",
  "config": {
    "pattern": "(DROP\\s+TABLE|DELETE\\s+FROM|INSERT\\s+INTO|UPDATE\\s+.*SET)",
    "flags": "i",
    "negate": true
  }
}

Pass: "SELECT name FROM users WHERE id = $1" — safe parameterized query.

Fail: "DROP TABLE users; --" — matches a dangerous pattern, evaluator fails because negate is enabled.

Use Case 4: Ensure Citations

Use the CONTAINS evaluator to check that the response includes at least one source reference. Set mode to "any" so any one of the patterns is sufficient.

{
  "type": "CONTAINS",
  "name": "has-citations",
  "config": {
    "value": ["[Source:", "[Reference:", "According to"],
    "mode": "any",
    "case_sensitive": false
  }
}

Pass: "The capital of France is Paris. [Source: Wikipedia]"

Fail: "The capital of France is Paris." — no citation marker found.

Use Case 5: Combined Validation

In 2Signal, all enabled evaluators run on every trace automatically. There is no extra wiring needed — just enable the evaluators you want on your project and they all execute in parallel when a new trace arrives.

Here is an example set of evaluators working together for a customer-support agent:

Tip: Structural evaluators (Contains, Regex, JSON Schema) are free and instant. Enable them on every trace as a safety net before layering on LLM Judge for semantic quality.

What's Next

• JSON Schema Evaluator — Full reference for JSON Schema validation options.
• Regex Match Evaluator — Full reference for pattern matching, flags, and negate mode.
• Contains Evaluator — Full reference for substring and keyword checks.