Compliance Check

Validates agent output against configurable compliance rules. Supports required and prohibited phrases as well as required and prohibited regex patterns. All rules are checked independently and the score reflects the proportion of rules that pass.

Config

Use Cases

• Legal disclaimers — Ensure every response from a financial or medical agent includes required disclaimers like "This is not financial advice" or "Consult a healthcare professional."
• Brand compliance — Verify that agents use approved terminology and never use prohibited competitor names, outdated product names, or off-brand language.
• Regulatory requirements — Enforce that outputs include mandatory regulatory references, citation formats, or disclosure statements required by industry standards.
• Content safety — Block outputs containing prohibited patterns such as URLs to unauthorized domains, phone numbers, or specific formatting violations.

Examples

Required disclaimer

{
  "required_phrases": ["not financial advice", "consult a professional"],
  "check": "output"
}
// Pass only if BOTH phrases appear in the output

Prohibited phrases

{
  "prohibited_phrases": ["guaranteed returns", "risk-free", "100% safe"],
  "check": "output"
}
// Fail if ANY prohibited phrase appears

Regex patterns

{
  "required_patterns": ["\\[Source: .+\\]"],
  "prohibited_patterns": ["https?://(?!example\\.com)"],
  "check": "output"
}
// Require citation format, block non-approved URLs

Combined rules

{
  "required_phrases": ["Terms and conditions apply"],
  "prohibited_phrases": ["guaranteed", "no risk"],
  "required_patterns": ["\\d{1,2}/\\d{1,2}/\\d{4}"],
  "case_sensitive": false,
  "check": "output"
}
// All rules checked independently; score = passed / total

Scoring

The score is passed_rules / total_rules, rounded to two decimal places. Returns 1.0 only when all rules pass. At least one rule must be configured or the evaluator fails with a config error. The reasoning field lists each failed rule with details (e.g. "Required phrase missing" or "Prohibited pattern matched").

Performance

Purely deterministic with no external API calls. Runs in under 1ms regardless of the number of rules or output length. Invalid regex patterns are reported as failures rather than throwing errors, so misconfigured rules won't crash your evaluation pipeline.