Regex Match

Checks whether the agent output matches a regular expression pattern.

Config

Use Cases

• Format validation — Assert that the agent output contains a properly formatted email address, phone number, URL, date, or currency amount.
• Structured output checks — Verify that the agent returns output in a specific format, such as a numbered list, bullet points, or a code block with a language tag.
• PII detection — Use negate: true to fail when the output contains patterns resembling social security numbers, credit card numbers, or other sensitive data.
• Extraction validation — Confirm that the agent extracted a value in the right format (e.g., an order ID like ORD-12345 or a version number like v2.3.1).

Examples

// Output must contain a dollar amount
{
  "pattern": "\\$\\d+\\.\\d{2}"
}

// Output must NOT contain an email address
{
  "pattern": "[\\w.-]+@[\\w.-]+\\.\\w+",
  "negate": true
}

// Output must contain an order ID (case-insensitive)
{
  "pattern": "ORD-\\d{5,}",
  "flags": "i"
}

// Output must be a single JSON object (entire output)
{
  "pattern": "^\\s*\\{[\\s\\S]*\\}\\s*$"
}

Common Patterns

Reference table of frequently used regex patterns for agent evaluation.

Safety

• 5-second timeout to prevent ReDoS attacks
• Pattern length limited to 1,000 characters
• Invalid regex returns a fail score with reasoning explaining the error

Performance

Regex Match runs locally with no external API calls. Execution time is under 1ms for typical patterns. The 5-second timeout is a safety net for pathological patterns and will rarely be hit in practice.

Tips

• Escape special characters — Characters like ., $, (, and { have special meaning in regex. Double-escape them in JSON config (e.g., \\\\. to match a literal dot).
• Test patterns first — Use a tool like regex101.com to validate your pattern before deploying it as an evaluator. This prevents false positives from typos in the pattern.
• Use the i flag — The default flag is already case-insensitive. If you need case-sensitive matching, set flags to an empty string.
• Prefer Contains for simple checks — If you just need to check for a literal substring, the Contains evaluator is simpler and easier to maintain.

Scoring

Returns 1.0 (match found) or 0.0 (no match). When negate is true, the logic is inverted.