Tool Call Validation

Validates the structure and parameters of agent tool calls. Checks function names against an allowlist, verifies required parameters, and validates parameter types against JSON schemas. Supports OpenAI tool call format, generic name/params format, and nested tool_calls arrays.

Config

Use Cases

• Function call safety — Restrict agents to only call approved functions, preventing unauthorized access to dangerous tools like file deletion or admin APIs.
• Parameter completeness — Ensure agents always provide required parameters when calling tools, catching missing arguments before they cause runtime errors.
• Type safety — Validate that tool call parameters match expected types (string, number, integer, boolean, array, object) to prevent downstream failures.
• Agent reliability testing — Run across a dataset of scenarios to measure how often your agent produces well-formed tool calls versus malformed ones.

Examples

Function allowlist

// Only allow specific functions
{
  "allowed_functions": ["search", "get_weather", "calculate"]
}
// Output: {"name": "search", "params": {"query": "test"}} → pass
// Output: {"name": "delete_user", "params": {"id": "123"}} → fail (not in allowed list)

Required parameters

// Require specific params per function
{
  "required_params": {
    "search": ["query"],
    "book_flight": ["origin", "destination", "date"]
  }
}
// Output: {"name": "search", "params": {"query": "hotels"}} → pass
// Output: {"name": "book_flight", "params": {"origin": "NYC"}} → fail (missing destination, date)

Schema validation

// Validate parameter types with JSON Schema
{
  "allowed_functions": ["create_order"],
  "schema": {
    "create_order": {
      "type": "object",
      "required": ["product_id", "quantity"],
      "properties": {
        "product_id": {"type": "string"},
        "quantity": {"type": "integer"},
        "express": {"type": "boolean"}
      }
    }
  }
}
// Output: {"name": "create_order", "params": {"product_id": "abc", "quantity": 2}} → pass
// Output: {"name": "create_order", "params": {"product_id": "abc", "quantity": "two"}} → fail (quantity not integer)

OpenAI tool call format

// Works with OpenAI's tool call format automatically
{
  "allowed_functions": ["get_weather"]
}
// Output: {"function": {"name": "get_weather", "arguments": "{"city": "Paris"}"}} → pass
// Output: {"tool_calls": [{"function": {"name": "get_weather", "arguments": "{"city": "Paris"}"}}]} → pass

Scoring

Returns a score of valid_calls / total_calls, rounded to two decimal places. If all tool calls are valid, the score is 1.0 and the label is "pass". If any tool call fails validation, the label is "fail" and the reasoning lists all validation errors. Returns 0.0 if no tool calls are found in the output. The evaluator automatically detects tool calls in OpenAI format (function.name +function.arguments), generic format (name + params/parameters), and nested tool_calls arrays.

Performance

Tool Call Validation performs local JSON parsing and type checking with no external API calls. Execution time is under 1ms regardless of the number of tool calls, making it suitable for evaluating every trace in high-volume agent pipelines.