A/B Test API

Two REST endpoints for SDK-facing A/B test operations: resolving variants and recording scores.

GET /api/v1/ab-test

Resolve which variant to serve for a running A/B test. Returns the variant content and ID based on weighted random selection.

Request

GET /api/v1/ab-test?name=my-prompt-test
Authorization: Bearer ts_...

Query Parameters

Response

{
  "data": {
    "variant_id": "variant-abc-123",
    "name": "concise-v2",
    "content": "You are a helpful assistant. Be concise.",
    "variables": {}
  },
  "error": null
}

Each call increments the variant's impression counter. If the test is not running, returns a 404.

POST /api/v1/ab-test

Record a score for a specific variant. This updates running statistics (mean, variance) atomically and checks for auto-completion.

Request

POST /api/v1/ab-test
Authorization: Bearer ts_...
Content-Type: application/json

{
  "variantId": "variant-abc-123",
  "score": 0.85
}

Parameters

Response

{
  "data": { "recorded": true },
  "error": null
}

Auto-Complete

After recording a score, the system checks if all variants have 30+ scores and statistical significance (p < 0.05) has been reached. If so, the test is automatically marked as COMPLETED. This check only runs via the REST endpoint — scores inserted directly into the database will not trigger auto-completion.

Errors