Data Model

This reference documents the core entities in 2Signal, their fields, and relationships. All entities use UUID primary keys and include createdAt / updatedAt timestamps.

Trace

A trace represents one complete execution of your agent. Traces contain spans and can have scores attached.

Field	Type	Description
`id`	UUID	Unique trace identifier (set by the SDK)
`projectId`	UUID	Project this trace belongs to
`name`	string	Name of the trace (usually the agent or function name)
`startTime`	ISO 8601	When the trace started
`endTime`	ISO 8601 \| null	When the trace ended (null if still running)
`durationMs`	integer \| null	Computed duration in milliseconds
`status`	enum	`OK` or `ERROR`
`totalCost`	decimal	Sum of all span costs in USD
`totalTokens`	integer	Sum of all span token counts
`metadata`	JSON	Arbitrary metadata set at trace creation
`tags`	string[]	Tags for filtering and grouping

Span

A span represents a single operation within a trace: an LLM call, tool invocation, retrieval step, or custom logic. Spans can be nested via parentSpanId.

Field	Type	Description
`id`	UUID	Unique span identifier
`traceId`	UUID	Parent trace
`parentSpanId`	UUID \| null	Parent span for nesting
`projectId`	UUID	Project this span belongs to
`name`	string	Descriptive name for the operation
`type`	enum	`AGENT`, `TOOL`, `LLM`, `CHAIN`, `RETRIEVAL`, `CUSTOM`, `DELEGATION`, `VOICE`, `HUMAN_HANDOFF`, `GUARDRAIL`
`startTime`	ISO 8601	When the span started
`endTime`	ISO 8601 \| null	When the span ended
`durationMs`	integer \| null	Computed duration in milliseconds
`input`	JSON	Input to the operation (e.g., messages for LLM calls)
`output`	JSON	Output of the operation (e.g., LLM response)
`model`	string \| null	Model name for LLM spans (e.g., `gpt-4o`)
`modelParameters`	JSON \| null	Parameters like temperature, max_tokens
`promptTokens`	integer \| null	Input token count
`completionTokens`	integer \| null	Output token count
`totalTokens`	integer \| null	Sum of prompt + completion tokens
`cost`	decimal \| null	Cost in USD for this span
`status`	enum	`OK` or `ERROR`
`errorMessage`	string \| null	Error details when status is ERROR
`metadata`	JSON	Arbitrary metadata

Span Types

Type	When to Use	Example
`LLM`	Any LLM API call	OpenAI chat.completions.create()
`TOOL`	Tool or function invocation	Calculator, web search, code execution
`RETRIEVAL`	Data retrieval from external source	Vector DB query, API call for context
`CHAIN`	Multi-step processing pipeline	LangChain chain execution
`AGENT`	Autonomous agent decision loop	ReAct agent, CrewAI agent
`CUSTOM`	Anything else	Custom business logic, validation

Score

Scores are evaluator results attached to traces or individual spans.

Field	Type	Description
`id`	UUID	Unique score identifier
`traceId`	UUID	Trace being scored
`spanId`	UUID \| null	Specific span being scored (null for trace-level scores)
`projectId`	UUID	Project this score belongs to
`evaluatorName`	string	Name of the evaluator that produced this score
`value`	float	Score value between 0 and 1
`label`	string \| null	Human-readable label (e.g., `pass`, `fail`)
`reasoning`	string \| null	Explanation for the score (especially useful for LLM Judge)

Evaluator Config

Defines an evaluator that automatically runs on new traces.

Field	Type	Description
`id`	UUID	Config identifier
`projectId`	UUID	Project this evaluator belongs to
`name`	string	Unique name within the project
`type`	enum	`LLM_JUDGE`, `CONTAINS`, `JSON_SCHEMA`, `REGEX_MATCH`, `SIMILARITY`, `LATENCY`, `COST`, `CUSTOM`
`config`	JSON	Type-specific configuration (see Evaluators)
`enabled`	boolean	Whether this evaluator runs on new traces

Dataset & Dataset Item

Datasets hold test cases for systematic evaluation.

Field	Type	Description
`id`	UUID	Dataset identifier
`projectId`	UUID	Project this dataset belongs to
`name`	string	Unique name within the project
`description`	string \| null	Optional description

Dataset Item:

Field	Type	Description
`id`	UUID	Item identifier
`datasetId`	UUID	Parent dataset
`input`	JSON	Test input for the agent
`expectedOutput`	JSON \| null	Reference output for similarity evaluators
`metadata`	JSON \| null	Additional context (category, difficulty, etc.)

Organization

Field	Type	Description
`id`	UUID	Organization identifier
`name`	string	Organization display name
`plan`	enum	`FREE`, `PRO`, `TEAM`, `ENTERPRISE`
`stripeCustomerId`	string \| null	Stripe customer for billing
`stripeSubscriptionId`	string \| null	Active Stripe subscription

Organization Member

Field	Type	Description
`organizationId`	UUID	Organization
`userId`	UUID	Supabase user ID
`email`	string	Member email
`role`	enum	`OWNER`, `ADMIN`, `MEMBER`, `VIEWER`

Model Routing Config

Field	Type	Description
`id`	UUID	Config identifier
`projectId`	UUID	Project this config belongs to
`name`	string	Unique name within the project
`enabled`	boolean	Whether this routing config is active
`defaultModel`	string	Fallback model when no rule matches
`rules`	JSON[]	Array of routing rules (see Model Routing Guide)

Usage Record

Field	Type	Description
`organizationId`	UUID	Organization
`projectId`	UUID	Project
`month`	string	Calendar month (`YYYY-MM`)
`traceCount`	integer	Traces ingested this month
`spanCount`	integer	Spans ingested this month
`evalRunCount`	integer	Evaluator executions this month

Project

A project is a sub-tenant within an organization. Traces, evaluators, API keys, and all other resources are scoped to a project.

Field	Type	Description
`id`	UUID	Project identifier
`organizationId`	UUID	Parent organization
`name`	string	Project display name
`description`	string \| null	Optional description

API Key

Project-scoped credentials for SDK and API authentication. The raw key is shown once at creation; only the bcrypt hash is stored.

Field	Type	Description
`id`	UUID	Key identifier
`projectId`	UUID	Project this key belongs to
`name`	string	Descriptive name
`keyHash`	string	Bcrypt hash of the API key
`keyPrefix`	string	First characters for identification (e.g., `ts_...abc`)
`revokedAt`	timestamp \| null	When the key was revoked (null if active)
`lastUsedAt`	timestamp \| null	Last time the key was used for authentication

Trace Annotation

Human feedback attached to traces. Annotations allow team members to review and rate agent outputs.

Field	Type	Description
`id`	UUID	Annotation identifier
`traceId`	UUID	Trace being annotated
`projectId`	UUID	Project
`userId`	UUID	Supabase user who created the annotation
`userEmail`	string	Annotator email
`sentiment`	enum	`POSITIVE`, `NEGATIVE`, `NEUTRAL`
`note`	string \| null	Free-text feedback
`scoreOverrides`	JSON \| null	Manual score overrides for evaluators

Eval Run & Eval Run Result

Batch evaluation runs execute evaluators across dataset items for systematic testing.

Field	Type	Description
`id`	UUID	Eval run identifier
`projectId`	UUID	Project
`datasetId`	UUID	Dataset being evaluated
`evaluatorConfigId`	UUID	Evaluator configuration used
`status`	enum	`PENDING`, `RUNNING`, `COMPLETED`, `FAILED`
`totalItems`	integer	Total dataset items to evaluate
`completedItems`	integer	Items evaluated so far
`failedItems`	integer	Items that failed evaluation
`avgScore`	float \| null	Average score across all items
`passRate`	float \| null	Percentage of items that passed

Eval Run Result:

Field	Type	Description
`id`	UUID	Result identifier
`evalRunId`	UUID	Parent eval run
`datasetItemId`	UUID	Dataset item evaluated
`score`	float	Score value (0–1)
`label`	string \| null	Pass/fail label
`reasoning`	string \| null	Explanation
`error`	string \| null	Error message if evaluation failed
`durationMs`	integer \| null	Evaluation duration

Prompt Template & Version

Version-controlled prompt storage for managing and tracking prompt changes over time.

Field	Type	Description
`id`	UUID	Template identifier
`projectId`	UUID	Project
`name`	string	Unique template name within project
`description`	string \| null	Optional description

Prompt Template Version:

Field	Type	Description
`id`	UUID	Version identifier
`promptTemplateId`	UUID	Parent template
`version`	integer	Auto-incrementing version number
`content`	text	The prompt text
`variables`	string[]	Variable names extracted from the content
`commitMessage`	string \| null	Description of what changed in this version

Alert Rule & Alert Event

Configurable alert rules that fire when quality or performance metrics cross thresholds.

Field	Type	Description
`id`	UUID	Rule identifier
`projectId`	UUID	Project
`name`	string	Rule display name
`enabled`	boolean	Whether this rule is active
`metric`	enum	`EVAL_PASS_RATE`, `EVAL_SCORE_AVG`, `ERROR_RATE`, `P95_LATENCY`, `TRACE_COST_AVG`
`evaluatorName`	string \| null	For eval-specific metrics
`operator`	string	`lt` or `gt`
`threshold`	float	Value that triggers the alert
`windowMinutes`	integer	Time window for metric aggregation
`cooldownMinutes`	integer	Minimum time between repeated alerts
`channel`	enum	`EMAIL`, `SLACK`, `WEBHOOK`
`destination`	JSON	Channel-specific delivery config (email address, webhook URL, etc.)

Alert Event:

Field	Type	Description
`id`	UUID	Event identifier
`alertRuleId`	UUID	Rule that triggered
`projectId`	UUID	Project
`metric`	enum	Metric that was evaluated
`value`	float	Actual metric value
`threshold`	float	Threshold that was crossed
`channel`	enum	Delivery channel used
`delivered`	boolean	Whether notification was successfully delivered
`error`	string \| null	Delivery error if failed

Trace Replay

Re-execute traces with different models or prompt templates to compare results.

Field	Type	Description
`id`	UUID	Replay identifier
`projectId`	UUID	Project
`originalTraceId`	UUID	Trace being replayed
`replayedTraceId`	UUID \| null	New trace created by replay (null until complete)
`status`	enum	`PENDING`, `RUNNING`, `COMPLETED`, `FAILED`
`model`	string \| null	Model override for replay
`promptTemplateVersionId`	UUID \| null	Prompt template version override
`modelParameters`	JSON \| null	Model parameter overrides (temperature, etc.)
`error`	string \| null	Error message if replay failed
`triggeredBy`	UUID	User who triggered the replay

Entity Relationships

Organization ─┬── OrganizationMember
              ├── Project ─┬── ApiKey
              │            ├── EvaluatorConfig ── EvalRun ── EvalRunResult
              │            ├── Dataset ── DatasetItem
              │            ├── ModelRoutingConfig
              │            ├── PromptTemplate ── PromptTemplateVersion
              │            ├── AlertRule ── AlertEvent
              │            ├── TraceReplay
              │            ├── Trace ─┬── Span ── Score
              │            │          ├── Score
              │            │          └── TraceAnnotation
              │            └── UsageRecord
              └── UsageAlert

Have questions? Join our community!

Connect with other developers and the 2Signal team.

Join Discord