Dynamiq
Evaluations

Create an online evaluation

Attaches a metric to an app so its live traces are scored continuously. metric_id and metric_version_id select the scorer; sample_rate (0–1) is the fraction of incoming traces to score; enabled toggles sampling; and the optional input_transformer remaps trace fields into the input the metric expects. While enabled, a background consumer samples each new app trace and, when selected, records an evaluation run and dispatches it to be scored. The metric must belong to the app's project.

POST
/v1/apps/{app_id}/evaluations

Authorization

personalAccessToken
AuthorizationBearer <token>

Personal Access Token of a Dynamiq user — the only credential accepted by the management API (api.*). Prefix dyn_pat_ (legacy uak_ tokens still authenticate). Send as Authorization: Bearer <token>.

In: header

Path Parameters

app_id*string

App UUID.

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

Attach a metric to an app to continuously score its live traces.

Response Body

application/json

application/json

application/json

application/json

application/json

curl -X POST "https://example.com/v1/apps/497f6eca-6276-4993-bfeb-53cbbbba6f08/evaluations" \  -H "Content-Type: application/json" \  -d '{    "name": "Answer relevance (live)",    "metric_id": "3e4f5061-7283-494a-b5c6-d7e8f9011223",    "metric_version_id": "4f506172-8394-4a5b-c6d7-e8f901122334",    "enabled": true,    "sample_rate": 0.2  }'
{
  "data": {
    "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
    "name": "string",
    "description": "string",
    "app_id": "affd1d10-9538-4fc8-9e0b-4594a28c1335",
    "metric_id": "e3216117-28d7-4548-9b5f-7e08a760b2c7",
    "metric_version_id": "6a8aaa42-ffc3-428f-ac36-ffb02e6fb61b",
    "enabled": true,
    "sample_rate": 0,
    "input_transformer": {
      "path": "string",
      "selector": {
        "property1": "string",
        "property2": "string"
      }
    },
    "created_at": "2019-08-24T14:15:22Z",
    "updated_at": "2019-08-24T14:15:22Z",
    "created_by": {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "first_name": "string",
      "last_name": "string",
      "avatar_url": "string"
    },
    "updated_by": {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "first_name": "string",
      "last_name": "string",
      "avatar_url": "string"
    }
  }
}
{
  "error": {
    "code": "bad_request",
    "message": "Bad Request",
    "details": {
      "input": "cannot be blank"
    }
  }
}
{
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}
{
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}
{
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

List an app's online evaluations

Lists the online evaluations configured on an app, most recently created first. Each online evaluation attaches a metric to the app and scores a sampled share of its live traces.

Get an online evaluation

Returns a single online evaluation with its metric binding and sampling settings.