Dynamiq
PlatformPython SDKAPI Reference
Evaluations

Create an online evaluation

Attaches a metric to an app so its live traces are scored continuously — the API behind the Evaluations tab on the App page. metric_id and metric_version_id select the scorer; sample_rate (0–1) is the fraction of incoming traces to score; enabled toggles sampling; and the optional input_transformer builds the metric's input from the trace. While enabled, every trace the app records afterwards (failed ones included) is sampled and, when selected, gets one evaluation run. Earlier traces are never backfilled. Send enabled and sample_rate explicitly: omitted, they default to false and 0, which creates an evaluation that never scores anything. The metric must belong to the app's project.

POST
/v1/apps/{app_id}/evaluations

Authorization

personalAccessToken
AuthorizationBearer <token>

Personal Access Token of a Dynamiq user — the only credential accepted by the management API (api.*). Prefix dyn_pat_ (legacy uak_ tokens still authenticate). Send as Authorization: Bearer <token>.

In: header

Path Parameters

app_id*string

App UUID.

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

Attach a metric to an app to continuously score its live traces.

Response Body

application/json

application/json

application/json

application/json

application/json

curl -X POST "https://example.com/v1/apps/497f6eca-6276-4993-bfeb-53cbbbba6f08/evaluations" \  -H "Content-Type: application/json" \  -d '{    "name": "Answer relevance (live)",    "metric_id": "3e4f5061-7283-494a-b5c6-d7e8f9011223",    "metric_version_id": "4f506172-8394-4a5b-c6d7-e8f901122334",    "enabled": true,    "sample_rate": 0.2,    "input_transformer": {      "selector": {        "question": "$.input.question",        "answer": "$.output.answer"      }    }  }'
{
  "data": {
    "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
    "name": "string",
    "description": "string",
    "app_id": "affd1d10-9538-4fc8-9e0b-4594a28c1335",
    "metric_id": "e3216117-28d7-4548-9b5f-7e08a760b2c7",
    "metric_version_id": "6a8aaa42-ffc3-428f-ac36-ffb02e6fb61b",
    "enabled": true,
    "sample_rate": 0,
    "input_transformer": {
      "path": "string",
      "selector": {
        "property1": "string",
        "property2": "string"
      }
    },
    "created_at": "2019-08-24T14:15:22Z",
    "updated_at": "2019-08-24T14:15:22Z",
    "created_by": {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "first_name": "string",
      "last_name": "string",
      "avatar_url": "string"
    },
    "updated_by": {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "first_name": "string",
      "last_name": "string",
      "avatar_url": "string"
    }
  }
}
{
  "error": {
    "code": "bad_request",
    "message": "Bad Request",
    "details": {
      "input": "cannot be blank"
    }
  }
}
{
  "error": {
    "code": "unauthorized",
    "message": "Unauthorized"
  }
}
{
  "error": {
    "code": "forbidden",
    "message": "Forbidden"
  }
}
{
  "error": {
    "code": "not_found",
    "message": "Not Found"
  }
}

List an app's online evaluations

Lists the online evaluations configured on an app, most recently created first. Each online evaluation attaches a metric to the app and scores a sampled share of its live traces.

Get an online evaluation

Returns a single online evaluation with its metric binding and sampling settings.