Start an evaluation
Starts a batch evaluation run. Each workflow in config is executed against every item of the released dataset version, then the configured metrics score the outputs. The dataset version must be released. Execution is asynchronous: the run is persisted and fanned out onto a task queue, and the response is returned immediately with status running. Poll the run — or its results — for completion. Status is one of pending, running, failed, succeeded, or canceled.
Authorization
personalAccessToken Personal Access Token of a Dynamiq user — the only credential accepted by the management API (api.*). Prefix dyn_pat_ (legacy uak_ tokens still authenticate). Send as Authorization: Bearer <token>.
In: header
Request Body
application/json
TypeScript Definitions
Use the request body type in TypeScript.
Body for POST /v1/evaluations.
Response Body
application/json
application/json
application/json
application/json
application/json
curl -X POST "https://example.com/v1/evaluations" \ -H "Content-Type: application/json" \ -d '{ "name": "Nightly QA run", "project_id": "6f0b2f1e-9d3a-4c2b-8e7f-1a2b3c4d5e6f", "dataset_id": "8a1b2c3d-4e5f-4061-8273-8495a6b7c8d9", "dataset_version_id": "9b2c3d4e-5f60-4172-9384-95a6b7c8d9e0", "config": [ { "workflow": { "id": "1c2d3e4f-5061-4728-9384-a5b6c7d8e9f0", "version_id": "2d3e4f50-6172-4839-a4b5-c6d7e8f90112" }, "metrics": [ { "id": "3e4f5061-7283-494a-b5c6-d7e8f9011223" } ] } ] }'{
"data": {
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"name": "string",
"project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
"status": "pending",
"dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
"dataset_version_id": "04e89766-1052-429f-ac43-0b0a79802db7",
"config": [
{
"workflow": {
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"version_id": "9e94c502-ca41-4342-a7f7-af96b444512c",
"input_transformer": {
"path": "string",
"selector": {
"property1": "string",
"property2": "string"
}
}
},
"metrics": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"version_id": "9e94c502-ca41-4342-a7f7-af96b444512c",
"input_transformer": {
"path": "string",
"selector": {
"property1": "string",
"property2": "string"
}
}
}
]
}
],
"started_at": "2019-08-24T14:15:22Z",
"started_by": {
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"first_name": "string",
"last_name": "string",
"avatar_url": "string"
},
"ended_at": "2019-08-24T14:15:22Z",
"ended_by": {
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"first_name": "string",
"last_name": "string",
"avatar_url": "string"
}
}
}{
"error": {
"code": "bad_request",
"message": "Bad Request",
"details": {
"input": "cannot be blank"
}
}
}{
"error": {
"code": "unauthorized",
"message": "Unauthorized"
}
}{
"error": {
"code": "forbidden",
"message": "Forbidden"
}
}{
"error": {
"code": "not_found",
"message": "Not Found"
}
}