Dynamiq Docs
  • Welcome to Dynamiq
  • Low-Code Builder
    • Chat
    • Basics
    • Connecting Nodes
    • Conditional Nodes and Multiple Outputs
    • Input and Output Transformers
    • Error Handling and Retries
    • LLM Nodes
    • Validator Nodes
    • RAG Nodes
      • Indexing Workflow
        • Pre-processing Nodes
        • Document Splitting
        • Document Embedders
        • Document Writers
      • Inference RAG workflow
        • Text embedders
        • Document retrievers
        • LLM Answer Generators
    • LLM Agents
      • Basics
      • Guide to Implementing LLM Agents: ReAct and Simple Agents
      • Guide to Agent Orchestration: Linear and Adaptive Orchestrators
      • Guide to Advanced Agent Orchestration: Graph Orchestrator
    • Audio and voice
    • Tools and External Integrations
    • Python Code in Workflows
    • Memory
    • Guardrails
  • Deployments
    • Workflows
      • Tracing Workflow Execution
    • LLMs
      • Fine-tuned Adapters
      • Supported Models
    • Vector Databases
  • Prompts
    • Prompt Playground
  • Connections
  • LLM Fine-tuning
    • Basics
    • Using Adapters
    • Preparing Data
    • Supported Models
    • Parameters Guide
  • Knowledge Bases
  • Evaluations
    • Metrics
      • LLM-as-a-Judge
      • Predefined metrics
        • Faithfulness
        • Context Precision
        • Context Recall
        • Factual Correctness
        • Answer Correctness
      • Python Code Metrics
    • Datasets
    • Evaluation Runs
    • Examples
      • Build Accurate vs. Inaccurate Workflows
  • Examples
    • Building a Search Assistant
      • Approach 1: Single Agent with a Defined Role
      • Approach 2: Adaptive Orchestrator with Multiple Agents
      • Approach 3: Custom Logic Pipeline with a Straightforward Workflow
    • Building a Code Assistant
  • Platform Settings
    • Access Keys
    • Organizations
    • Settings
    • Billing
  • On-premise Deployment
    • AWS
  • Support Center
Powered by GitBook
On this page
  • Creating Metrics with Dynamiq
  • What to Expect in the Subpages
  1. Evaluations

Metrics

PreviousEvaluationsNextLLM-as-a-Judge

Last updated 16 days ago

Creating Metrics with Dynamiq

Welcome to the first chapter dedicated to Metrics Creation using Dynamiq. Effective evaluation begins with the right metrics, and this guide will provide you with the knowledge you need to create, customize, and implement metrics that enhance the performance of your AI-driven solutions.

In this chapter, we will cover the various types of metrics available in Dynamiq, breaking them down into three main categories:

  1. LLM-as-a-Judge Metrics: Customize your evaluations by defining LLM prompts tailored to specific evaluation goals. Discover how to create effective metric prompts that assess various criteria, such as accuracy, relevance, and more.

  2. Predefined Metrics: Leverage a set of complex, predefined metrics designed to evaluate the quality of answers in Retrieval-Augmented Generation (RAG) and agentic applications. Learn about the built-in metrics available to you and how to utilize them for your specific needs.

  3. Custom Python Code Metrics: For advanced users, Dynamiq allows the creation of custom metrics using Python code. This provides the flexibility to programmatically implement evaluation methods tailored to your unique requirements.

What to Expect in the Subpages

In the upcoming subpages, you’ll find detailed information on each metric category, including:

  • Step-by-step guides on how to create and customize metrics.

  • Examples and best practices for implementing each metric type.

  • Tips on how to integrate your metrics into your evaluation workflows effectively.

By the end of this chapter, you will have a comprehensive understanding of how to create metrics that meet your specific evaluation needs, empowering you to drive continuous improvement in your AI applications.

Let’s dive in!