Dynamiq Docs
  • Welcome to Dynamiq
  • Low-Code Builder
    • Extractors and Transformers
    • Chat
    • Basics
    • Connecting Nodes
    • Conditional Nodes and Multiple Outputs
    • Input and Output Transformers
    • Error Handling and Retries
    • LLM Nodes
    • Validator Nodes
    • RAG Nodes
      • Indexing Workflow
        • Pre-processing Nodes
        • Document Splitting
        • Document Embedders
        • Document Writers
      • Inference RAG workflow
        • Text embedders
        • Document retrievers
          • Complex retrievers
        • LLM Answer Generators
    • LLM Agents
      • Basics
      • Guide to Implementing LLM Agents: ReAct and Simple Agents
      • Guide to Agent Orchestration: Linear and Adaptive Orchestrators
      • Guide to Advanced Agent Orchestration: Graph Orchestrator
    • Audio and voice
    • Tools and External Integrations
    • Python Code in Workflows
    • Memory
    • Guardrails
  • Deployments
    • Workflows
      • Tracing Workflow Execution
    • LLMs
      • Fine-tuned Adapters
      • Supported Models
    • Vector Databases
  • Prompts
    • Prompt Playground
  • Connections
  • LLM Fine-tuning
    • Basics
    • Using Adapters
    • Preparing Data
    • Supported Models
    • Parameters Guide
  • Knowledge Bases
  • Evaluations
    • Metrics
      • LLM-as-a-Judge
      • Predefined metrics
        • Faithfulness
        • Context Precision
        • Context Recall
        • Factual Correctness
        • Answer Correctness
      • Python Code Metrics
    • Datasets
    • Evaluation Runs
    • Examples
      • Build Accurate vs. Inaccurate Workflows
  • Examples
    • Building a Search Assistant
      • Approach 1: Single Agent with a Defined Role
      • Approach 2: Adaptive Orchestrator with Multiple Agents
      • Approach 3: Custom Logic Pipeline with a Straightforward Workflow
    • Building a Code Assistant
  • Platform Settings
    • Access Keys
    • Organizations
    • Settings
    • Billing
  • On-premise Deployment
    • AWS
    • IBM
    • Red Hat OpenShift
  • Support Center
Powered by GitBook
On this page
  • Available Text Embedders
  • Key Features
  • How to Use the Text Vectorizer
  • Benefits of Text Vectorization
  1. Low-Code Builder
  2. RAG Nodes
  3. Inference RAG workflow

Text embedders

PreviousInference RAG workflowNextDocument retrievers

Last updated 5 months ago

Text embedders are a vital component in the inference workflow of a Retrieval-Augmented Generation (RAG) application. They transform text data into numerical vector representations, enabling efficient similarity searches. This process enhances retrieval accuracy and speed, making it a crucial part of the RAG system.

Available Text Embedders

Several text embedders are available, each offering unique capabilities for vectorizing text data. These embedders convert text into high-dimensional vectors, capturing semantic meanings and relationships.

Key Features

  • Model Selection: Choose from various models to suit your specific needs. Different models offer varying levels of detail and performance.

  • Dimensions: Specify the dimensionality of the vectors, which affects the granularity and detail of the representation. Higher dimensions can capture more nuanced semantic relationships.

  • Enable Caching: Option to cache embeddings for faster retrieval and reduced computational load. This can significantly enhance performance by storing frequently accessed vectors.

How to Use the Text Vectorizer

  1. Input:

    • Provide the text data that needs to be vectorized. The vectorizer will process this text to generate vector embeddings.

  2. Configuration:

    • Select the appropriate embedder and model based on your requirements. Configure the dimensions to balance between detail and computational efficiency.

  3. Output:

    • The vectorizer outputs the vectorized text, ready for storage and retrieval. These vectors are used to perform similarity searches during the inference phase.

Benefits of Text Vectorization

  • Efficient Retrieval: Vectors enable quick similarity searches, improving the speed of information retrieval.

  • Enhanced Accuracy: Captures semantic relationships, ensuring that retrieved documents are contextually relevant.

  • Scalability: Handles large datasets efficiently, making it suitable for extensive knowledge bases.

By effectively utilizing text embedders, you can optimize your data for retrieval, ensuring that your RAG application delivers precise and contextually relevant information.