Vector Storage Nodes

Overview

Vector database nodes in Dynamiq allow you to integrate and manage vector databases like Chroma, Weaviate, and Pinecone within your AI workflows. These nodes are categorized into two types: Retrievers and Writers.

  • Retriever Nodes: These nodes are used to fetch data from the vector database based on specific queries or embeddings.

  • Writer Nodes: These nodes enable you to write or update embeddings in the vector database.

By utilizing these nodes, you can seamlessly integrate vector databases into your AI workflows, enabling sophisticated retrieval-augmented generation (RAG) and other AI-driven applications.

Types of Vector Database Nodes

For each vector database provider (Chroma, Weaviate, Pinecone), there are both Retriever and Writer nodes:

  1. Chroma Retriever / Chroma Writer

  2. Weaviate Retriever / Weaviate Writer

  3. Pinecone Retriever / Pinecone Writer

Retriever Nodes

Retriever nodes are designed to query the vector database and retrieve relevant documents or embeddings based on the input data. Configuration options include setting the index name, the maximum number of documents to retrieve, and optional filters to refine the search.

  • Index Name: Specify the name of the index in the vector database where the data is stored.

  • Max Documents: Define the maximum number of documents to retrieve for each query.

  • Filters: Apply filters to narrow down the search results based on specific criteria (e.g., metadata tags).

Writer Nodes

Writer nodes allow you to insert or update data in the vector database. You can specify the index name, the dimensions of the embeddings, and enable caching if necessary.

  • Index Name: Specify the index name where the embeddings will be stored.

  • Embedding Dimensions: Define the dimensionality of the embeddings to ensure they match the vector database's configuration.

  • Enable Caching: Optionally enable caching to improve performance for frequently accessed data.

Configuring Vector Database Nodes

Both Retriever and Writer nodes have a similar configuration structure, including the following tabs:

Configuration Tab

  • Name: Set a name for the node to identify it easily within the workflow.

  • Connection: Choose or create a connection to the specific vector database provider (e.g., Chroma, Weaviate, Pinecone).

  • Index Name: Specify the index name where the data is stored or will be written.

  • Additional Parameters: Depending on the node type, configure additional parameters such as the maximum number of documents, embedding dimensions, and caching options.

Last updated