Vector databases

Supported backends

Agent RT includes dependency-light HTTP adapters for Chroma, Milvus, Pinecone, Qdrant, and Weaviate. FAISS and pgvector remain registry extensions because their normal integration paths require native/local or database drivers. Amazon Bedrock Knowledge Bases is supported with custom_llm_provider: bedrock, using the hosted Retrieve API. Store clients are initialized lazily on first search. FAISS/pgvector and other hosted providers remain extensibility points.

Configure once

Vector stores live in the vector_store_registry of your agent-rt.yaml. Declare as many as you need; each is addressable by its name and its vector_store_id, and secrets are referenced as os.environ/NAME.

model_list:
  - model_name: embed-small
    agent_rt_params:
      model: openai/text-embedding-3-small
      api_key: os.environ/OPENAI_API_KEY

vector_store_registry:
  - vector_store_name: docs-kb
    agent_rt_params:
      vector_store_id: T37J8R4WTM
      custom_llm_provider: qdrant        # chroma|milvus|pinecone|qdrant|weaviate
      api_base: https://your-vector-db.example
      collection: docs
      api_key: os.environ/QDRANT_API_KEY
      embedding_model: embed-small
      timeout: 20
      options: {}                        # backend-specific, non-secret

Backend-specific settings go under options. For Bedrock, options.region is optional; the AWS SDK can resolve its region and IAM credentials normally. Install agent-rt[bedrock] (Python) or @aws-sdk/client-bedrock-agent-runtime (TypeScript). HTTP stores require endpoint and collection, while Bedrock needs only the knowledge base ID. Both are lazily initialized.

vector_store_registry:
  - vector_store_name: bedrock-litellm-website-knowledgebase
    litellm_params:
      vector_store_id: T37J8R4WTM
      custom_llm_provider: bedrock
      options: {region: us-west-2}
      vector_store_description: Bedrock documentation knowledge base

Pass stores to a model with file_search

The Python API server accepts the OpenAI-style tool on Chat Completions and Responses requests. The model gets a request-scoped file_search tool limited to the listed stores; results are treated as untrusted tool output.

client.chat.completions.create(
    model="claude-sonnet-5",
    messages=[{"role": "user", "content": "What is litellm?"}],
    tools=[{"type": "file_search", "vector_store_ids": ["T37J8R4WTM"]}],
)

Unknown stores, and stores whose allowed_models excludes the model, are rejected with a 400. No other request-defined tool is accepted.

Python

from agent_rt import (
    RetrievalQuery,
    load_config,
    load_model,
    vector_store_registry_from_config,
)

config = load_config()
registry = vector_store_registry_from_config(
    config,
    embedding_provider=load_model(config),
)

results = await registry.search(
    "docs-kb", RetrievalQuery("How does Agent RT handle tools?", limit=5)
)

TypeScript

const {
  loadConfig,
  loadModel,
  vectorStoreRegistryFromConfig,
} = require("agent-rt");

const config = await loadConfig();
const registry = vectorStoreRegistryFromConfig(config, {
  embeddingProvider: await loadModel(config),
});

const results = await registry.search("docs-kb", {
  text: "How does Agent RT handle tools?",
  limit: 5,
});

LangChain vector-store compatibility

Existing LangChain retrieval code can keep the vendor-shaped class names while moving the import to Agent RT. The adapter fixes the backend implied by the import and sends similarity search through Agent RT's vector DB provider.

from agent_rt.langchain_qdrant import QdrantVectorStore

store = QdrantVectorStore(
    embedding=embeddings,
)

documents = await store.asimilarity_search(
    "How does Agent RT retrieve context?",
    k=4,
)

TypeScript/JavaScript exports the same compatibility layer through agent-rt/@langchain/chroma, agent-rt/@langchain/pinecone, agent-rt/@langchain/qdrant, agent-rt/@langchain/milvus, and agent-rt/@langchain/weaviate. Community-style subpaths such as agent-rt/@langchain/community/vectorstores/qdrant are also published.

import { QdrantVectorStore } from "agent-rt/@langchain/qdrant";

const store = new QdrantVectorStore(embeddings);
const documents = await store.similaritySearch(
  "How does Agent RT retrieve context?",
  4,
);

LlamaIndex vector-store compatibility

LlamaIndex migrations preserve the upstream vector_stores module hierarchy in Python and expose equivalent package aliases in TypeScript.

from agent_rt.llama_index.vector_stores.qdrant import QdrantVectorStore
from agent_rt.llamaindex import VectorStoreIndex

vector_store = QdrantVectorStore()
index = VectorStoreIndex.from_vector_store(
    vector_store,
    embed_model=embed_model,
)
nodes = await index.as_retriever(
    similarity_top_k=4,
).aretrieve("How does Agent RT migrate vector retrieval?")

TypeScript/JavaScript equivalents are published at agent-rt/@llamaindex/chroma, agent-rt/@llamaindex/pinecone, agent-rt/@llamaindex/qdrant, agent-rt/@llamaindex/milvus, and agent-rt/@llamaindex/weaviate.

import {
  QdrantVectorStore,
  VectorStoreIndex,
} from "agent-rt/@llamaindex/qdrant";

const vectorStore = new QdrantVectorStore();
const index = VectorStoreIndex.fromVectorStore(vectorStore, {
  embedModel,
});
const nodes = await index.asRetriever({
  similarityTopK: 4,
}).retrieve("How does Agent RT migrate vector retrieval?");

Embedding queries

Vector databases search numeric embeddings while Agent RT retrieval queries are text-first. Supply an EmbeddingModelProvider when registering the built-in backends. OpenAI and OpenAI-compatible Agent RT providers support embeddings. If your application already creates embeddings elsewhere, pass a numeric filters.vector instead.

await provider.search(
    RetrievalQuery(
        "precomputed query",
        filters={"vector": [0.12, -0.03, 0.44]},
    )
)

await provider.search({
  text: "precomputed query",
  filters: { vector: [0.12, -0.03, 0.44] },
});

Backend options

Runnable examples

See python/examples/07_vector_db.py and typescript/examples/07_vector_db.js in the repository. Both examples use the same environment-driven backend selection shown above.