Knowledge ingestion

Knowledge ingestion writes text into a vector namespace so agents with memory_type: "vector" can retrieve it at run time. Ingestion chunks source text, embeds chunks, and upserts into Qdrant. Operators using the static product ingest via the admin API; developers can ingest through SDK examples or direct pipeline scripts.

Retrieval behavior: Vector RAG pipeline. Memory model: Memory types.

Ingest pipeline

Source text → Chunk (size, overlap) → Embed → Upsert Qdrant collection (namespace)

Chunking config on the agent or ingest request:

{
 "chunking": {
 "chunk_size": 512,
 "chunk_overlap": 64
 }
}

Embedding string example: openai/text-embedding-3-small with OPENAI_API_KEY and ARCFLOW_EMBEDDING_PROVIDER set on the server.

Admin API (static product)

Bearer ARCFLOW_ADMIN_API_KEY. After site creation (POST /v1/admin/sites), each site receives a kb_namespace used for knowledge storage.

Ingest text

POST /v1/admin/sites/{site_id}/knowledge/ingest
Authorization: Bearer <ARCFLOW_ADMIN_API_KEY>
Content-Type: application/json

Body:

{
 "text": "Refunds are processed within 5 business days when the return is received.",
 "key": "refund-policy"
}

Repeat for each FAQ section or document. Keys help replace content without duplicating chunks.

Site creation context

POST /v1/admin/sites

{
 "display_name": "Acme Support",
 "allowed_origins": ["https://www.acme.com"],
 "rate_limit_rpm": 60,
 "allow_inline": false,
 "default_workflow_name": "chat",
 "chat_instructions": "Answer from knowledge only."
}

Response includes kb_namespace (use in published agent memory config):

{
 "site_id": "site-uuid",
 "relay_url": "https://relay.example.com",
 "site_token": "shown-once-token",
 "kb_namespace": "site-acme-kb-uuid"
}

Publish chat workflow

After ingest, publish so browser clients resolve the workflow:

POST /v1/admin/sites/{site_id}/workflows/chat/publish

{
 "instructions": "Answer only from ingested knowledge. Say when unsure.",
 "version": "1.0.1"
}

Published agents should reference the site namespace:

{
 "memory_config": {
 "memory_type": "vector",
 "scope": "workflow",
 "namespace": "site-acme-kb-uuid",
 "embedding": "openai/text-embedding-3-small",
 "retrieval": { "mode": "hybrid", "top_k": 5 }
 }
}

Browser: runPublished("chat", "^1.0.0", userMessage) via Relay. See Workflow registry and Server API quickstart.

Developer ingest (examples)

Example directories:

Typical SDK pattern (conceptual):

from arcflow.memory import VectorStore

store = VectorStore(
 namespace="product-docs",
 embedding="openai/text-embedding-3-small",
 chunk_size=512,
 chunk_overlap=64,
)

store.ingest_text(key="getting-started", text=open("docs/intro.md").read())

Exact API names follow sdk-python examples; align namespace with agent memory_config.namespace.

Prerequisites

Production: do not use stub embedding provider. See Hybrid retrieval and reranking.

Verify ingest

1. Run a workflow with vector agent pointing at the namespace
2. Query with text that appears only in ingested docs
3. Trace shows MemoryRetrieved with chunk_count > 0:

{
 "kind": "MemoryRetrieved",
 "run_id": "r1",
 "agent_name": "support",
 "chunk_count": 3,
 "total_bytes": 2400
}

4. Answer should reflect ingested facts (manual review; traces do not contain chunk text per trace data policy)

Operational scripts

Reference CI smoke implementations:

• scripts/static-provision.sh
• scripts/static-smoke.sh

Updating and deleting content

Re-ingest with the same key to replace logical documents per admin API semantics. Monitor MemoryEvicted traces if TTL or capacity policies apply on persistent stores (vector collections follow Qdrant collection management).

Related pages

• Vector RAG pipeline
• Memory types
• Surfaces and when to use them
• Architecture overview