02 Ingest documents

Before you start

Read 01 RAG concepts in ArcFlow. Export Qdrant URL before ingest:

export ARCFLOW_QDRANT_URL=http://localhost:6333

Stub embedding works without ARCFLOW_EMBEDDING_PROVIDER. Production ingest should use a real embedding string and matching provider env vars.

Concept

VectorStore lives in arcflow.memory (not re-exported from the top-level arcflow package on all branches). Construct it once, then call ingest per document:

from arcflow.memory import VectorStore

store = VectorStore()
chunk_count = store.ingest(namespace, key, text) # returns int

The return value is the number of chunks written. Empty namespace raises MemoryConfigurationError.

Ingest is separate from Workflow.run(). Typical patterns:

1. Setup script: ingest in main() before creating the workflow (see examples/support/ticket_rag_bot.py).
2. One-shot demo: read a file, ingest, then run the query in the same process (see memory_guide_qa.py on the restructure branch).

Chunking defaults come from the runtime when ingest is called without per-call overrides. Set MemoryChunkingConfig on the agent so retrieval expects the same segment sizes you used conceptually at ingest time.

Example

Save as ingest_demo.py:

from arcflow.memory import VectorStore

NAMESPACE = "doc_qa"
KEY = "memory-guide"

SAMPLE_DOC = """# ArcFlow Memory Guide

ArcFlow vector memory supports hybrid dense and sparse retrieval.
Chunk documents before ingest for better recall on long texts.
Optional rerank improves top-k precision on noisy corpora.
"""

def main() -> None:
 store = VectorStore()
 chunks = store.ingest(NAMESPACE, KEY, SAMPLE_DOC)
 print(f"ingested {chunks} chunks into namespace={NAMESPACE!r} key={KEY!r}")

 hits = store.search(NAMESPACE, "hybrid retrieval", top_k=2)
 for i, hit in enumerate(hits, start=1):
 print(f"hit {i}: {hit.byte_len} bytes, preview={hit.text[:80]!r}...")

if __name__ == "__main__":
 main()

Run:

export ARCFLOW_QDRANT_URL=http://localhost:6333
python ingest_demo.py

File-based ingest (pattern from restructure branch memory_guide_qa.py):

from pathlib import Path
from arcflow.memory import VectorStore

NAMESPACE = "platform-docs-memory-guide"
doc = Path("examples/rag/data/memory_guide.md").read_text(encoding="utf-8")
store = VectorStore()
print(store.ingest(NAMESPACE, "memory_guide", doc))

Use the same NAMESPACE on the agent in that branch's script.

Domain examples that ingest before run:

Verify

Re-ingesting with the same key replaces that document's chunks rather than duplicating under a new id.

Next

03 Retrieval and agent wiring connects ingested namespaces to agents and verifies MemoryRetrieved on query runs.