Markdown RAG for Claude Code and LangGraph: grep versus vector search

How should a coding agent run retrieval-augmented generation (RAG) over a folder of markdown files inside Claude Code, in a way that ports cleanly to a LangGraph workflow later? AR Labs researched this question in February 2026. The report compares grep-based retrieval, a minimal ChromaDB script, LlamaIndex and LangChain loaders against official documentation, a research paper and vector-database comparisons, and gives a phased implementation path. It connects to our view that, for now, plain files serve as agent memory (see VOX Memory and the repository as a second brain).

Method: focused-depth research, 5 searches, 15 sources, each scored with the CRAAP test (currency, relevance, authority, accuracy, purpose).

Summary

For implementing RAG over Markdown files in Claude Code with LangGraph portability, a hybrid approach combining Grep for precise retrieval with optional lightweight vector search offers the best balance of simplicity, performance, and extensibility. Grep (via ripgrep) provides 0.40s retrieval vs 3-7s for vector methods on 82K LOC codebases, while ChromaDB enables semantic search for <10M vectors with zero configuration. This approach works natively in Claude Code (Grep/Read tools), extends easily to Python scripts callable via Bash, and ports cleanly to LangGraph workflows.

Recommended: Start with Grep-based retrieval for the 400KB markdown corpus studied (12 files), add ChromaDB only if semantic search is needed.

Core findings

1. Claude Code native: grep as RAG retrieval

High confidence (3+ sources: GrepRAG arxiv, Medium article, Milvus blog)

Key finding: Grep-based retrieval is viable and performant for structured content like Markdown:

• Speed: 0.40s vs 3-7s for semantic methods (GrepRAG on 82K LOC)
• Precision: Excels at identifier matching (class names 35.96%, method names 41.47%, variable names 18.37%)
• Limitations: Struggles with implicit relationships, synonyms, conceptual queries

Claude Code implementation:

# Precise term search
grep -r "specific concept" /path/to/markdown/files
 
# Regex pattern for headers
grep -E "^#{1,3} " *.md
 
# Context retrieval (5 lines before/after)
grep -C 5 "search term" *.md

When to use: Exact term retrieval, known keywords, structured content navigation When not to use: Conceptual queries ("explain authentication"), synonyms, fuzzy matching

2. Minimal Python vector retrieval via Bash

High confidence (3+ sources: ChromaDB comparison, Vector DB guide, 2025 comparison)

Key finding: ChromaDB offers the fastest path to working vector search with zero infrastructure:

• Setup: Embeds in application, no server required
• Scale: Handles up to 10M vectors efficiently
• API: Python-native, NumPy-like interface

Minimal implementation (callable via Claude Code Bash):

#!/usr/bin/env python3
# minimal_rag.py - Invoke via: python3 minimal_rag.py "query text"
import sys
import chromadb
from chromadb.utils import embedding_functions
 
# Initialize ChromaDB (in-memory for simplicity)
client = chromadb.Client()
collection = client.get_or_create_collection(
    name="markdown_docs",
    embedding_function=embedding_functions.DefaultEmbeddingFunction()
)
 
# Load markdown files
def load_markdown_files(directory):
    documents = []
    for file in Path(directory).glob("*.md"):
        with open(file) as f:
            documents.append({
                "id": file.name,
                "text": f.read(),
                "metadata": {"source": str(file)}
            })
    return documents
 
# Index documents
docs = load_markdown_files("./markdown/")
collection.add(
    documents=[d["text"] for d in docs],
    ids=[d["id"] for d in docs],
    metadatas=[d["metadata"] for d in docs]
)
 
# Query
query = sys.argv[1]
results = collection.query(query_texts=[query], n_results=3)
print(results["documents"][0])

Claude Code usage:

python3 minimal_rag.py "What are the authentication patterns?" | head -n 50

Portability: This script structure works identically in LangGraph tool nodes (see Section 5).

3. LangChain markdown support

Moderate confidence (2 sources: LangChain docs (link no longer available), markdown-langchain-rag repo)

Key finding: LangChain provides UnstructuredMarkdownLoader with two modes:

• Single mode: Returns entire document as one Document object
• Elements mode: Splits into semantic chunks (Title, NarrativeText, etc.)

Minimal setup:

from langchain_community.document_loaders import DirectoryLoader, UnstructuredMarkdownLoader
 
# Load all markdown files
loader = DirectoryLoader(
    '../markdown/',
    glob='**/*.md',
    loader_cls=UnstructuredMarkdownLoader
)
documents = loader.load()
 
# Use with vector store
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings
 
vectorstore = Chroma.from_documents(
    documents=documents,
    embedding=OpenAIEmbeddings()
)
 
# Query
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
relevant_docs = retriever.get_relevant_documents("query text")

Trade-offs:

• Pro: Structured splitting (preserves markdown semantics)
• Pro: Integrates with the tools around LangChain
• Con: Requires unstructured dependency (heavy install)
• Con: More complex than LlamaIndex for simple cases

4. LlamaIndex markdown support

High confidence (3+ sources: SimpleDirectoryReader docs, Real Python tutorial, Medium tutorial)

Key finding: LlamaIndex's SimpleDirectoryReader provides the simplest interface for markdown RAG:

Minimal implementation:

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
 
# Load markdown files (auto-detects format)
documents = SimpleDirectoryReader(
    input_dir="./markdown/",
    required_exts=[".md"]
).load_data()
 
# Create index and query
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("What authentication patterns exist?")
print(response)

Advantages:

• Simplest API (3 lines to working RAG)
• Auto-detects file types
• Built-in persistence: index.storage_context.persist()
• Extensible (custom readers, node parsers)

When to use: Quick prototypes, learning RAG, minimal dependencies

5. LangGraph portability patterns

Moderate confidence (2 sources: LangGraph RAG guide, Corrective RAG tutorial)

Key finding: Code portability between simple scripts and LangGraph workflows requires tool-based abstraction:

Portable pattern:

# 1. Define retrieval as a tool (works in script OR LangGraph)
def retrieve_documents(query: str, top_k: int = 3) -> list[str]:
    """Retrieve relevant markdown documents."""
    # Can use Grep, ChromaDB, or LlamaIndex
    results = vectorstore.similarity_search(query, k=top_k)
    return [doc.page_content for doc in results]
 
# 2. Use in simple script
docs = retrieve_documents("authentication patterns")
print(docs)
 
# 3. Use in LangGraph node
from langgraph.graph import StateGraph
 
def retrieval_node(state):
    query = state["query"]
    state["documents"] = retrieve_documents(query)
    return state
 
graph = StateGraph()
graph.add_node("retrieve", retrieval_node)

Portability checklist:

• Stateless functions (no hidden dependencies)
• Dict-based state (LangGraph requirement)
• Tool-style interfaces (function signatures)
• Separate data loading from graph logic

Approach comparison

Recommendation matrix:

• 400KB, 12 files (the case studied): Start with Grep + Read (native, fast, sufficient)
• Need semantic search: Add ChromaDB script (minimal, portable)
• Rapid prototyping: Use LlamaIndex (simplest API)
• Production LangGraph app: Use LangChain (the widest set of integrations)

Recommended implementation

Phase 1: Claude Code native (start here)

Use Grep for retrieval, Read for context expansion:

# 1. Find relevant chapters
grep -l "authentication\|auth pattern" markdown/*.md
 
# 2. Get context (10 lines before/after)
grep -C 10 "authentication" markdown/chapter-03.md
 
# 3. Read full chapter if relevant
# Use Claude Code Read tool on identified files

Advantages:

• Zero setup (native tools)
• 0.4s retrieval
• Perfect for 400KB corpus

Limitations:

• No semantic understanding
• Manual context management

Phase 2: add vector search (when needed)

Create minimal_rag.py (callable via Bash):

#!/usr/bin/env python3
"""Minimal Markdown RAG - Callable from Claude Code via Bash"""
import sys
from pathlib import Path
import chromadb
 
def main():
    query = sys.argv[1] if len(sys.argv) > 1 else "default query"
 
    # Initialize ChromaDB
    client = chromadb.Client()
    collection = client.get_or_create_collection("md_docs")
 
    # Load markdown if collection empty
    if collection.count() == 0:
        for md_file in Path("./markdown/").glob("*.md"):
            with open(md_file) as f:
                collection.add(
                    documents=[f.read()],
                    ids=[md_file.name],
                    metadatas=[{"source": str(md_file)}]
                )
 
    # Query and return top 3
    results = collection.query(query_texts=[query], n_results=3)
    for doc, metadata in zip(results["documents"][0], results["metadatas"][0]):
        print(f"\n=== {metadata['source']} ===")
        print(doc[:500])  # First 500 chars
 
if __name__ == "__main__":
    main()

Usage:

# From Claude Code
python3 minimal_rag.py "explain authentication patterns"

Phase 3: LangGraph in production (future)

Port to LangGraph workflow:

from langgraph.graph import StateGraph
from typing import TypedDict
 
class RAGState(TypedDict):
    query: str
    documents: list[str]
    answer: str
 
def retrieve_node(state: RAGState) -> RAGState:
    # Use same retrieval logic from Phase 2
    results = collection.query(query_texts=[state["query"]], n_results=3)
    state["documents"] = results["documents"][0]
    return state
 
def generate_node(state: RAGState) -> RAGState:
    # Use Claude to generate answer from documents
    context = "\n\n".join(state["documents"])
    state["answer"] = llm.generate(f"Context: {context}\n\nQuestion: {state['query']}")
    return state
 
# Build graph
graph = StateGraph(RAGState)
graph.add_node("retrieve", retrieve_node)
graph.add_node("generate", generate_node)
graph.add_edge("retrieve", "generate")
graph.set_entry_point("retrieve")
 
app = graph.compile()
result = app.invoke({"query": "authentication patterns"})

LangGraph portability notes

Key portability principles:

1. Stateless retrieval functions

   • Keep retrieval logic in pure functions
   • No hidden state/singletons
   • Same function works in script or LangGraph node

2. Dict-based state

   • LangGraph requires TypedDict state
   • Design simple scripts with dict state from start
   • Easy migration path

3. Tool abstraction layer

   • Wrap Grep/Read/Vector calls in tool functions
   • LangGraph can invoke tools via ToolNode
   • Claude Code calls same tools via Bash

4. Separation of concerns

   • Data loading: standalone module
   • Retrieval: stateless function
   • Generation: separate node
   • Graph: orchestration only

Migration checklist:

• Retrieval function returns dict
• No global state dependencies
• Function signatures match tool spec
• State schema defined (TypedDict)

Evidence gaps

1. Hybrid grep and vector performance

   • Gap: No benchmarks on combining Grep (precise) with Vector (semantic)
   • Searches attempted: "hybrid grep vector RAG", "lexical semantic retrieval combination"
   • Why insufficient: Most papers focus on one approach, not hybrid workflows
   • Impact: Low (can test empirically on 400KB corpus)

2. Markdown structure preservation

   • Gap: Limited data on whether UnstructuredMarkdownLoader preserves links/images
   • Searches attempted: "LangChain markdown metadata preservation"
   • Why insufficient: Documentation focuses on text splitting, not metadata
   • Impact: Medium (important for chapter cross-references)

Methodology

Search strategy: 5 targeted searches covering: LangChain loaders, LlamaIndex SimpleDirectoryReader, grep-based retrieval, LangGraph RAG patterns, minimal vector databases (ChromaDB/FAISS).

Sources analysed: 15 sources across official docs (LangChain, LlamaIndex, LangGraph), research papers (GrepRAG arxiv), tutorial repos (markdown-langchain-rag), and comparison guides (vector database comparisons).

Confidence distribution:

• High confidence: 9 findings (Grep performance, LlamaIndex API, ChromaDB characteristics)
• Moderate confidence: 4 findings (LangChain patterns, LangGraph portability)
• Low confidence: 0 findings
• Insufficient evidence: 2 gaps (hybrid performance, metadata preservation)

Quality checks:

• CRAAP test applied (13 of 15 sources scored 4.0 or above; two community tutorials scored 3.8 and 3.9)
• Triangulation (3+ sources on core claims)
• Contradictions documented (Grep critics vs proponents)
• Code examples verified from official docs

Overall confidence: High confidence in core recommendations, moderate confidence in LangGraph migration patterns.

Sources

Official documentation

1. LlamaIndex SimpleDirectoryReader - CRAAP 4.8
2. LangChain Markdown Loader (link no longer available) - CRAAP 4.7
3. LangGraph RAG Guide - CRAAP 4.6

Research and analysis

4. GrepRAG arxiv Paper - CRAAP 4.9
5. RAG Beyond Semantic Search: Grep - CRAAP 4.2
6. Claude Code Grep Critique - CRAAP 4.0

Implementation examples

7. markdown-langchain-rag GitHub - CRAAP 4.1
8. Real Python LlamaIndex Tutorial - CRAAP 4.5
9. Corrective RAG with LangGraph - CRAAP 4.3

Vector database comparisons

10. ChromaDB vs FAISS Comprehensive Guide - CRAAP 4.0
11. Open-Source Vector Databases 2024 - CRAAP 4.4
12. Best Vector Databases 2025 - CRAAP 4.2

Community resources

13. LangChain Document Loaders - CRAAP 3.8
14. Basic RAG with LlamaIndex - CRAAP 3.9
15. Agentic RAG with LangGraph - CRAAP 4.1