Some answers were never written down in one place. Ask a research archive which contract labs ran assays for compounds that later failed on liver toxicity, and a retrieval system built on chunks hands back five passages. One mentions a lab, two mention assays, two mention liver toxicity.
None of them holds the answer. It sits in the links between a lab, the compounds that lab touched, and what happened to those compounds two reports later.
That’s the case for a graph index. Since Microsoft open-sourced GraphRAG in July, I’ve seen it treated as the next version of retrieval, the thing you upgrade to. I’d describe it more narrowly: a second index that answers two kinds of question well and is priced like a small project.
Chunks answer one kind of question
I sort the questions people ask a document system into three kinds.
A lookup has its answer in one or two passages. “What was the Phase 1 dropout rate for ST-204?” Hybrid search with a reranker on top handles these, and when it doesn’t, the cause is usually how the documents were chunked.
A multi-hop question connects things that live in different documents, like the lab question above. Top-k retrieval finds one end of the chain and stops there.
A corpus-wide question asks about everything at once. “What are the recurring reasons our formulation studies get delayed?” The ten chunks nearest to that sentence are a sample of delays. The question wanted a tally.
A graph index is built for the second and third kinds. At indexing time a model reads every chunk and pulls out entities and the relations between them. Those merge into a graph, the graph gets clustered into communities, and a model writes a summary of each community. Multi-hop questions walk the edges. Corpus-wide questions get answered from the summaries. A classifier in front decides which index a question goes to, and lookups keep going to the vector index.
Sort the questions before you price anything
Pull a couple hundred real questions from your logs and put them in a sheet like this:
id,question,kind,vector_correct,graph_correct,notes
q017,"What was the Phase 1 dropout rate for ST-204?",lookup,yes,,one report holds it
q052,"Which contract labs ran assays for compounds that later failed on liver toxicity?",multi_hop,no,yes,lab to compound to outcome
q131,"What are the recurring reasons our formulation studies get delayed?",corpus_wide,no,yes,no single passage holds it
Two rules keep the sheet honest. A person labels the kind before anyone runs anything, so the label can’t drift toward the result. And the vector column has to come from your best chunk setup, with hybrid search and reranking switched on. Otherwise the graph gets credit for fixing retrieval you hadn’t finished.
The graph column needs a graph, which is awkward when the point is to decide whether to build one. Index a slice first, one product line or one year of documents, and score the questions that belong to it.
To show what the tally looks like, I’ve invented a company. Stannard Biotech has 12,000 research reports, split into 180,000 chunks, and 200 questions its analysts typed into the existing search.
| Kind | Questions | Correct with vectors | Correct with graph |
|---|---|---|---|
| Lookup | 141 | 124 (88%) | not routed |
| Multi-hop | 38 | 13 (34%) | 27 (71%) |
| Corpus-wide | 21 | 2 (10%) | 14 (67%) |
Vectors alone get 139 of 200 right. Routing the two hard kinds to the graph gets 165. That’s 26 more answers, all of them from the 59 questions that weren’t lookups.
What the extraction bill looks like
Embedding is nearly free. 180,000 chunks at 600 tokens each is 108 million tokens, about $2 at two cents per million.
Extraction is where the money goes, because a chat model reads every chunk at least once, and usually twice: one pass to extract, one to ask what it missed. The prompt with its examples is bigger than the chunk.
def index_cost(chunks, in_tok, out_tok,
communities, sum_in_tok, sum_out_tok,
in_price, out_price, batch=False):
"""Prices are dollars per million tokens. in_tok and out_tok are per
chunk, all extraction passes added up. sum_* are per community summary."""
per_m = 1_000_000
extraction = chunks * (in_tok * in_price + out_tok * out_price) / per_m
summaries = communities * (sum_in_tok * in_price + sum_out_tok * out_price) / per_m
total = extraction + summaries
return total / 2 if batch else total
# Stannard, illustrative prices: $2.50 in, $10.00 out per million tokens
print(index_cost(180_000, 5_000, 1_500, 15_000, 6_000, 1_500, 2.50, 10.00))
# 5400.0
About $5,400, of which $4,950 is extraction and $450 is summaries. The prices are illustrative, so put in your own. Indexing isn’t urgent, which makes it a good fit for OpenAI’s Batch API: half price if you can wait up to a day for results. That brings Stannard to $2,700, still over a thousand times the embedding bill.
Changed documents are the cost that keeps coming
Suppose 6 percent of Stannard’s chunks change in a month. That’s 10,800 chunks to re-extract, roughly $300 at the same rates. The money is the easy part.
A changed chunk can add an entity, drop a relation, or shift which community a node belongs to. Each of those leaves a summary describing a graph that no longer exists. You can’t recluster on every edit, and you can’t ignore edits either, so write the policy down before the first build:
rebuild_policy:
detect:
key: chunk_content_hash
unchanged: skip
on_changed_chunk:
- delete entities and relations sourced only from the old chunk
- re-extract the new chunk
- mark every community it touches as stale
stale_summaries:
refresh: nightly
use: batch
serve_meanwhile: old summary, flagged with its build date
full_recluster:
when_any:
- changed_chunks_since_last_recluster_pct >= 15
- days_since_last_recluster >= 90
prompt_or_model_change:
action: full re-extraction
budget: same as the first build
The last entry is the expensive one. Changing the extraction prompt changes what counts as an entity, and old and new extractions don’t mix. Every prompt improvement costs another first build.
Where I’d draw the line for Stannard
I’d build it. Thirty percent of their questions are the kinds a graph is for, and those are the questions an analyst otherwise answers by reading reports for an afternoon.
I’d also fix the 17 lookups that failed first, because that work is cheaper and it helps the larger group. My line is about one question in ten. Under that, the hard ones can go to a person, or to a model with a long context window and the right twenty reports.