June 4, 2025
HippoRAG is a way to create a "map" of a corpus of documents with interconnected entities and concepts. See my summary of the paper in the ML papers page. It's an improvement over GraphRAG because you can more easily create / update / delete documents. It introduces a context / concept tradeoff though that I thought was interesting.
Let's handle the document updating first. What if you need to update a document? You would need to remove the document and insert the new one but how do you remove a document? You can remove all KG nodes with passages that come from that document but what if there's a node with a chunk from the document and other chunks as well? Well you keep it then, but which outgoing links should be removed? Is it safe to keep all the links except the ones going to nodes that will be deleted? What if some of the links that will be kept were originally added by the document to be deleted?
Say we add document A which causes us to create nodes and links (x)->[a]->(y) (among others). Then we add node B which causes us to add (x)->[b]->(y) so now that link is (x)->[a,b]->(y). Now we delete document A. Both nodes x and y have chunks from both documents. We delete the chunks from document A but both nodes remain since they're safe because they have chunks from B. The problem is that you still have link a between those two nodes. That link won't be removed even though it's out of date. To fix this, we would have to store document IDs on each link as well. (Not so bad I guess.)
There's also a significant problem with missing context. Say I have two documents which are both rental car confirmation vouchers. The nodes they'll be saved under will be things like "rental car", "car", "Enterprise", "Boston", "XYZ Street", etc. They might even contain entities like dates and car models. The problem is that if I ask "what's the confirmation number for my latest car rental", they'll all match. The car node itself will contain all of them, but then it will fan out to nearby nodes which will (a) certainly contain all of them and (b) contain lots of random other stuff too. Even if the date were a node in the graph and was connected to my document, the other documents would come in too.
But worse, it wouldn't be the full document, just the chunks. So I'd get a chunk from a reservation years ago saying "your confirmation number is 123" and another chunk saying 456. The final LLM would lose the context of the full documents, even if it only got the car rental documents and weren't confused by everything else related to cars and Boston. In fact sometimes it gets even worse than that because important context can sometimes be found outside the document, like if the user says "I'm going to send you my sister's rental car voucher.". How is the LLM supposed to know that that one is the sister's and not mine? Or "Here's the car rental voucher but the date is printed wrong, it should be XXX.".
This brings us to the core of the context / concept tradeoff. The knowledge graph in HippoRAG excels at abstracting information into a network of reusable concepts. Nodes like "rental car" or "Nick Hoff" become hubs, connecting disparate facts and enabling the multi-hop path-finding retrieval that makes the system powerful. This is the "concept" side of the tradeoff. The cost of this abstraction however is the dissolution of the original document's context. A single document, like a car rental confirmation, is a self-contained unit of information where the date, price, and confirmation number are intrinsically linked. By shredding this document into conceptual nodes and edges, the graph loses the boundary of that original unit. When you query for your rental car you're not retrieving a document, you're activating the "rental car" concept node which pulls in related chunks from every rental car document ever indexed. This means the choice of representation depends entirely on the task. For synthesizing knowledge across a corpus, the conceptual graph is superior. For tasks that rely on the integrity of a single information source, the graph can be a liability, mixing contexts and creating confusion.
Actually, all of these graph RAG ideas seem like a fancier version of semantic vector similarity. Yes, it encodes relationships between things that aren't semantically related (like artists and the songs they publish), but the search is still essentially semantic. The search results include the linked items but not their context. In fact the context is not stored on the graph anywhere and sometimes it's not even recorded. This is not a problem with HippoRAG but it is a problem for using something like HippoRAG for Second Brain.