Community detection
Graph clustering
Splitting a graph into groups of nodes that connect more densely to each other than to the rest, so each group can be treated as one topic.
In DocSlurpGlossary entry
On this page
A knowledge graph built from a few hundred documents is usually one big connected blob. A handful of entities (the company name, “FY2027”, “London office”) appear everywhere and link every topic to every other. Asking “which parts belong together?” with connected components returns one answer: all of it.
Community detection looks for groups of nodes with many edges inside the group and few edges leaving it. In a company corpus, that might give three communities: the earnings story (FY2027 revenue, operating margin, EBITDA guidance), the London launch (London office, $12M expansion budget, first-year bookings) and executive travel (Learjet 75, travel desk, charter agreement). Each one is a topic, and a topic can be summarized.
Common algorithms
- Label propagation: nodes repeatedly copy the most common label among their neighbors. Fast and simple, though results can depend on visiting order.
- Louvain and Leiden: greedily maximize modularity, a score comparing edges inside groups with what random wiring would give. Leiden fixes a Louvain flaw that can produce internally disconnected communities, and is what Microsoft’s GraphRAG uses.
Either can run again on the communities themselves, giving a hierarchy from fine topics to broad themes.
In DocSlurp
DocSlurp uses weighted label propagation in two levels: level 0 groups entities, level 1 groups level-0 communities by the edges between them (core/worker/src/graph/communities.ts). Communities with one member are dropped. Each community gets a summary, and summaries whose member set did not change are reused rather than rewritten.
The gotcha
Communities are a property of the graph, not of the truth. If entity extraction splits one real thing into two nodes, or merges two different things, the clusters faithfully reflect that error.
In DocSlurp
In DocSlurp
When the project graph is on, DocSlurp clusters it with weighted label propagation, not Leiden: level 0 groups entities, level 1 groups those communities, and singletons are dropped (core/worker/src/graph/communities.ts).