The Undiscovered Public Knowledge Method
Connect distant islands of specialist knowledge that have never met
Information scientist Don Swanson identified a powerful pattern: as academic specialization accelerates, bodies of knowledge that are directly relevant to each other become siloed in separate sub-disciplines that never cite one another. The connections between these silos constitute 'undiscovered public knowledge,' discoveries waiting to be made by anyone willing to bridge the gap.
Swanson demonstrated the concept by cross-referencing medical literature databases from different specialties. He found eleven connections between magnesium deficiency research and migraine research that had never been made, even though both literatures were publicly available. He proposed that magnesium could treat migraines, and decades later the American Headache Society confirmed it. All the information existed; it was simply in silos that never communicated.
As knowledge fragments further, the opportunities for this kind of synthesis multiply. The more specialized the world becomes, the more valuable the generalist connector becomes. InnoCentive found that the further a problem solver's expertise was from the problem domain, the more likely they were to solve it, because they brought entirely different frameworks that the specialists had not considered.
- As specialization increases, the volume of publicly available but unconnected knowledge grows exponentially
- Major discoveries can come from connecting existing knowledge rather than creating new knowledge
- The most valuable connections often span the greatest disciplinary distances
- Outsiders succeed where specialists fail precisely because they bring different frameworks to old problems
- The barrier to discovery is often not lack of knowledge but lack of connections between existing knowledge
- Identify two or more specialist domains relevant to your questionLook for domains that are conceptually related but practically separate. They should be studying different aspects of the same underlying system but never citing each other.Pro tipCasadevall noted that immunology specialists studying B cells and those studying macrophages work in parallel trenches and rarely look at each other's work, even though B cells and macrophages interact constantly in the immune system.
- Cross-reference their literaturesSystematically look for concepts, findings, or mechanisms in one domain that have direct but unrecognized relevance to the other. Use search tools that span disciplines rather than staying within one.Pro tipSwanson's Arrowsmith system automated this by identifying articles from different databases that shared key terms but never appeared together in citation networks.
- Propose and test the connectionsOnce you identify a potential connection, formulate it as a testable hypothesis. The connection between magnesium and migraines was proposed by Swanson through literature synthesis and later confirmed through clinical research.Pro tipYou do not need to be an expert in either domain to make the connection. Jill Viles was a substitute teacher who connected her own genetic condition to an Olympic sprinter's physique, leading to medical discoveries that eluded specialists.WarningYour proposed connections need to be tested rigorously. The synthesis stage is for hypothesis generation, not conclusion.
- Build bridges, not wallsShare your findings with specialists in both domains. The connection you have made may be obvious once pointed out but invisible before. Be prepared for initial resistance from specialists who may feel that an outsider is overstepping.Pro tipFrench biologist Etienne Lefai changed the direction of his entire research team after Jill Viles contacted him with a connection he had never considered. He said, 'I had no idea of what I can do with genetic diseases before she contacted me.'
Don Swanson cross-referenced databases of medical literature from different specialties and found eleven connections between magnesium deficiency research and migraine research. No scientist in either field had ever cited the other, even though the connections were directly relevant.
Jill Viles, a substitute teacher with a rare genetic condition, noticed that Olympic sprinter Priscilla Lopes-Schliep had the same pattern of missing limb fat visible in Jill's own body. She hypothesized they shared the same gene mutation. Specialists dismissed the idea.
Don Swanson was a librarian and information scientist who worried that increasing specialization would make sub-disciplines like galaxies flying apart until each was invisible to every other. In 1960, the U.S. National Library of Medicine used about one hundred unique index term pairs; by 2010, it was nearing one hundred thousand. Swanson created a computer system called Arrowsmith to help researchers find connections between specialist literatures that never overlapped. His magnesium-migraine connection was the proof of concept, and it ignited a field of information science dedicated to connecting fragmented knowledge.