This was co-written with a brilliant theoretical physicist, with input from various others.
The scientific community should turn black swans white and make discoveries that are unimaginable today: it is currently failing to do either
Reality surprises us by revealing ‘black swans’ –namely, ideas once near unthinkable turn out to be true. This reinforces how we are in fact deeply ignorant of the future growth of knowledge in all fields. Much of scientific progress is driven by such large-influence, unpredictable long-tail phenomena that dislodge dogma. Science has historically had great influence by finding beneficial black swans, such as antibiotics and the theory of evolution, whilst protecting against potential future negative black swans such as the sudden prospect of the development of nuclear weapons by axis powers in WW2. Note also that often black swans are found while looking to solve fundamental problems, with no immediate application to technology that can be commercialised. A well-known example is Einstein’s general theory of relativity, whose discovery led to great technological advances (such as the Global Positioning System), but was developed by Einstein ‘just’ for the sake of deepening the understanding of physical reality. Such results are a key justification for investment in transformative, exploratory and fundamental research, but they depend on the scientific community’s structure and incentives.
Today’s scientific community is structured to make black swan-like paradigm shifts less likely. Rather than encouraging diversity, which is key in a healthy intellectual landscape, it enforces consensus. Rather than encouraging the pursuit of risky, but potentially ground-breaking, ideas, it rewards predictable production. We are concerned that well-intentioned management practices in the research system in the past decades have sharply reduced the ability of science to find and deal with these black swans, producing a globally homogenized scientific community in which deviation from the consensus view is deeply harmful for one’s career whilst conformity and political maneuvering are rewarded. Productivity by entertaining the tastes of current peers is strongly valued over long-term excellence. In place of science, nowadays we have a collective endeavour that does not fulfill its historic role of thinking the unthinkable and turning it to our benefit.
Historic innovative structures permitted iconoclasm, ‘ill-shapen’ newborns, and cognitive diversity
The history of research and innovation is full of misfits, rebels, eccentrics, and iconoclasts: people and start-up style teams convinced that the mainstream was wrong and determined to prove it. Prior innovative societies prized these groups, shielding them from the false consensus and allowing their embryonic ideas–—like ‘ill-shapen’ newborns as Francis Bacon called them–to be nurtured and explored.
A quick glance at the origins of Western Science reveals how paradigm shifts were enabled when society allowed their iconoclasts to flourish. The origins of logic may have come from the relative isolation of the Greek polis: each classical Greek natural philosopher could freely pursue their curiosity knowing that if they offended their community they could take refuge in another. As a consequence, disruptive political sciences, the disruptive mathematics of irrational numbers and then controversial forms of logic were developed. In the Renaissance period, which tried to recapture the intellectual energy of this earlier period, the Medici shielded Galileo who could, without fear of the papacy, overturn Aristotelian mechanics and turn his gaze to his Heavens. And the young Newton in the nascent United Kingdom was protected by special exemption from punishment for his anti-trinitarianism, a conviction punishable, in theory, by death. Newton the misanthrope also famously sought isolation from the networks of science, instead living a near monastic life, continuing only because his mentor Isaac Barrow gave his Professorship to Newton to keep him in Trinity College.
In the twentieth century, productivity in modern science was similarly fostered by enabling iconoclastic opinion. Einstein in his worldly cloister, the patent office in Switzerland that was his intellectual home when no other would take him, wrote four of the fundamental papers in history. Bell Laboratories offered a refuge for teams and academics who wanted to prevent their science from being sculpted by short-term bureaucratic demands of University departments. Oxford’s Clarendon laboratory offered to a bunch of weird physicists and computer scientists, led by David Deutsch, the necessary space to pioneer quantum computing — then considered a heretical idea both in computer science and physics, now a pillar of the future IT technology. And the Cambridge University Laboratory of Molecular Biology (LMB) created a collaborative culture of creativity that catalysed the discoveries of what we now know as the central dogmas of biology: the molecular basis of inheritance. Brenner described those outside the original LMB as the ‘heathens’ – a joke at how the LMB was seen as cult-like and eccentric in its early days. These locations were not simply racing to do what was extramurally fashionable to get funding but instead were able to create meaningful innovation by thinking differently. By being liberated to think differently, they could hunt for Black Swans and introduce innovations that other parts of their contemporary scientific communities could not.
The most pressing and fundamental question of today’s scientific community is arguably not simply the specific questions about how the natural world works, but rather how to encourage the scientific community so that it can, as it has in past periods of history, collectively bring about the unpredicted by exploring ideas that conventional wisdom regards as too risky, impossible, or plain wrong.
Today’s bureaucratic science optimises the median and promotes herding
In the post war period, structures such as Bell labs, the early LMB, and ARPA allowed individual judgement to select promising, iconoclastic ideas and teams, then nurturing them – embracing the failures that sometimes this exploration may bring, as a necessary overhead for the high reward that some of these ideas do eventually deliver. Yet contemporary systemic pressures push scientists in the opposite direction, sharply rewarding conformity, pleasing peers’ conventional wisdom and blocking team efforts.
The most constraining factor is the bureaucratisation and central management of science through committees and project-based applications, rather than allowing individual judgement to allocate resources with local, specific knowledge. This has turned science into a bureaucratic ‘factory’ focussed on individual outputs such as citation counts, not system-wide outcomes, and predictable deliverables. It has created an entrenched, stagnating monoculture. This is no surprise. A system that incentivises metrics will eventually produce a potemkin village-style academy where people compete to create the appearance of progress, devoting their creativity not to making genuine progress, but to a diligent box-ticking exercise – we note that only 14% of respondents to the Wellcome Trust 2020 survey felt that modern research’s reliance on metrics had improved science. In such a system it is far better to flock to the herd, where you will get more citations and better peer reviews, than it is to be the unpopular lone dissenter critiquing the endeavour. Further, the inherent risk-aversion of bureaucratic hierarchies concentrates power in the past, the previous generation, rather than taking ‘risky’ bets on the young who historically drive science. We appear to have forgotten the obvious point that risk involves large amounts of failure. This means that collectively the research endeavour explores only narrow parts of the scientific unknown, located in a small neighbourhood of what’s already known, reinforcing faddish trends that appear ‘impactful’ but in reality are often the collective delusions that form so much of science history. To be a dissenter is to aggravate the powerful and not get funded as a consequence.
A second factor is the increased individualisation and pyramidisation of science. This has occurred through the overtraining, by factors that may exceed 20:1, of trainees relative to jobs. This has been concurrent with the growth of lab sizes in biology, and a failure to modify career structures to account for the increasing technological dependency of much of science where no one person can master everything. Rather than nurturing the young, senior individuals are now strongly incentivised to build empires based largely on taking credit for the work of others and boost metrics to get funding – a vicarious science. Though some bad actors exist, it is mostly the only way that good people can now thrive. This shocking situation of exploitation is now culturally normalised across much of science. Further, more generally individual productivity is rewarded in the short-term to ascend this steep hierarchy, and when we reward being productive over being correct or creative in the long-term, phenomena such as empire building, taking credit for others’ work, exploitative practices, poor quality control, misleading analysis and other such problems rise. This system has now been in place for decades with myriad implications for the overall composition of our international science leadership.
Relatedly, the steep hierarchy and poor job security also means that can afford to take a risk as the price of failure is individually too high. We note that the business world is aware of the need for competitive displacement and the ability for ‘startups’ to challenge the consensus: ossified academic hierarchies instead actively block this phenomena by concentrating research resources and power in the past. In the vast majority of cases, the young cannot be equal partners with the old, let alone displace them. Science must therefore proceed ‘one funeral at a time’…. Government should actively oppose, not strongly encourage, this natural phenomena, as they do with antitrust regulation, seeking to give power to the young and outsiders questioning group think. Government must be concierge’s of a healthy scientific community, not bureaucratic soviet-style central managers expecting Potemkin reports on a good harvest.
Fourth, since WW2 brought almost all government research under Universities, that career structure has become strongly dominant globally. It has strengths, especially for teaching, yet it also inherently lacks dynamism, freshness, and crucially the ability to form teams. Even in the 1950s, JCR Licklider struggled to assemble the team within MIT that would birth ideas leading to ARPA-IPTO and xerox PARC: the University career structure could not facilitate the formation of a lasting team. It ingrains an individualistic hierarchy, a belief in the soloist. We must instead diversify career structures.
A fifth factor is the global homogenisation of science. Science is fundamentally international and this is a great strength that we must encourage. But this is different to the totally connected, immediate information exchange we have now, where geography no longer provides unique perspectives as they did in the examples above and instead ‘global opinion’ strongly shapes local funding decisions. Cajal, a key founder of modern neuroscience, could not read many of his competitor’s works, and his own were unread also, which gave him an ‘isolation’ and somewhat unique do-it-yourself perspective. Diversity and autonomy at every level is collapsed by the extensive interdependencies of the system. A broken global publishing system is an example of this phenomena, where no one country feels able to make the ‘first move’ despite widespread agreement that the tournament to get into for-profit magazines like Nature has greatly damaged science. To be open and connected to the world does not require being totally locked into it.
The chronic tendency of bureaucracy to meddle and monitor is also opposed to the requirements of long-term, patient science. Accountability and evaluation must be done at a higher level – is this the system as a whole working? This requires a shift to a ‘portfolio’ mindset.
Next: revive the underlying structures of innovative scientific communities
Governments currently seek to use bureaucracy to improve research. Yet the vast bureaucratic enterprises of modern science were notably absent during the great scientific revolutions of the 20th century–we should study the organisational structures that created them, rather than create what Douglas Adams called ‘small green pieces of paper’ to try to ‘solve’ ‘problems’. Rather than only adding another 0.1% investment to the global university system, we should in addition create something in parallel with different strengths and weaknesses to provide a globally unique scientific ecosystem. We will shortly be putting forward proposals for comment, which aim to: create structural diversity in the UK research ecosystem that can attract global talent fed up with the current system, level up the UK by creating a network of talent attractors, and together birth a truly 21st century economy.
- The Dream Machine – Mitchell Waldrop – about how teams built around a vision creating personal computing in the postwar period
- The Idea Factory – Jon Gertner – the best description of the dynamic,
- How Academia and Publishing are Destroying Scientific Innovation: A Conversation with Sydney Brenner
- My Life in Science – Sydney Brenner
- How? – Alan Kay for the MacArthur Foundation