Most people don’t realize the true driver of progress. If you ask 10 random people how civilizations progress or improve, chances are most will not mention innovation in their answers.
If you take a moment to reflect, you’ll realize that civilizations advance through new innovations. Imagine if James Watt had never invented the steam engine, Gutenberg had never created the printing press, Charles Babbage had not conceptualized the computer, or Vint Cerf had never thought of the internet. Each of these innovations has been a cornerstone of progress, shaping the world as we know it today.
All the comforts we enjoy today are thanks to someone’s inventions. A friend of mine often asks, “Why don’t we invent like we did in the 20th century or before?” I never had a clear answer for him.
While I don’t completely agree that inventions have dried up (there are certainly innovations in the 21st century), he is not entirely wrong to say that the pace of innovation has slowed.
Recently, I was reading a paper that strived to answer some of these tough questions.1 I found some of the arguments by authors insightful. At least the next time when I meet my friend, I could probably give him some rationale for his queries.
In 2020, a paper was published which concluded innovation indeed is getting harder to find across various industries.2 How did they come up with this conclusion?
They found-
“Research effort is rising substantially while research productivity is declining sharply.”
To achieve the same level of output as 50 years ago, we now require more than double the input. In medical science, more scientists are needed to save the same number of years of life. In agriculture, more scientists are needed to increase crop yields by the same proportion. This same trend holds across the fields. The bottom line is that more people are needed to innovate something over time.
But why is that? What has changed now?
The Burden of Knowledge
In 1676, Isaac Newton wrote famously to Robert Hooke
“If I have seen further it is by standing on the sholders of Giants”
Newton believed knowledge produces new knowledge. This is what we observe in practice. No baby is born as a scientist. Extensive education and training in the fundamentals are necessary before one can even think about innovation.
To stand on the shoulders of giants, one must first climb up there. The more knowledge there is, the harder or at least more time-consuming it becomes to reach that level.
If innovation requires a grasp of existing knowledge, as more problems are solved, we require additional knowledge to solve the newer problems. This creates a burden of knowledge on today’s innovators.
Today, it takes more than 30 years of age to become an independent scientist. Compare this with the fact that in the 20th century, some scientists got Nobel prizes around this age (Lawrence Bragg won the Nobel prize at 25 and Heisenberg at 31).
If people lived longer then the burden of knowledge is not a problem. It might take the first 30 years to learn the basics but if people could live and work for 100 years, they have plenty of time to innovate something exceptional.
But people die around 70 to 80 years of age and don’t work actively after 60 years. That means an independent researcher today hardly gets 15-20 years to innovate something meaningful (and innovation takes time most often).
If you observe, each generation requires longer training to become independent scientists. This is evident in the extended PhD and postdoctoral training we see today. As existing knowledge increases, each successive generation undergoes more training. This leaves less time for innovation, ultimately slowing down the cycle of progress.
The Age of Innovation for Individuals
Just to take the previous point forward, what’s the age when most researchers do their notable work?
A paper published by Jones demonstrated the age when Nobel prize winners and great inventors did their notable work. Over the last century, it rose by 5 more years and keeps on increasing.
as a Function of Age from Jones (2010)
You can see this phenomenon almost everywhere. Today, most people publish their first-authored papers between the ages of 25 and 30. A century ago, this age was between 20 and 25. Even if someone is exceptional in a subject, they often can’t turn their ideas into reality until they become independent, usually around 30 to 35 years old. Additionally, it takes time to establish oneself in the scientific community.
On one hand, researchers begin to execute their ideas in their 30s and then also not living longer (hence not working longer), altogether diminishing the potential to innovate something substantial.
The Rise of Specialization
One way researchers have tried to overcome the burden of knowledge is by dividing into teams. If individuals have to spend more time learning basics then working in teams might accelerate the cycle of innovation.
The rise of specialization is evident in the last 40-50 years. The number of authors in scientific patents and publications has increased significantly. We can again see this trend in Nobel prizes. Usually, 1-3 people are awarded the Nobel prize every year. When more than one people are awarded, that’s probably due to the contribution of multiple people to a discovery.
The figure below shows the average number of laureates per contribution over the years. With time more people have contributed to a single scientific discovery.
This makes sense when you consider the time it takes to learn a topic from the basics. By working in teams, scientists and innovators can reduce the time they need to spend learning. In fields like physics, chemistry, and medicine, specialization is increasingly valued over generalization. If you specialize in chemistry and then want to submit a patent in physics, chances are people would raise their eyebrows.
In history, only three people have won two noble prizes for scientific discoveries, Marie Curie (1903, 1906), John Bardeen (1956, 1972), Frederick Sanger (1958, 1980), and Barry Sharpless (2001, 2022). After 1980, only Sharpless has won two Nobel prizes. Keeping in mind the need for specializations in today’s age, it is immensely difficult (if not impossible) to work in two different research fields and invent something significant.
The Way Forward and My Viewpoint
Various research/surveys done over the last decade have unambiguously proven that ideas are indeed getting harder to find or implement. Solving new problems demands creating new knowledge but creating new knowledge requires understanding the existing knowledge (Which is increasing).
The burden of knowledge may not be as significant in the future. While it’s essential to understand the basics before inventing something, knowing everything about a subject might become less critical with the advancement of AI.
People can delegate a lot of detailed knowledge to AI as it continues to improve. You only need to master the fundamentals and then use AI to assist you on your innovation journey. AI can act as a research assistant for scientists, a role it is already beginning to fulfill to some extent.
The age of innovation should be relaxed at least for exceptional researchers. Why should a researcher start thinking about innovation in their 30s when he has shown enough promise to do so in his 20s?
Some universities do allow bright young researchers to start independent labs in their 20s (or late 20s), but that’s more of an exception than the norm. If Universities create better incentives for early career researchers, it would encourage more of them to showcase their potential from a younger age.
Generalists should be valued just like specialists. Both bring their strength in innovation. Breakthrough innovations are more likely to come when one can think about connecting unexpected ideas. If we only rely on specialists, there may not be anyone who can think about those connections.
Creating new knowledge is always hard. Fewer than 1% of people who ever lived on Earth have contributed to innovations that advance society. To foster progress, we need to offer more opportunities for young people to innovate and make those efforts rewarding. A paradigm shift in our civilization can only occur through a paradigm shift in how we approach innovation.
