My response to François Chollet’s “The Impossibility of Intelligence Explosion”
Francois Chollet argues in his recent essay that an intelligence explosion is very unlikely. So the fast progress we see today is a chimaera, more linear than we think and more likely to slow down, because:
Doing science in a given field gets exponentially harder over time — the founders of the field reap most the low-hanging fruit, and achieving comparable impact later requires exponentially more effort.
And that even the open-source networked approach to research that has driven so much recent progress has limits because:
Sharing and cooperation between researchers gets exponentially more difficult as a field grows larger.
Essentially, these manifest themselves as contingent bottlenecks and diminishing returns.
I’m very sympathetic to Francois’s arguments, but I do have a couple of points of departure. First: We have already seen examples where we have seen exponential-style growth. These includes human systems which are dependent on the complexity of interactions between people. We have already seen a rise in complexity which, when mapped against linear time, shows an exponential hockey-stick graph. Whether that is hominid cranial capacity (inflection point about 2m years ago), or world GDP (looks very linear on a 1950–2016 scale, clearly inflecting on the scale of AD1 to today with the kink in the chart coming around the industrial revolution), or even human population (which is still in its hockey-stick growth phase).
Many of these things are logistic (or S-curves) reaching a point of saturation, or diminishing returns, so they flatten out. Human cranial capacity is unlikely to treble in the next two million years, although that is what it did in the previous two million. It better not, otherwise our cranial capacity would reach 4,500cm^3 and our skulls would be 45% bigger in every direction.
But Ray Kurzweil’s description of accelerating returns is that exponential relationships are often the layering of S-curves from multiple subsequent technical paradigms that gives us our familiar exponential curve. In fact, you only need to look at this graph from Nvidia’s investor day.
As the improvements we can push out of CPU architectures peters out, we exploit a new architecture which is accelerating at the steep point of its S-curve. In this case the GPU & CUDA pairing. Over the next 10 years, CPUs might only improve 2–3-fold, but GPUs will increase 60-fold to 1000 times the performance of their CPU cousins. That feels better than linear…
Second, it seems like the open-source community around AI may be structured in a way that fosters long-term innovation rather than atrophy. In his wonderful book, Scale, complexity scientist Geoff West investigates complex social systems, and relevant here, cities and companies. Both analyses form a useful lens to look at one of Francois’s other observations: the declining returns from collaboration as things get complex. Geoff points out that empires and companies do often collapse under their own weight as they get larger. The half-life of firms in the S&P 500 is about 10 years, a result of the stagnation of goals and increasingly bureaucratic control needed to oversee execution. (See the UK edition, page 408–410).
But equally, Geoff makes the case that cities have very different characteristics as they get bigger and more complex. They are prototypically multidimensional, enjoy superlinear scaling resulting in “open-ended growth, … expanding social networks…, resilience, sustainability, and seemingly immortal.”
My sense is the multi-disciplinary, and increasingly inter-disciplinary, praxis of AI research in these open source communities has more in common with cities (which survive the rise and fall of the empires in which they are contained) than companies (which, per Geoff, seem to collapse under the weight of their own directionless bureaucracy.)
If that is the case, then perhaps those dynamics are not really brakes on the path to an intelligence explosion.
If you enjoyed reading this, you’re going to love my weekly newsletter Exponential View