In What will GPT-2030 look like?, Jacob Steinhardt imagines what large pretrained ML systems might look like in 2030. He predicts that a hypothetical GPT2030 would:
- Be superhuman at specific tasks
- Work and think quickly
- Be run in many parallel copies
- Learn quickly in parallel
- Be trained on additional modalities
Thinking through the implications of any of these predictions is I think pretty interesting - but here I want to focus on getting a handle on 2-4.
As I understand it, 4 is basically a function of 2 and 3: systems will learn quickly in parallel in proportion with how fast they work/think/learn individually, and how many systems can be run in parallel. (Probably there are other technical details here; I’m going to ignore them for now.)
It makes intuitive sense to me that GPT2030 will be fast at thinking and fast at learning. But how fast is fast?
Jacob predicts that:
- GPT2030 will think/work 5x as fast as a human
- GPT2030 will be trained on enough compute to perform 1.8 million years of work at human speed
- It will be feasible to run at least 1 million copies of GPT2030 in parallel
On the back of this, he estimates that:
- GPT2030 will be able to perform 1.8 million years of work in 2.4 months
- GPT2030 will be able to do 2,500 years of learning in 1 day
For now, let’s just assume that these forecasts are sensible. How much learning is 2500 years? What would 1.8 million years of work in 2.4 months look like?
(An aside on why I’m interested in this: I want to have a sense of how ‘good’ AI might be in 2030/the future. 1.0e28 training FLOPs doesn’t viscerally mean anything to me. 1.8 million years of work starts to mean something - but I still don’t really grok what it would mean if you could fit that many years of work into 2.4 months.)
Assuming that people work 8 hours a day, 5 days a week, 50 weeks a year, and then taking the first estimates I found on the internet for how many people are in different groups, then in a single day:
- PhD students study for a bit more than 8000 years
- DOD employees work for close to 2000 years
- Amazon employees work for around 900 years
- Mathematicians work for just over 400 years
- AI researchers work for nearly 200 years
- Microsoft and Alphabet each work for a bit over 100 years
- Google DeepMind works for a year and a half
- OAI works for nearly 10 months
- Anthropic works for nearly 4 months
So an AI system that could fit 2500 years of learning/thinking/working into one day would be doing:
- A third of the learning of all PhD students in the world
- 10x the work of all AI researchers
- Thousands of times the work of OAI, GDM and Anthropic combined
Every day.
What about doing 1.8 million years of work in 2.4 months?
Making the same assumptions about human working time, you’d need around 40 million humans to fit 1.8 million years of work into that time.
For scale:
- There are around 30 million software engineers in the world
- The entire labour force of Germany is around 44 million
So GPT2030 would be doing more work than all software engineers combined.
__________________________________________________
So, where does all of this get me to? Mostly, that fast is pretty darn fast.
On Jacob’s predictions, it’s not the case that GPT2030 could do more work than humanity combined or anything crazy like that - but GPT2030 could be doubling the amount of software engineering, or ten-x-ing the amount of AI research, or thousand-x-ing the amount of AGI research.
I think worlds like that could look pretty strange.
Thanks to Owen Cotton-Barratt, Oscar Delaney, and Will MacAskill for comments; and to Max Dalton for unblocking me on posting.
Interesting. People probably aren't at peak productivity or even working at all for some part of those hours, so you could probably cut the hours by 1/4. This narrows the gap between what GPT2030 can achieve in a day and what all humans can together.
Assuming 9 billion people work 8 hours that's ~8.22 million years of work in a day. But given slowdowns in productivity throughout the day we might want to round that down to ~6 million years.
Additionally, GPT2030 might be more effective than even the best human workers at their peak hours. If it's 3x as good as a PhD student at learning, which it might be because of better retention and connections, it would be learning more than all PhD students in the world every day. The quality of its work might be 100x or 1000x better, which is difficult to compare abstractly. In some tasks like clearing rubble, more work time might easily translate into catching up on outcomes.
With things like scientific breakthroughs, more time might not result in equivalent breakthroughs. From that perspective, GPT2030 might end up doing more work than all of humanity since huge breakthroughs are uncommon.