Comments
My timeline for AGI hasn't changed much, but my timeline for 'semi-transformational narrow AI' has become shorter.
Lots of academics are talking about how ChatGPT (and student and teacher use thereof) will force a revolution in how we assign homework, writing assignments, exam questions, and even class discussion questions. The whole experience of school and college will either have to change dramatically (e.g. a return to in-person lectures & discussions, and in-person paper-and-pencil exams), or schools and colleges will become empty rituals in which teachers using AIs pretend to teach and grade things, and students using AIs pretend to learn things.
Similar pressures will arise in many other industries, social practices, and relationships.
I think what ChatGPT highlights, fundamentally, is that even 'narrow AI' will be transformational enough to impose a dizzying rate of change on our civilization, and to impose qualitatively new kinds of risks.
Correct me if I’m wrong but my understanding is most everything ChatGPT can do was already possible with GPT3 (especially post InstructGPT) but it just took more intentional wrangling. What ChatGPT seems to be offering is a much more accessible interface.
That sounds accurate. The key difference with ChatGPT is that there's a LOT more public attention to the underlying capabilities of GPT-3.
text-davinci-003 (which is effectively ChatGPT) was a bit better than text-davinci-002 anecdotally and when I benchmarked it on TriviaQA. It was only released about a week before ChatGPT so it's not necessarily unreasonable to lump them together. If you do, then the interface isn't the only change one might associate with ChatGPT.
This is probably a stupid question, but: do we actually know if ChatGPT uses text-davinci-003?
When I talk to ChatGPT with the Network tab of Chrome DevTools open, filter for the name "conversation," and look at any request payload, I see that it has the key-value pair
Which seems to indicate that it might not be using text-davinci-003.
The blog post says ChatGPT is trained with proximal policy optimization. This documentation says text-davinci-003 was trained with PPO, but not text-davinci-002.
However, it is interesting what you're saying about the request payloads, because this seems to be contradictory. So I'm not quite sure anymore. It's possible that ChatGPT was trained with PPO on top of the non-PPO text-davinci-002.
Yeah, it didn’t update my timeline number much since I’d seen other language models, but it started to make short-timeline intuitions feel a lot more real as the capabilities are a lot more obvious now.