Is there a news-tracker about GPT-4? Why has everything become so silent about it?

Franziska Fischer

Comments 4

Sorted by

New & upvoted

Charles He

"Become silent about it"

If you mean online media or the general vibe, one explanation is that DALL-E and diffusion models have been huge, taken up a lot of attention and OpenAI might have focused on that.
1. Like, very sophisticated tech leaders have gotten interested in these models, even when IMO language models have overwhelmingly larger business relevance for their products.
Actually, Copilot/codex have taken off and might be big. IMO this is underrated.
1. This might involve collab (with MS), and collabs are slower and harder/less gains to coordinate a buzzy media event on.
As a digression (or maybe not a digression) GPT-3 hasn't had the sort of massive tractability you (and others) might have expected, even after 2 years, and this reduces the incentives for further, investment on a 10x model.
1. Related to this, several other companies have spawned their own language models of similar/larger size. So this might somewhat reduce the attraction to winning on size or even reduce the returns to climbing the hill of dominating LLM in some broader sense.
  1. The instruct models and other sophistication is less legible and provide a larger barrier to entry (and produce other sorts of value), whereas parameter size / compute spend is not a moat.
2. GPT-3 has been somewhat uneconomic to run, for the larger models. Making a 10x model would make this even worse.
3. There is uh, deeper stuff. Umm, maybe more on this later.
Like any other buzz cycle, the first iteration is going to get a lot of buzz and appear in a lot of outlets. Articles mentioning GPT-3 haven't even stopped coming out.

openAI has made very extreme progress in the last years and decided to not publicise it for strategic reasons (e.g. to prevent from increasing the "race to AGI")

It's good and valid you write this, but this is very likely different from the truth. For one thing, in multiple senses/channels, OpenAI isn't totally locked down, and extreme sorts of developments would come out. Secondly, I think we've been truthfully told what they are working on (in the other comments), e.g. training refinements/sophistication approach to LLM.

Also, as a meta comment, in my opinion, it seems possible the beliefs that formed this valid question, may come from an information environment that seems not identical to an ideal information environment that would optimally guide future decisions related to AI or AI safety.

Franziska Fischer

Thanks for the elaborate answer. I'd be curious to hear a bit more of your thoughts regarding the meta-comment in your last paragraph + hints what to change about the information environment you suspect here, if you have time

note: feel free to be as unrigorous as you want with the response, you don't need to justify beliefs, just flesh them out a bit, I don't intend to contest them but want to use them to improve my understanding of that situation

Charles He

Ok, writing quickly. Starting on the "object level about the beliefs":

It seems like sentiment or buzz, like the tweets about GPT-4 mentioned in the other comment can be found. That gives a different view than silence mentioned in your post. It seems it could be found by searching twitter or other social media.
It seems like the content in my comment (e.g. I've suggested that there are various projects that OpenAI has under way and these compete for attention/PR effort) is sort of publicly apparent, low hanging speculation.
Let's say that OpenAI was actually unusually silent on GPT models and let's also accept many views of AI safety in EA. It's unclear why P(very extreme progress, more than 1 year with no OpenAI GPT release) would be large given some sort of extreme progress.
- In the most terrible scenarios, we would be experiencing some sort of hostile situation already.
- In other scenarios, OpenAI would aptly "simulate" a normally functioning OpenAI, e.g. releasing an incremental new model.
In my view P(very extreme progress | more than 1 year with no new GPT release) is low because many other underlying states of the world would produce silence that is unrelated to some extreme breakthrough, e.g. pivot, management change, general dysfunction, etc.
- It seems like it's a pretty specific conjunction of beliefs where there is some sort of extreme development and:
  - OpenAI is sort of sitting around, looking at their baby AI, but not producing any projects or other work.
  - They are stunned into silence and this disrupts their PR
  - This delay is lasting years
  - No one has leaked anything about it

I tried to stay sort of "object level" above, because the above sort of lays out the base level reasoning which you can find flaws in or contrast to my own.

From a meta sense, it's not just a specific conjunction of beliefs, but it's very hard to disprove a scenario where people are orchestrating silence about something critical—generally this seems like a situation where I want to make sure I have considered other viewpoints.

HStencil

There's been some anticipatory buzz about it on Twitter. No clue how credible this is, but the claim seems to be that we should expect it to be unveiled in early 2023. Also consider these comments from Sam Altman last year.

Comments

Charles He

"Become silent about it"

If you mean online media or the general vibe, one explanation is that DALL-E and diffusion models have been huge, taken up a lot of attention and OpenAI might have focused on that.
1. Like, very sophisticated tech leaders have gotten interested in these models, even when IMO language models have overwhelmingly larger business relevance for their products.
Actually, Copilot/codex have taken off and might be big. IMO this is underrated.
1. This might involve collab (with MS), and collabs are slower and harder/less gains to coordinate a buzzy media event on.
As a digression (or maybe not a digression) GPT-3 hasn't had the sort of massive tractability you (and others) might have expected, even after 2 years, and this reduces the incentives for further, investment on a 10x model.
1. Related to this, several other companies have spawned their own language models of similar/larger size. So this might somewhat reduce the attraction to winning on size or even reduce the returns to climbing the hill of dominating LLM in some broader sense.
  1. The instruct models and other sophistication is less legible and provide a larger barrier to entry (and produce other sorts of value), whereas parameter size / compute spend is not a moat.
2. GPT-3 has been somewhat uneconomic to run, for the larger models. Making a 10x model would make this even worse.
3. There is uh, deeper stuff. Umm, maybe more on this later.
Like any other buzz cycle, the first iteration is going to get a lot of buzz and appear in a lot of outlets. Articles mentioning GPT-3 haven't even stopped coming out.

openAI has made very extreme progress in the last years and decided to not publicise it for strategic reasons (e.g. to prevent from increasing the "race to AGI")