When you say "I don't know how you can be confident(>50%) to say that it'll surpass human", I'm not sure if you mean "...in 20 years" or "...ever". You mention 20 years in one place but not the rest of your question, so I'm not really sure what you meant.
I mean "ever", thanks for the question
Unless there's a theory that said faster computing is the only thing AI lacks now to surpass humam, you mentioned the algorithms now are already enough for superintelligence AI, I wonder if there are articles talking on this
It's a hypothesis, without a strong consensus. This theory is called "we are not in a hardware overhang".
The theoretical basis for inability to forecast a principled upper bound on capabilities has I think mostly to do with all the mystery and confusion baked into ML. Before GPUs, the level of expertise that it was forecasted AI engineers would need to get impressive results was higher. And of the many candidate directions to go in, it may have felt kinda random that gradient descent (which you can do with AP coursework in high school, not that I was an AP student) took over.
But I would say it's much more about the ability to knock down confident assertions that things will top out than about confidence in assertions that things won't.
Sorry, I tried to make different paragraphs in my writing, but it keeps omitting the spaces I made between each sentences automatically.
(I'm an computer science outsider, so there may be a big knowledge gap between us.) Most experts think human-level AGI is just a matter of time and probably occur in 20 years. To conclude the reasons I've heard to support AI surpassing human:"Since AI has improved far beyond our expectations in history, and we are putting more and more efforts into developing AI, so we're optimistic AI's going to surpass human" I think my main problem is I (and mainstream of society) don't think "intelligence" is a simple thing. So, if AI systems now are still "far from" human-level intelligence in some aspects(such as doing science research, discovering a meaningful question by its own and design an experiment to prove it), though AI already had an impressive development history, I don't know how you can be confident(>50%) to say that it'll surpass human sooner or later. (Unless there's a theory that said faster computing is the only thing AI lacks now to surpass humam, someone mentioned the algorithms now are already enough for superintelligence AI, I wonder if there are articles talking on this) Or do you think AI's skill, such as GPT systems, is already near human-level? (It is in some aspects, but in areas such as scientific research, it seems still weak?) And for AI automating itself development in the future, yes it's possible, but it just means the AI development speed would be much faster, but it doesn't prove high-level intelligence is probable(>50%) if enough efforts are put. For myself, I am an AI outsider, so I can't predict the AI timelines. I just don't know why experts have confidence that the probabilty of AI surpassing human is >50%(this is different from non-expert people, most common people are not confident about this) I'm just talking on my doubts and hope I could know more convincing arguments.
Sections 1 and 2 of the Superintelligence FAQ may go some way toward answering your question.[1]
The FAQ doesn't discuss GPT systems since it's a few years old, but I still think it's the best non-technical intro to “Why will AI surpass humans, and why might this be dangerous?”. (The first two sections address the “Why will AI surpass humans?” question.)
Thanks for your replying. I've read this, but it doesn't say the reason why we expect AI to keep growing at an explanatory speed. Though the computing speed (FLOPs) is accelerating like Moore's law, but does fast computing=AGI can do most things on earth better than humans?