This post reads a little like someone pushing at an open door to me. So you write that FTX should ask themselves whether humanity should create AGI. The feeling I get from that is that you think FTX assume that AGI will be good. But the reason they've announced the contest is that they think the development of AGI carries a serious risk of global catastrophe.

Two of the propositions focus on when AGI will arrive. This makes it seem like AGI is a natural event, like an asteroid strike or earthquake. But AGI is something we will create, if we create it.

There are immense (economical, other) incentives to build AGI, so while humanity can simply choose not to build AGI, FTX (or any other single actor) is not in a position to choose not to build AGI. I expect FTX is open to considering interventions aimed at making that happen (not least as there's been some discussion on whether to try to slow down AI progress recently). But whether those work at all is not obvious.

How would know we had successful AGI if/when we created it? It would nothing like human intelligence, which is shaped not only by information processing, but by embodiment and the emotions central to human existence. ... So AGI cannot be like human intelligence.

As far as I'm aware, writers on AGI risk have been clear from the beginning that there's no reason to expect an AGI to take the same form as a human mind (unless it's the result of whole-brain emulation). E.g. Bostrom roughly defines AGI as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest" (Superintelligence ch. 2). There's no reason to think a highly capable but alien-to-us intelligence poses less of a threat than one that's similar to us.

AGI might be helpful for thinking about complex human problems, but it is doubtful that it would be better than task specific AI. Task specific AI has already proven successful at useful, difficult jobs (such as cancer screening for tissue samples and hypothesizing protein folding structures). Part of what has enabled such successful applications is the task specificity. That allows for clear success/fail training and ongoing evaluation measures.

There are advantages to generality too, like reducing the need for task-specific data. There's at least one example of a general intelligence being extremely successful, and that is our own, as evidenced by the last few billions' years of evolutionary history. An example of fairly successful general-ish AI is GPT-3, which was just trained on next-word prediction but ended up being capable of everything from translation and spell-checking to creative writing and chess-playing.

I'm excited to see what you come up with!

But even then, I think my logic stands: notice that OP talks about EA orgs in particular. Meaning OP does want to see a higher concentration of posts with views correlated to those of EA org employees. But that means a lower concentration of posts from people with views who don't directly align with EA orgs - which would cause a cycle of blocking more diverse views.

I suspect OP doesn't want more posts from employees at EA orgs because they are such employees -- I understood OP as wanting higher quality posts, wherever they come from.

True, the post does suggest that employees at EA orgs make higher quality posts on average, and that they may have less time to post on the Forum than the average user, but those are empirical matters (and seem plausible to me, anyway).

Edit to add: I generally didn't get the feeling that OP wholeheartedly supports intervening on any or each of these possible explanations, or that doing so wouldn't risk other negative consequences (e.g. increased groupthink).

From the wiki: "An existential risk is the risk of an existential catastrophe, i.e. one that threatens the destruction of humanity’s longterm potential." That can include getting permanently locked into a totalitarian dictatorship and things of that sort, even if they don't result in extinction.

This is nitpicky, but I wouldn't call that "an obscure academic screed":

• It was written by Charity Navigator leadership, who presumably felt threatened or something by GiveWell. So I think it was more like a non-profit turf war thing than an academic debate.
• I wasn't around at the time, but I have the impression that it was pretty (in)famous in EA circles at the time. Among other things it prompted a response by Will MacAskill. So it also feels wrong to call it obscure.

Possibly relevant: The Pope’s AI adviser on ensuring algorithms respect human dignity.

Thus any strategy to address SCD should definitely include increasing access to IVF/PGT in low resource countries.

That's a pretty bold claim. Are you sure that would be more cost-effective than the newborn screening and treatment intervention proposed in that post? IVF seems pretty expensive compared to the costs of screening and treatment.

Looks like the doc isn't publicly shared -- I get "You need access" when I try to view it.

I think "Reality doesn’t grade on a curve" might originally be from Scott Alexander's Transhumanist Fables.