Thanks for writing up your thinking on this, and also more in general for doing the hard work of community building. 🙌

I think Tess makes good points in their comment about the huge amounts of uncertainty contained in that impact potential factor, and I would go a step further and say:

The factor i is a red herring, and it is harmful for community building to try and predict it.

• "intuitively" predicting this is extremely succeptible to bias and rationalizations
• overconfident community builders using this to justify their decisions will come off as arrogant and elitist
• it will increase impostor syndrome ("my local group doesn't have a high enough impact potential")
• it's dependent on which ethical frameworks are correct
• core EA orgs weren't able to predict the hugely negative impact of SBF

Using it to decide who is "most worthy of outreach" is in direct conflict with core strenghts of the EA community: cause-neutrality and openness to neglected approaches to making the world a better place.

What should be done instead?

Aim to build a broad, diverse community that is welcoming and non-judgemental and accurately communicates EA ideas to a large audience. Build a community that you would want to be part of. Does this mean abandoning quantitative measures? No, we can still measure if we achieve our goals:

• measure participant's understanding of core EA ideas
• measure diversity
• measure people reached
• measure giving pledges
• measure money donated

The high impact potential people (those mythical creatures) will find their way once they have come in contact with an accurate representation of EA ideas. It's not the job of community builders to identify and guide them (because they can't), their job is to build an awesome community that communicates EA ideas with high fidelity so that many people come in contact with good first hand accounts of these ideas instead of simplified straw-man versions of EA in take-down articles about scandals in the EA community.

Yeah, I mainly posted it because it's good comedy. That doesn't mean I think Give Directly should adopt it.

As inspiration, here's a quote by Scottish stand-up comedian Frankie Boyle on how patronizing the teaching-to-fish idea is:

https://www.youtube.com/watch?v=4X5KmQxvO5U&t=1537s

Give a man a fish, and he can eat for a day. Give him a fishing rod, and he can feed himself. Alternatively, don’t poison the fishing waters, abduct his great-grandparents into slavery, then turn up 400 years later on your fucking gap year talking a lot of shite about fish.

A […] report linking deforestation in the Amazon to soy and beef production was published in Science […], Forbes and BBC then mistakenly announced that 1/5 of the entire national exports were so contaminated.

Sadly this kind of misrepresentation of scientific papers or overgeneralization of their actual results seems to be a common pattern.
Have you tried to contact the authors at Forbes and BBC? Maybe writing letters to the editor could be a quick way to curb some of the harms of sub par science reporting? (at least in cases where  the article misinterprets the underlying paper)

Do all EA orgs need to use the same tools? 

No, and I hope I didn't imply that there is a one-size-fits-all solution that everybody needs to switch to.

can those with added security needs (e.g. those active in countries with government repression) just use different tools

Yes, that is of course possible, and I would expect that to happen automatically. Just note that this means in some cases that we will exclude those people with added security needs from community spaces.

Things that would make me less worried about "using whatever works best":

• switching later is actually easier than I currently think
• information becoming available to future adversarial actors is actually not as bad because:
  • either people are really good at not posting compromising stuff
  • or there is almost no content that will become problematic in the next years

Another possible reason against might be:
In some countries there is a growing number of people who intentionally don't use Facebook. Even if their reasons for their decision may be flawed, it might make recruiting more difficult. While I perceive this as quite common among German academics, Germany might also just be an outlier.

Moving certain services found on Facebook to other sites: [...], making it easier for people to reach out to each other (e.g. EA Hub Community directory). Then it may be easier to move whatever is left (e.g. discussions) to a new platform.

I think the EA Hub is in a good position to grow and replace some of the functions that Facebook is currently being used for in the community.

Thank you Aaron for the thoughtful reply.

I find your suggestions on better questions to get more achievable types of evidence very useful. @Manuel_Allgaier and me will ask them or similar ones on the EA Berlin Slack, a German EA Telegram channel and in the FB group you mentioned.

 [...] a strong prior in favor of valuing extra marginal security more than the convenience we'd lose in order to achieve that.

Yes, that is a good way to rephrase my position.

Google may be more vulnerable to breaches, but it seems much less likely than a small private server to lose data because someone makes a technical error or loses a key password.

This is probably a central crux for some. If you came to believe that the risk of data loss through admin error on a self-hosted system were lower than the breach-risk at Google, would that change your view on the convenience-security trade-off?

Thank you, you're right I added the predictions at the last minute, and should have spent a few more minutes making sure that they are operationalized well.

I added a clarification about the kind of leaks I meant, as you noted if any individual sharing a screenshot counts, it would not be a useful prediction.

Same for government repression – I added another question for US,UK and EU.

 

Thanks, I'll try and embed them here:
 

(Here i mean data leaks that happen on a company level, meaning the service provider leaks full message histories. This excludes leaks through missteps by individuals such as accidentally setting wrong channel permissions, or leaked screenshots)

(for example withdrawal of operating license, denial of work visas, etc. in any country)
 

Ok, this seems to be a good workaround: For posts with footnotes, use the markdown editor. Then after posting, switch to the regular editor in your profile settings, and post your elicit prediction links in a comment.

edit: updated with clarifications and a fourth question as meerpirat suggested