Researcher of causal models and human-aligned AI at FHI | https://twitter.com/ryancareyai
Posts
Wiki Contributions
Comments
There is also the PhilPapers Survey!
My advice for math is that it's often possible to think you understand something even if you don't, so it's good to do at least some exercises. Also, the methodology, and general "mathematical maturity" is often what you'll reuse the most in research - being able to reason by following specific allowed/disallowed steps, and knowing that you can understand a claim by reading Wolfram Mathworld, Wikipedia, textbooks, etc. So to some extent it doesn't matter so much what you learn, as that you learn something well. Having said that, the first half of a math textbook tends to be much more useful than the second half - there are diminishing returns in each subfield.
For programming, the same is often true - what you're aiming to get is a general sort of maturity, and comfort with debugging and building programs. So probably you want to mostly read tutorials for initial few weeks, then mostly do a project after that.
In both cases, I agree that a tutor is super-useful for getting unstuck, if you have the luxury of being able to afford one.
If this argument concluded that belief in consequentialism had bad consequences on net, it would be a more serious problem for consequentialism.
People often tell me that they encountered EA because they were Googling "How do I choose where to donate?", "How do I choose a high-impact career?" and so on. Has anyone considered writing up answers to these topics as WikiHow instructionals? It seems like it could attract a pretty good amount of traffic to EA research and the EA community in general.
I like the idea of an EA newspaper or magazine, and agree with using it to grow the EA community. But I think this pitch is somewhat inward-looking and unambitious. Moreover, journalism is the wrong business to be in for mitigating negative coverage. Posting a rebuttal in a magazine is going to increase the exposure of criticism, and pushback, as will the existence of a magazine in general. Posting B-tier profiles is a very indirect way to push back against elitism, and would not attract readers. An outlet should choose a content niche that people want to read, not just what you want them to read, and B-tier profiles seem like an example of the latter.
The question, then, is what content niche would some people be eager to read about, that we are equipped to do, and want to tell them about. What topics have EAs written about previously, that lots of people have wanted to read? I can think of some possibilities:
- Detailed analyses of scientific questions that matter to people
- Including the content of, & impact of policies
- Fact-checking the fact-checkers
- News on topics that matter for the long-term.
- Explainers of useful concepts
For a broader, less inward-looking paper, I don't know exactly the right name, but I don't think "The Altruist" is it.
I think that you should engage more seriously with the case of Future Perfect. Is it succeeding? What is its niche? What has gone well/poorly? What other niches do they think might be out there? And so on.
You also need to engage more seriously with the question of where you would find talent. Who would write for this outlet? Who could be the editor? In order to excite that founding team, you might need to give them a lot of leeway in shaping its direction.
In general you're allowed to review, but not vote on your own posts.
I agree with you, and with Issa that insofar as it's just a series of readings and discussions, "fellowship" is misleading.
And I agree with the OP that it's good to fund people, to incentivise students to learn and contribute. But I think paying reading group attendees is a weird place to start. Better to pay tutors, offer prizes, fund people if they recruit people to EA jobs, and so on.
Agreed. But if you're not an audio-creator, and you want to seriously refer to a podcast, it would usually make most sense to just transcribe it. Especially as this becomes automatable.
I'd rather keep the EA Forum as far away from images/video/audio as possible, so that it can best support serious discourse. There are better ways to widen the reach of EA, like:
- creating a magazine, like Works in Progress, Unherd, etc. Mostly this is about creating content.
- using non-EA platforms, i.e. we go against our programmer instincts (wanting to build a platform) and moderator instincts (trying to regulate conversation) and just communicate on Twitter etc. Again, mostly content.
- promoting simplified versions of content to get more views: promoting a paper with a Medium post, a blog post on Twitter, a Tweet with a meme, etc.
None of these is perfect, or even an uncontroversially good idea, but I think they're much better than trying to fit a round peg into a square hole by modifying the Forum into something more like Web 2.0, or a classic subreddit. In general, I find people to be systematically unimaginative about how to promote EA, and they fixate on existing entities like the Forum more than makes sense: Forum prizes (rather than prizes for content anywhere), meme pages on the Forum (rather than on r/effectivealtruism), videos on the forum, et cetera. The Forum is great for what it does, but it makes little sense to try to shoehorn such ambitions into it, when there are so many other possible ways to post and organise content.
The Safety/Capabilities Ratio
People who do AI safety research sometimes worry that their research could also contribute to AI capabilities, thereby hastening a possible AI safety disaster. But when might this be a reasonable concern?
We can model a researcher i as contributing intellectual resources of si to safety, and ci to capabilities, both real numbers. We let the total safety investment (of all researchers) be s=∑isi, and the capabilities investment be c=∑ici. Then, we assume that a good outcome is achieved if s>c/k, for some constant k, and a bad outcome otherwise.
The assumption about s>b/k could be justified by safety and capabilities research having diminishing return. Then you could have log-uniform beliefs (over some interval) about the level of capabilities c′ required to achieve AGI, and the amount of safety research c′/k required for a good outcome. Within the support of c′ and c′/k, linearly increasing s/c, will linearly increase the chance of safe AGI.
In this model, having a positive marginal impact doesn't require us to completely abstain from contributing to capabilities. Rather, one's impact is positive if the ratio of safety and capabilities contributions si/ci is greater than the average of the rest of the world. For example, a 50% safety/50% capabilities project is marginally beneficial, if the AI world focuses only 3% on safety.
If the AI world does only focus 3% on safety, then when is nervousness warranted? Firstly, technical researchers might make a big capabilities contribution if they are led to fixate on dangerous schemes that lie outside of current paradigms, like self-improvement perhaps. This means that MIRI's concerns about information security are not obviously unreasonable. Secondly, AI timeline research could lead one to understand the roots of AI progress, and thereby set in motion a wider trend toward more dangerous research. This could justify worries about the large compute experiments of OpenAI. It could also justify worries about the hypothetical future in which an AIS person launches a large AI projects for the government. Personally, I think it's reasonable to worry about cases like these breaching the 97% barrier.
It is a high bar, however. And I think in the case of a typical AI safety researcher, these worries are a bit overblown. In this 97%-capabilities world, the median person should worry a bit less about abstaining from safety contribution, and a bit more about the size of their contribution to safety.