All of michaelchen's Comments + Replies
I noticed that "Will humans build goal-directed agents?" was changed from being a required reading to Week 2 to being an optional reading. I don't disagree with this choice, as I didn't find the post very convincing, though I was rather fond of your post "AGI safety from first principles: Goals and Agency". However, now all the required readings for Week 2 essentially take for granted that AGI will have large-scale goals. Before I participated in AGI Safety Fundamentals in the first round this year, I never considered the possibility that AGI could be non-... (read more)
Your calculation looks correct to me. (WolframAlpha confirms "10^52 * 1% * 1 billionth * 1 billionth * 1%" is 10^30.) It seems that Nick Bostrom is underestimating the expected value by 10^10.
I have a hard time seeing longtermism being at risk for embracing eugenics or racism. But it might be interesting to look at the general principles for why people in the past advocated eugenics or racism—perhaps, insufficient respect or individual autonomy—and try to learn from those more general lessons. Is that what you're arguing for in your post?
These largely focus on the (indisputable) fact that avoiding X-risks can be tied to racist or eugenic historical precedents. This should be worrying;
I think most social movements can be traced to some sort of unsavory historical precedent. For example:
- Feminism and eugenics
- Socialism and human rights violations
- Anti-racism and the 1804 Haiti massacre
- Racial equality and Black supremacy
- Abolitionism and massacre
- Anti-imperialism and massacre
- Egalitarianism and massacre
I provide these examples not to criticize these movements but because I think these ... (read more)
I think that ignoring historical precedent is exactly what Scott was pointing out we aren't doing in his post, and I think the vast majority of EAs think it would be a mistake to do so now.
My point was that we're aware of the skulls, and cautious. Your response seems to be "who cares about the skulls, that was the past. I'm sure we can do better now." And coming from someone who is involved in EA, hearing that view from people interested in changing the world really, really worries me - because we have lots of evidence from studies of organizational decision making and policy that ignoring what went wrong in the past is a way to fail now and in the future.
It seems pretty bizarre to me to say that these historical examples are not at all relevant for evaluating present day social movements. I think it's incredibly important that socialists, for example, reflect on why various historical folks and states acting in the name of socialism caused mass death and suffering, and likewise for any social movement look at it's past mistakes, harms, etc., and try to reevaluate their goals in light of that.
To me, the examples you give just emphasize the post's point — I think it would be hard to find someone who di... (read more)
What does AI safety movement building look like? What sorts of projects or tasks does this involve? What are the relevant organizations where one could do AI safety movement building work?
Reading your suggested required readings, S-risks: An introduction and S-risk FAQ, I don't get a clear sense of why s-risks are plausible or why the suggested interventions are useful. I like S-risks: Why they are the worst existential risks, and how to prevent them (EAG Boston 2017) a bit more for illustrating why they are plausible, and I've added it as an optional reading in the uni chapter intro program I'm running. Unfortunately, it doesn't give more than a cursory overview of how s-risks could be reduced. I'd be hesitant about making an s-risk readi... (read more)
So I see https://www.povertyactionlab.org/initiative/crime-and-violence-initiative and https://www.poverty-action.org/topics/crime but based on a quick examination, I have no idea how cost-effective these interventions are. Does anyone have links providing an estimate of the cost-effectiveness of violence prevention?
I can't comment on the Google Doc version of this post. Can you add Impact CoLabs?
Figuring out how to stop AI systems from making extremely bad judgments on images designed to fool them, and other work focused on helping avoid the "worst case" behaviors of AI systems.
I haven’t seen much about adversarial examples for AI alignment. Besides https://www.alignmentforum.org/tag/adversarial-examples (which only has four articles tagged), https://www.alignmentforum.org/posts/9Dy5YRaoCxH9zuJqa/relaxed-adversarial-training-for-inner-alignment, and https://cset.georgetown.edu/publication/key-concepts-in-ai-safety-robustness-and-adversarial-exa... (read more)
Is progressive activism for the climate more cost-effective than nonpartisan activism (such as by Citizens' Climate Lobby)?
At least for me, I think daily accountability—and having to write a reflection if you fail to meet your goals—is a lot greater of an incentive than the threat of donating to PlayPumps a few months down the line.
I'm surprised that retreats are low-effort to plan! What sorts of sessions do you run? What draws people in to attend?
I think you'd get a lot more answers if you ask your question in the EA Groups Slack: https://efctv.org/groupslack
Giving What We Can's guide to talking about effective altruism has some good tips.
Inviting people to come to a nearby EA meetup or to apply to a locally hosted EA Fellowship sounds good.
One thing you can see if you can set up is presenting to someone in charge of philanthropy at the organization about EA. I know someone who just had a presentation like that at his internship company, though he wasn't successful in causing them to give to effect... (read more)
Looks good! Some minor suggestions:
- Remove "Made with Squarespace" in the footer
- Add a favicon to the website
Hey Markus, I'm only getting started with organizing an EA group, but here are my thoughts:
- I think 6 hours per week is enough time to sustain a reasonable amount of growth for a group though, but I don't have enough personal experience to know. If you think funding would enable you to spend more time on community building, you can apply for an EA Infrastructure Fund. And you can always get Group Support Funding to cover expenses for things you think would help, such as snacks, flyers, books etc.
- I think the Intro EA Program is a surprisingly effective wa
While I think a write up of my experience as a web development intern wouldn’t add much value compared to the existing web developer post, I’d be interested in writing a guide to getting a (top) software engineering internship/new grad position as a university student. (Not saying my past internships are top-tier!) I'm planning on giving an overview about (or at least link to resources about) to how to write a great resume, preparing behavioral interview answers, preparing for technical interviews with LeetCode-style or system design questions, and so on. ... (read more)
Phil Torres's tendency to misrepresent things aside, I think we need to take Phil Torres's article as an example of the severe criticism that longtermism is liable to attract, as currently framed, and reflect on how we can present it differently. It's not hard to read this sentence on the first page of (EDIT: the original version of) "The Case for Strong Longtermism":
... (read more)The idea, then, is that for the purposes of evaluating actions, we can in the first instance often simply ignore all the effects contained in the first 100 (or even 1000) years, focussing pr
I listen to a good amount of podcasts using the Pocket Casts app (or at least I did for a couple years up until a few weeks ago when I realized that I find YouTube explanations of tech topics a lot more informative). But when I'm browsing the EA Forum, I'm not really interested in listening to podcasts, especially podcast versions of posts I've already read that I could easily re-read on the EA Forum. I think this is a cool project but after the first couple of audio narration posts which were good for generating awareness of this podcast, I don't think it... (read more)
Somewhat low-effort answer:
I know Doing Good Better mentions the logarithmic relationship between income and wellbeing in one of its earlier chapters.
Relevant posts/articles:
It seems inconvenient if applicants potentially have to fill out the Virtual Programs application form too and receive a second acceptance/rejection decision—could we have just one application form for them to fill out and one acceptance/rejection decision notification? I was thinking that hopefully we could have something like the following process:
- Have applicants apply through the EA Virtual Programs form, or have a form specific to our chapter which puts data into the EA Virtual Programs application database. (I don't know enough about Airtable to kno
In Stuart Russell's Human Compatible (2019), he advocates for AGI to follow preference utilitarianism, maximally satisfying the values of humans. As for animal interests, he seems to think that they are sufficiently represented since he writes that they will be valued by the AI insofar as humans care about them. Reading this from Stuart Russell shifted me toward thinking that moral circle expansion probably does matter for the long-term future. It seems quite plausible (likely?) that AGI will follow this kind of value function which does not directly care ... (read more)
I see this sentence as suggesting capitalizing on the (relative) popularity of anti-racism movements and trying to use society's interest in anti-racism toward genocide prevention.
The Introductory EA Virtual Program has been invaluable for getting enough people engaged in EA for me to be able to start a group at my university, and for that I'm extremely grateful to those who have helped organize it and develop the curriculum. If you're in the same position I was a few months ago, reasonably interested in starting an EA group but having difficulty finding enough people interested in EA, I'd highly recommend advertising the Introductory EA Program!
I'm interested in running a local in-person program at my university from September to O... (read more)
I agree that people should edit a Wikipedia article directly or discuss on the talk page instead of complaining about it elsewhere. Leaving a comment on the talk page can be a quick way of helping shift the consensus for a controversial topic. In my experience though, unless it's a very popular page, it's often the case that when someone leaves a comment on the talk page describing overall changes that they want to be made, no one will respond and no changes will be taken. Or someone responds with an agreement or disagreement, and nothing happens. Thus, es... (read more)
I agree with this advice.
I think a simple way to get involved with Wikipedia is to "adopt" an article on an important topic you are familiar with but which is currently covered inadequately. This will allow you to see how your changes are received, develop a relationship with other editors who contribute regularly on that page, and experience the satisfaction of seeing the article (hopefully) improve over time in part thanks to your efforts.
Something similar to explicit credence levels on claims is how Arbital has inline images of probability distributions. Users can vote on a certain probability and contribute to the probability distribution.
Effective Altruism Lodge
Effective Altruism House
I've seen a mix of some people capitalizing effective altruism, maybe more often in communications with a more general audience, and some people not capitalizing it. I generally try leave it uncapitalized, following CEA policy, but sometimes I capitalize it when it makes it clearer that effective altruism is an actual Thing, not just altruism that is effective, and not a generic made-up compound term like, say, "efficient humanitarianism" or "impactful lovingkindness". For example, if I write in a self-introduction "I'm passionate about effective altruism"... (read more)
Anecdotally, it's quite tiring to put credence levels on everything. When I started my blog I began by putting a probability on all major claims (and even wrote a script to hide this behind a popup to minimise aesthetic damage). But I soon stopped.
Interesting! Could you provide links to some of these blog posts?
According to Fleck's thesis, Matsés has nine past tense conjugations, each of which express the source of information (direct experience, inference, or conjecture) as well as how far in the past it was (recent past, distant past, or remote past). Hearsay and history/mythology are also marked in a distinctive way. For expressing certainty, Matsés has a particle ada/-da and a verb suffix -chit which mean something like "perhaps" and another particle, ba, that means something like "I doubt that..." Unfortunately for us, this doesn't seem more expressive than ... (read more)
I'm pretty interested in linguistics so after reading gavintaylor's comment which you linked to, I decided to read part of David Fleck's doctoral thesis from 2003 on the Matsés language. For distinguishing levels of certainty, Matsés has a particle ada/-da and a verb suffix -chit which mean something like "perhaps" and another particle, ba, that means something like "I doubt that...", and English speakers already naturally express these distinctions of certainty. Something distinctive that Matsés speakers do though is that they always mark statements about... (read more)
An extended transcript of the talk is available at https://www.alignmentforum.org/posts/jS2iiDPqMvZ2tnik2/ai-alignment-philosophical-pluralism-and-the-relevance-of. There's also a lot more discussion there.
I initially thought that CEA here stood for Centre for Effective Altruism, and only later did I realize that it stood for cost-effectiveness analysis.
I upvoted, but here are some comments I have. Looking at the titles of the first three videos, it wasn't clear how they related to rationality.
- How Does Bitcoin Work? Simple and Precise 5…
- Why Your Life Is Harder When You Are Ugly | The… (didn't notice the "Halo Effect" in the thumbnail at first)
- Why You STRUGGLE to Finish on Time | The…
So perhaps people downvoted based on first impressions that it doesn't seem that related to rationality?
I enjoyed the Bitcoin, halo effect, and planning fallacy videos, but I didn't think that the video "If You Want to F... (read more)
How long does it take to play through this game?
What topics are you thinking of making videos for in the future? (Or is this information reserved for Patreons?)
You say "first three videos", but I can also see the following videos:
- redstone neuron in minecraft!
- minecraft 24 hour digital clock
- etc.
I don't think that's a big deal though—you can keep them up and maybe they'll attract some more viewers to your channel.
Also, one month ago on your YouTube channel, you posted a link to your Patreon, but it actually links to https://www.patreon.com/rationalanima (which is dead) instead of https://www.patreon.... (read more)
The blog post links "Ridiculous math things which ethics shouldn't depend on but does" and "Kill the young people" are dead. You can find archived versions of the posts at the following links:
Some things I do:
- Set my computer to automatically shutdown at certain times, such as 12:10am, 12:30am, and 1am. I chose the time of 12:10am because assignments are generally due at 11:59pm, so there's no need to stay up later. Since I'm using Windows, I can do this with Task Scheduler: https://www.techrepublic.com/article/how-to-schedule-a-windows-10-shutdown-for-a-specific-date-and-time/
- Set an alarm on my computer at 12am to remind me that my computer is going to shut down soon. This alarm plays only when the computer is awake (i.e., when I might be us
Nitpick: The first link leads to https://forum.effectivealtruism.org/posts/3mnLZtbeBSWZqQCLb/cisac.fsi.stanford.edu/stanford-existential-risks-initiative, which is a dead link. I think you mean to link to cisac.fsi.stanford.edu/stanford-existential-risks-initiative.
I think this point from the Black VegFest 7 points of allyship (for the white vegan community) is reasonably straightforward:
White vegans/ARs will respect the sanctity of Black space and will not enter unless their presence is necessary. Black space is for the growth and betterment of Black people. Allyship and being accomplices begins with white people learning to respect Black space.
My understanding is that there can be spaces for only Black people to discuss, though white people can participate if necessary (presumably, if they are invited). Part of... (read more)
Nitpick: I really wish SJ-aligned people would clarify what they mean by "capitalism" in these contexts.
I found this post to be quite refreshing compared to the previous one criticizing Effective Altruism Munich for uninviting Robin Hanson to speak. I’m not against “cancel culture” when it’s cancelling speakers for particularly offensive statements they’ve made in the past (e.g., Robin Hanson in my opinion, but let’s not discuss Robin Hanson much further since that’s not the topic of this post). Sometimes though, cancelling happens in response to fairly innocuous statements, and it looks like that’s what ACE has done with the CARE incident.
Sure, debate may involve a lot of techniques that are counterproductive to truth-seeking, and I wouldn't want people to write on the EA Forum like it's a debate, for example. However, I think there are many places where it would help to be able to convey more convincing arguments even if being more convincing doesn't improve truth-seeking—speaking with non-EAs about EA, for example.
I generally want us to use truth-seeking methods when engaging with outsiders as well. Of course, that isn't always possible, but I also really don't want us to have a reputation for using lots of rhetorical tricks to convince others (and generally think that doing so is pretty bad).
Are there recordings of the debates? I'd be interested in watching them.
If necessary, it might be good to frame the arguments from religious texts as connecting with traditional Jewish thought, not in a way that demands a belief (or lack of belief) in the literal accuracy of the Talmud—basically what (my understanding of) Reform Judaism does. It might be good to intersperse religious arguments with secular arguments as well.
... (read more)Out there, though, you can meet real people, with their own rich and complex lives; you can make real friends, and be part of real relationships, communities, and institutions. You can wander cities with real history; you can hear stories about things that really happened, and tell them; you can stand under real skies, and feel the heat of a real sun. People out there are doing real science, and discovering real things. They’re barely beginning to understand the story they’re a part of, but they can understand. You can understand, too; you can be a part o
I ended up making a post based on Kuhan's. It's somewhat shorter and has less of a focus on careers. I ended up getting 14 reactions (for reference, I have a bit over 600 friends on Facebook). I wonder if accompanying the post with a new profile picture would have been a good way to get more engagement haha.
... (read more)My birthday is today! For the past two years, I did birthday fundraisers, but this year, instead of gifts or donations, I’m asking for just five minutes of your attention.
In eighth grade, I discovered a social impact–oriented career site called 80000h
I'm not a fan of the name "Probably Good" because:
- if it's describing the advice, it seems like the advice might be pretty low-effort and not worth paying attention to
- if it's describing the careers, it sounds like the careers recommended have a significant chance of having a negative impact, so again, not worth reading about
I can see an AI creating clones or other agents to help it achieve its goals. And they might all try to help each other survive to work toward that goal. But that doesn't mean helping each other feel positive experiences (at least not necessarily). It could even involve a significant degree of punishment to shape actions to better achieve that goal, although I'm less sure about this.