Thank you, these are some really big questions! Most of them are beyond what we work on, so I'm happy to leave these to other people in this community and have them guide our own work. For example, the Centre for Long-Term Resilience published the Future Proof report in which they refer to a survey where the median prediction of scientists is that general human-level intelligence will be reached around 35 years from now.
I'll try to answer the last question about where our opinions might differ. Many academics and policymakers in the EU probably still...
Thank you, a lot of great questions. In response to question (3), some of our work focuses on EU member states as well. Because we are a small team, our ability to cover many member states is limited, but hopefully, with the new hire we can do a lot more on this front as well. If you know anybody suitable, please let us know. For example, we have engaged with Sweden, Estonia, Belgium, Netherlands, France, and a few other countries. Right now, the Presidency of the Council of the EU is held by France, next up are Czechia and Sweden, so work at the member state level in these countries is definitely important.
Regarding your 2nd question, I think it is an important argument and it's good that some people are thinking through both the arguments in favor and against working on EU AI governance. That said, there are so many ways for EU AI governance to play a major role regardless of whether it is an AI superpower or not. Some of these are mentioned in the post that you referred to, like the Brussels Effect as well as excellent opportunities for policy work right now. Some other ideas are mentioned in the comments under the post about EU not being an AI superpower,...
Thank you for the questions. I think that the biggest bottleneck right now is that very few people work on the issues we are interested in (listed here). We are trying to contribute to this by hiring a new person, but the problems are vast and there's a lot more room for additional people. Another issue is lack of policy research that would consider the longer-term implications but would at the same time be very practical. We are happy that in addition to the Future of Life Institute, a few other organizations such as Centre for the Governance of AI, Centr...
If anyone reading this post thinks that the arguments in favor outweigh the arguments against working on EU AI governance, then consider applying for the EU Policy Analyst role that we are hiring for at the Future of Life Institute: https://futureoflife.org/2022/01/31/eu-policy-analyst/. If you have any questions about the role, you can participate in the AMA we are running: https://forum.effectivealtruism.org/posts/j5xhPbj7ywdv6aEJc/ama-future-of-life-institute-s-eu-team.
Thank you for writing this summary!
I wanted to share this new website about the AI Act we have set up together with colleagues at the Future of Life Institute: https://artificialintelligenceact.eu/. You can find the main text, annexes, some analyses of the proposal, and the latest developments on the site. Feel free to get in touch if you'd like to discuss the proposal or have suggestions for the website. We'd like it to be a good resource for the general public but also for people interested in the regulation more closely.
Yeah, I feel that too. My daughter is just 1 year and 9 months. We are constantly high-fiving and fist-pumping.
Because (i) my wife wanted to have a child and I thought it would strenghten our relationship, (ii) I assumed my child was likely to become a happy person and possibly an EA, (iii) I'd potentially have a very close friend for life.
My little dude is only 2 but one of my best mates. Have never had more laughs than as a dad. But, never had more tears either. It's turbulent, but the highs are high.
Existential risks are not something they have worked on before, so my project is a new addition to their portfolio. I didn't mention this but I intend to have a section for other risks depending on space. The reason climate change gets prioritized in the project is that arguably the EU has more of a role to play in climate change initiatives compared to, say, nuclear risks.
Thanks for this database! I'm currently working on a project for the Foresight Centre (a think-tank at the Estonian parliament) about existential risks and the EU’s role in reducing them. I cover risks form AI, engineered pandemics, and climate change. For each risk, I discuss possible scenarios, probabilities, and the EUs role. I've found a couple of sources from your database on some of these risks that I hadn't seen before.
The same is the case with the effective altruism course at the LSE titled Effective Philanthropy: Ethics and Evidence. The reason for that was that the teacher Luc Bovens moved to work for another institution. I don't know about UCL.
It would also be more informative to assess risks of death from COVID-19. 'Micromorts' normally stand for one-in-a-million chance of death because the word is combined from micro and mortality. If 1000 μCoV were a thousand-in-a-million chance of death, then engaging in activities with such a risk would be quite reckless indeed. That would be about similar to climbing quite high mountains and doing a couple of base-jumps.
I have calculated COVID-19 risks for myself in the context of Estonia where I am currently. My numbers right now are abo...
"You should not do a PhD just so you can do something else later. Only do a PhD if this is something you would like to do, in itself."
Why do you think this is the case? For example, I have noticed based on my search that nearly 60% of research roles in think-tanks in Europe have PhDs and that proportion is greater for senior research roles and more academic think-tanks. This does not account for the unmeasurable benefits of PhDs such as being taken more seriously in policy discussions. Isn't it possible that 4-6 years of PhD work gives you more impressive career capital than the same amount of experience progressing from more junior roles to slightly more senior ones?
This post was actually published in 2018 for the first time, but for some reason I wasn't able to share the link with some people as it showed up as a draft. I resubmitted it and it has received some interest from the community again.
I think that the longer term evidence right now indicates that the impact of this was lower than the short-term evidence made me anticipate. I expected to have several highly engaged new members in the EA community longer term, but currently it appears that these people are only weakly involved with effective altruism. H...
A number of factors, but the biggest was suitedness to the role. I tend to get a lot of energy from talking to other people. My role at GPI was very independent - at the time I was the only operations person there, and academics tend to work fairly individually on their research. By comparison, my current role involves not just talking to people I advise, but the team is also more collaborative in general (for example, it makes sense for the research team and advising team to collaborate quite a bit because advising calls are both how we get out quite a bi...
Do you think there's more useful research to be done on this topic? Are there any specific questions you think researchers haven't yet answered sufficiently? What are the gaps in the EA literature on this?
It actually might be more complicated than what you say here, alexherwix. If a research analyst role at the Open Philanthropy Project receives 800+ job applications, then you might reasonably think that it's better for you to continue building a local community even if you were a great candidate for that option.
In addition, for the reasons that you mention, every possible local community builder might be constantly looking for new job options in the EA community making someone who doesn't do that a highly promising candidate. Furthermore, being ...
This is slightly relevant, in a recent 80,000 Hours' blog post they suggest the following for people applying for EA jobs:
We generally encourage people to take an optimistic attitude to their job search and apply for roles they don’t expect to get. Four reasons for this are that, i) the upside of getting hired is typically many times larger than the cost of a job application process itself, ii) many people systematically underestimate themselves, iii) there’s a lot of randomness in these processes, which gives you a chance even if you’re not truly th...
You can decide it by asking who wants to be the leader of a particular activity (the way that your group did) as well as inquire what resources and capital people have available to successfully lead that activity. Sometimes people have the motivation to lead activities, but they don't actually have the necessary resources to do it successfully yet.
Agreed on the failure-mode thinking. I guess if you only take the best-case scenario into consideration, then you forget to assess the risks involved. On the other hand, I'm not sure it should be included in this initial brainstorming session or later when a possible activity is selected as a top candidate.
So here are some of the main takeaways from this for me:
We recently had a several-hour strategy meeting. I can attest to that when community members participate in the task of developing a strategy they understand better what's going on and they feel more motivated as they are actually responsible for the vision now. And they can come up with wonder...
Great overview as always. I think Open Philanthropy Project's Funding for Study and Training Related to AI Policy Careers should be up here as well:
This program aims to provide flexible support for individuals who want to pursue or explore careers in AI policy1 (in industry, government, think tanks, or academia) for the purpose of positively impacting eventual societal outcomes from “transformative AI,” by which we mean potential future AI that precipitates a transition at least as significant as the industrial revolution ...
I think this accusation is uncalled for. There is more statistics in the report and I linked to it, including things like citation impact. But a comprehensive overview of European AI research is, of course, very welcome.
For what it's worth, according to Artificial Intelligence Index published in 2018:
Europe has consistently been the largest publisher of AI papers — 28% of AI papers on Scopus in 2017 originated in Europe. Meanwhile, the number of papers published in China increased 150% between 2007 and 2017. This is despite the spike and drop in Chinese papers around 2008.
(I'd post the graphs here, but I don't think images can be inserted into comments.)
My lived experience is that most of the papers I care about (even excluding safety-related papers) come from the US / UK. There are lots of reasons that both of these could be true, but for the sake of improving AGI-related governance, I think my lived experience is a much better measure of the thing we actually care about (which is something like "which region does good AGI-related thinking").
This strikes me as an isolated example of Europe leading on one metric. I plan to write something more comprehensive, but I think just seeing this statistic could create a wrong impression for some people.
(edited to remove accusatory tone)
You can post an image using standard markdown syntax:
![](link to the image)
For example, to insert the above image, I wrote:
![](https://nunosempere.github.io/ea/AI-Europe.png)
William MacAskill says the following in a chapter in The Palgrave Handbook of Philosophy and Public Policy:
As defined by the leaders of the movement, effective altruism is the use of evidence and reason to work out how to benefit others as much as possible and the taking action on that basis. So defined, effective altruism is a project rather than a set of normative commitments. It is both a research project—to figure out how to do the most good—and a practical project, of implementing the best guesses we have about how to do the most good.
But then he cont...
Thank you for writing this! This is a useful overview of active groups for me, because I intend to move to London in September to study at LSE and now need to think about ways to engage with the community there.
Your link referring to bdixon and climate change leads to Joey's post "Problems with EA representativeness and how to solve it". Can you share the post that discusses how Doing Good Better appears to underrate the degree of warming of climate change?
I found the part about philosophers being well-suited to many aspects of EA research especially interesting. You said this:
Contrary to popular stereotypes, philosophers often excel at quantitative thinking. Many philosophy PhDs have an undergraduate background in math or science. For subfields of philosophy like formal epistemology, population ethics, experimental philosophy, decision theory, philosophy of science, and, of course, logic, a strong command of quantitative skills is essential. Even beyond these subfields, quantitative acumen is prized. In an...
Can you expand on 3a and 3b? I guess 3b justifies 3a, but is that all? Watching and discussing a video with your local group appears to me to be more valuable than asking one question at a talk, but I may be missing some important benefits that you are aware. I would also add that these are not mutually exclusive. I have heard that some people struggle to set time to watch talks on their own, that is also something to consider.
You received almost 100 applications as far as I'm aware, but were able to fund only 23 of them. Some other projects were promising according to you, but you didn't have time to vet them all. What other reasons did you have for rejecting applications?
Hmm, I don't think I am super sure what a good answer to this would look like. Here are some common reasons for why I think a grant was not a good idea to recommend:
Realising that attendance and events are just part of a community, and potentially not the most important part
Agreed. Research and study groups, for example, seem to be a lot more useful than events. First and foremost, participants commit to longer term attendance in advance so you don't need to try to persuade them to participate every time. I dislike having to personally invite people to come to events. I assume that they don't care about EA enough if they don't come at a mere FB invitation.
Regarding attendance, we just recently organize...
If you're a thoughtful American interested in developing expertise and technical abilities in the domain of AI policy, then this may be one of your highest impact options, particularly if you have been to or can get into a top grad school in law, policy, international relations or machine learning. (If you’re not American, working on AI policy may also be a good option, but some of the best long-term positions in the US won’t be open to you.)
What do you think about similar type of work within the European Union? Could it potentially be a high-impact career path for those who are not Americans?
This post increased my interest in visiting the Boston area. Unfortunately, I cannot come to the EAGx this year, but perhaps another time. I'm quite surprised that you'd have the issue of brain drain as the area seems to be a very impressive place with top universities, lots of people interested in EA, and even a few great EA-aligned organizations. Do you have other ideas besides a full-time paid community builder to improve that?
Nice idea. I wrote my bio in third-person like you did even though on my website I have it in first-person: https://ristouuk.com. Usually, I feel weird about the third-person narrative when I'm the one who is talking about me, but it feels right for the forum.
As an application of this model, the Global Priorities Project estimates that research into the neglected tropical diseases with the highest global DALY burden (diarrheal diseases) could be 6x more cost-effective, in terms of DALYs per dollar, than the 80,000 Hours recommended top charities.
What are 80,000 Hours' recommended top charities? I think you mean some other organization here.
It would be nice if someone updated it regularly and had a note about when it was last updated on the top of the page. For example, according to Julia Wise there were 3855 Giving What We Can members at the beginning of 2019, whereas the number here is outdated with 1800+ members.
Let’s face it. Long-termism is not very intuitively compelling to most people when they first hear of it. Not only do you have to think in very consequentialist terms, you also have to be extremely committed to acting and prioritizing on the basis of fairly abstract philosophical arguments. In my view, that’s just not very appealing - sometimes even off-putting - if you’ve never even thought in terms of cost-effectiveness or total-view consequentialism before.
I agree. Because of this, the 2nd edition of the EA handbook doesn't seem appealing at all as a
...Thank you for writing this summary!
...
- Altruism: Passionate about helping others
- Effectiveness: Ambitious in their altruism, with a drive to do as much good as they can. Potential to be aligned with the central tenets of EA.
- Potential: Excited to dedicate their career to doing good or to donate a significant portion of their income to charity
- Open-mindedness: Open-minded and flexible, eager to update their beliefs in response to persuasive evidence
- Enthusiasm: Willing and able to commit ~3-4 hours per weekFit: How good a fit are the
I subscribe to CCC's newsletter and these are the latest stories in the newsletters:
I just wanted to provide more context on what they are focusing on.
If you were to organize an effective altruism course around William MacAskill's book Doing Good Better, what additional readings would you give to students to fill in the holes of the book?
This might be slightly off-topic, but you may have some insight into it. If a donor donates money to, for example, global health s/he can find pretty concrete numbers about impact based on GiveWell's estimates or information from specific organizations such as AMF. How can someone donating money to Meta justify those donations quantitatively and via concrete indicators?
1. I prefer "we".
2. I'm not sure what kind of references you are supposed to add here. Should they be accessible to everyone or can books, etc. be included as well? If the latter, then I'd add Daniel Kahneman's book Thinking Fast and Slow to the list. There are good parts about these concepts in the book. (e.g. Kindle version location 4220)
3. To me, it seems that the definitions of "inside view" and "outside view" are not clear enough, whereas the examples are very good. https://www.hybridforecasting.com/ had n...
You didn't mention anything about (a) the risk of becoming less altruistic in the future, (b) increasing your motivation to learn more about effective giving by giving now, and (c) supporting the development of the culture of effective giving. How much the giver learns over time isn't the only consideration. I'm referring to this forum post by listing these other considerations: http://effective-altruism.com/ea/4e/giving_now_vs_later_a_summary/.
I feel that the book contains too much fluff and even these commandments, despite appearing useful, seem to lack enough specificity to be useful. Does anyone have other book recommendations or guidelines for improving one's forecasting and probabilistic thinking? At the end of the day, it's important to actually practice forecasting and thinking probabilistically, but specific information for how to do that would be useful. E.g. how do you actually determine 40/60 and 45/55 or even 43/57 probabilities?
Thanks for putting it on EA Groups Resource Map! I think it'd be better if the link was to the Google Docs document rather than to this forum post, because we might edit it in the future.
If someone can't apply right now due to other commitments, do you expect there to be new roles for generalist research analysts next year as well? What are the best ways one could make oneself a better candidate meanwhile?
Thank you for the questions. Regarding emotions-based advertisement, you might find our recent EURACTIV (a top EU policy media network) op-ed about AI manipulation relevant and interesting: The EU needs to protect (more) against AI manipulation. In it, we invite EU policymakers to expand the definition of manipulation and also consider societal harms from manipulation in addition to individual psychological and physical harms. And here's a bit longer version of that same op-ed.