Thank you, a lot of great questions. In response to question (3), some of our work focuses on EU member states as well. Because we are a small team, our ability to cover many member states is limited, but hopefully, with the new hire we can do a lot more on this front as well. If you know anybody suitable, please let us know. For example, we have engaged with Sweden, Estonia, Belgium, Netherlands, France, and a few other countries. Right now, the Presidency of the Council of the EU is held by France, next up are Czechia and Sweden, so work at the member state level in these countries is definitely important. 

Hello Neil, thank you for these questions - I have sent you a DM with some additional answers. 

Hi @JBPDavies, thank you for your questions and happy to comment on examples from my home country.

Our current focus in Europe is on two priorities:  i) strengthened the AI Act, and ii) building support among European countries for a treaty on autonomous weapons systems. For the first priority, we work mainly with the EU institutions. For the second, our focus is with Member State capitals (due to the limited EU influence over security issues, as you rightly point out). 

We regularly evaluate our choice of projects, and are currently conduct an evaluation of our 'policy platform', which we hope to share on our website sometime later this year. Nevertheless, we current focus on the AI Act because it is the first piece of legislation on AI by a major regulator anywhere, and because it could set regulators around the world on a certain path that could impact how we deal with increasingly powerful AI systems. 

Our focus on autonomous weapons is partly driven by the Asilomar principles, which FLI helped coordinate and where the principle of avoiding an AI arms race (#18) got most support from attending experts working on beneficial AI. This line of effort also helps us understand global coordination problems, because autonomous weapons may be an early example of AI that we would want to regulate. 

In reply to your clarifying questions, and as I mentioned earlier, our advocacy on autonomous weapons is mainly targeted at the Member State level (please do note that this will not be the main focus of the job for which we are currently advertising!). We are a small team, but we do build coalitions with other civil society organisations, businesses and academics where we can. A recent example of this is the open letter we coordinated among German AI researchers in which they called for a more progressive stance on this issue in the German coalition agreement (https://autonomewaffen.org/FAZ-Anzeige/). 

Hope this is helpful, but please let me know if you have further questions or if I am unclear.

Thank you for the questions. I think that the biggest bottleneck right now is that very few people work on the issues we are interested in (listed here). We are trying to contribute to this by hiring a new person, but the problems are vast and there's a lot more room for additional people. Another issue is lack of policy research that would consider the longer-term implications but would at the same time be very practical. We are happy that in addition to the Future of Life Institute, a few other organizations such as Centre for the Governance of AI, Centre for Long-Term Resilience, and some other ones are contributing more here or starting to do so. I'm not sure about the next 5-10 years, so I'll leave it to someone else who might have some tentative answers. 

Thank you, these are some really big questions! Most of them are beyond what we work on, so I'm happy to leave these to other people in this community and have them guide our own work. For example, the Centre for Long-Term Resilience published the Future Proof report in which they refer to a survey where the median prediction of scientists is that general human-level intelligence will be reached around 35 years from now. 

I'll try to answer the last question about where our opinions might differ. Many academics and policymakers in the EU probably still don't think much about the longer-term implications of AI and don't think that AI progress can have such significant impact (negative or positive) that we do or don't think that it is reasonable to focus on it right now. That said, I don't think that there is necessarily a very big gap between us in practice. For example, many people who are interested in bias, discrimination, fairness, and other issues that are already prevalent, can also be concerned about more general purpose AI systems that will become more available on the market in the future, as these systems can present even bigger challenges and have more significant consequences in terms of bias, discrimination, fairness, etc. In the paper On the Opportunities and Risks of Foundation Models, it was stated that, "Properties of the foundation model can lead to harm in downstream systems. As a result, these intrinsic biases can be measured directly within the foundation model, though the harm itself is only realized when the foundation model is adapted, and thereafter applied."

Thank you for the questions. Regarding emotions-based advertisement, you might find our recent EURACTIV (a top EU policy media network) op-ed about AI manipulation relevant and interesting: The EU needs to protect (more) against AI manipulation. In it, we invite EU policymakers to expand the definition of manipulation and also consider societal harms from manipulation in addition to individual psychological and physical harms. And here's a bit longer version of that same op-ed.