This seems incredibly exciting! I know several organizations that are looking to spin up their own internal forecasting systems, but can't find a good open-source system to use as a basis. Metaculus is definitely the most advanced forecasting system available, and I am super excited to see whether it will be possible to host local versions of it! 

Are you here to win or the win the race? 
I've been reflecting on the various perspectives within AI governance discussions, particularly within those concerned about AI safety.

One noticeable dividing line is between those concerned about the risks posed by advanced AI systems. This group advocates for regulating AI as it exists today and increasing oversight of AI labs. Their reasoning is that slowing down AI development would provide more time to address technical challenges and allow society to adapt to AI's future capabilities. They are generally cautiously optimistic about international cooperation. I think FLI falls into this camp.

On the other hand, there is a group increasingly focused not only on developing safe AI but also on winning the race, often against China. This group believes that the US currently has an advantage and that maintaining this lead will provide more time to ensure AI safety. They likely think the US embodies better values compared to China, or at least prefer US leadership over Chinese leadership. Many EA organizations, possibly including OP, IAPS, and those collaborating with the US government, may belong to this group.

I've found myself increasingly wary of the second group, tending to discount their views, trust them less, and question the wisdom of cooperating with them. My concern is that their primary focus on winning the AI race might overshadow the broader goal of ensuring AI safety. I am not really sure what to do about this, but I wanted to share my concern and hope to think a bit in the future about what can be done to prevent a rift emerging in the future, especially since I expect the policy stakes will get more and more important in the coming years. 

I don't disagree with your final paragraph, and I think this is worth pursuing generally.

However, I do think we must consider the long-term implications of replacing long-established structures with AI. These structures have evolved over decades or centuries, and their dismantling carries significant risks.

Regarding startups: to me, it seems like their decline in efficiency as they scale is a form of regression to the mean. Startups that succeed do so because of their high-quality decision-making and leadership. As they grow, the decision-making pool expands, often including individuals who haven't undergone the same rigorous selection process. This dilution can reduce overall alignment of decisions with those the founders would have made (a group already selected for decent decision-making quality, at least based on the limited metrics which cause startup survival).

Governments, unlike startups, do not emerge from such a competitive environment. They inherit established organizations with built-in checks and balances designed to enhance decision-making. These checks and balances, although contributing to larger bureaucracies, are probably useful for maintaining accountability and preventing poor decisions, even though they also prevent more drastic change when this is necessary. They also force the decision-maker to take into account another large group of stakeholders within the bureaucracy.

I guess part of my point is that there is a big difference between alignment with the decision-maker and the quality of decision-making.

I think this post misses one of the concerns I have in the back of the mind about AI: How much is current pluralism, liberalism and democracy based on the fact that governance can't be automized yet? 

Currently, policymakers need the backing of thousands of bureaucrats to execute policy, this same bureaucracy provides most of the information to the policymaker. I am fairly sure that this makes the policymaker more accountable and ensures that some truly horrible ideas do not get implemented. If we create AI specifically to help with governance and automate a large amount of this kind of labor, we will find out how important this dynamic is... 

I think this dynamic was better explained in this post. 

Thank you for all the hard work!

I wasn't really surprised by anything here, except for the heavy emphasis on EAGxNetherlands 2024. Is this based on the EAGxRotterdam results? Did that lead to significant community growth? That would slightly surprise me because my sense of these events was that they mostly attract a group that is already pretty active in EA.

I am fairly sure that the JWS means to say that these subgroups are about to / should lose some of their dominance in the EA movement. 

Like Karthik, I don’t really understand what is so terrible about this, but I agree that the California edition is at least strange. It’s interesting how many of the ideas central to EA originate from California. While exploring the origin stories of these ideas is intriguing, I would be much more interested in an issue that explores ideas from far outside that comfort zone and see what they can teach us. 

However, I’m not an editor and don’t think I’d make a good one either 😅 

In the past few weeks, I spoke with several people interested in EA and wondered: What do others recommend in this situation in terms of media to consume first (books, blog posts, podcasts)?

Isn't it time we had a comprehensive guide on which introductory EA books or media to recommend to different people, backed by data?

Such a resource could consider factors like background, interests, and learning preferences, ensuring the most impactful material is suggested for each individual. Wouldn’t this tailored approach make promoting EA among friends and acquaintances more effective and engaging?

Thank you! And Bullet Journal seems like a great new addition, congratulations! 

I will take this post as an opportunity to ask a quick question about the company pledge: I got the feeling that it has been placed a little on the back burner. Or at least, I have never seen it promoted and only found out it existed when I was looking at the list of pledge takers. Is this still a product that is actively receiving attention? If not, why not?