OpenAI and Anthropic probably just aren't aligned to our collective interests. 

These nonprofits "solving the alignment problem" are going to align the AI to its profit seeking creators at best. The plan is an oligarchic utopia, and of course the end of democracy. That's the ultimate goal of AI alignment, the hope that the oligarchs will be sufficiently benevolent that the rest of us won't even miss democracy. 

When the time comes to "spend the money", you and I aren't going to have a significant voice on how it's spent. This ain't a democracy! 

Seems like one variable you're missing is that presumably Chinese models train on different data when it comes to language. Unsurprising Deepseek regurgitates typical English thought when you speak to it in English. How does this change when you speak to Deepseek in Chinese? 

My interest is in transformative reforms and the question of, "how could we do politics better?" How could decision making be improved? Politics is not just about funding some electoral candidate. 

In my opinion, and I think it's been well known for decades, that electoral democracy has woeful limitations on good decision making. This opinion is shared by numerous political scientists. Since the 1950's, Downs Paradox has suggested that it has never been rational, in a self interested sense, to vote. The following 60 years of research has also demonstrated the woeful incompetence of voters. This latest cycle also demonstrates their incompetence, how Donald Trump was elected and his supporters being surprised by his policies - with many immigrant and gen Z voters quickly supporting, and then opposing, Trump. Anybody "in the know" understood what the Trump 2nd term was going to be about. Any surprise is a result of incompetent decision making. 

The status quo of EA seems to be trying to get in on the rat race of electoral politics, to persuade these  ignorant voters. Yet propaganda has never been the strength of EA. I then claim that most potential future efforts of campaigning will have mediocre results. 

The typical fascist is always going to have an easier time. It's just easy to use a tried and true tactic - scapegoat a minority (ie trans people and immigrants), blame them for all our problems, and use them as a vehicle to take power. Fascism takes advantage of our tribal instincts, whereas something like EA demands a rationality that is too expensive to transmit through mass propaganda. 

Is there something better out there? In my opinion yes, and it's called sortition. The premise is simple. Instead of demanding everyone participate in politics, you draw a random sample. With fewer participants, you can now focus resources on the sample. 

Imagine you want to select a president or some other leadership role for government. You could use sortition to construct a representative electoral college. Pay these representatives to do the job. Give them months, or years, to make decisions. 

So whereas the typical voter might use a couple hours to make a voting decision, a full time committee could be making decisions on the time scale of months to years. In other words, a sortition electoral college would use literally thousands of times more man-hours to make a decision compared to the ignorant voter. 

This is no shower thought. Political scientists and advocates have been experimenting with sortition in the form of deliberative polls and citizens assemblies for around 3 decades starting in the 1990s. The results of these experiment in my opinion have been very promising. For example, whereas the American electorate has gone full xenophobic anti immigration, deliberative polls showed opposite results. American participants in citizen deliberation showed moderation and embrace of immigration. However most these expeiements remain in an advisory capacity with no real powers, though Citizens Assemblies for example have been implemented in Paris and Mongolia. 

Moreover, there is essentially no experiments with giving sortition style assemblies real political power. 

In my opinion there is enormous value in supporting experimentation to create a 21st century government capable of making informed decisions at a faster rate. The value added to superior government is likely at several additional percentage points of GDP every year. In other words, the the potential gain is is the multi trillions every year.

Alternatively we can believe that electoral democracy is already at its optimal state and that further efficiency gains would be miniscule. The election of candidates like Donald Trump in my opinion make this viewpoint unlikely. 

Of all the proposals I have heard about to reform democracy, sortition is the ONLY proposal with a viable strategy towards improving democratic decision making. alternatives to democracy in my opinion are mostly terrible and have no accountability mechanism to reign in the meritocrats turned oligarchs. 

Take for example the typical progressive policy that we "need more education". Progressives have been advocating for more education for more than a century. And we are better educated than ever. 

But education doesn't defeat Down's Paradox. Voting demands specific and up to date knowledge. Education can only teach you about the past, and hopefully use propaganda to inspire Americans to overcome their self interest in favor of a civic duty. Yet in our new world of artificial intelligence and LLM's, it's only going to get harder and harder to parse out truth and facts from disinformation. Our best efforts, amateur by definition, just aren't enough anymore. 

I think it's strange to talk about Christianity and then forget about heaven and eternal damnation. It sounds like if you believe in eternal damnation of non Christians, your effective priorities need to be drastically different. Prioritizing the infinite afterlife is infinitely more effective than prioritizing life. 

If Christians wanted to give effectively, they would surely continue giving to Christian branded charities that therefore continue spreading the popularity of Christianity to save the most souls. Their priorities are in conflict with secular organizations. 

We're not even capable of aligning of governments and corporations to humanity. How aligned is the US federal government? How aligned is the EU? how aligned is China? 

We're not capable of aligning the most powerful entities. 

Moreover EA seems disinterested to be in aligning any of these powerful entities to humanity. EA funds little to nothing in for example, improving democratic decision making, which IMO the only viable alignment strategy. The obvious first step in alignment with "humanity" is to bother to even find out what humanity wants. That demands collective preference evaluation. And there already are already existing techniques to do so, little which interests either EA or AI advocates. 

IMO if you were serious about alignment with humanity, you would be spending exorbitant amounts of alignment research on the lower hanging fruit, nation states and corporations, which presumably are less powerful than super intelligence. But you can't even align the mere human, good luck with the superhuman. AI alignment will be impossible as these groups align AI with their own interests. 

But please prove me wrong. Please show me a stronger commitment to democracy, to ensure that any entity can be aligned to "humanity". 

Let's imagine you solve the "alignment problem" tomorrow. So? Exactly who did you solve alignment for? AI aligned to the interests of Elon Musk, Donald Trump, or Vladimir Putin? Or AI aligned with Peter Singer? Or AI aligned to the interests of Google, Meta, TikTok, or Netflix? Or is it alignment with the Democratically determined interests and moral values of the public? 

We've never even solved the "alignment problem" with humans either. The interests of Google might be opposed to your interests. The interests of Vladimir Putin might be opposed to your interests. 

But of course, seeing who is funding AI alignment research, I'll readily bet that the goal is for AI to be aligned with the interests of tech companies and tech billionaires. That's the goal after all. Make AI safe enough so that AI can be profitable. 

>inherently uncontrollable, and thus not a tool.

If AI is an uncontrollable God, then alignment is impossible. Alignment to me implies some sort of control. Uncontrollable superintelligent AI sounds like a horrific idea. There's no guarantees or incentives for God to solve any of our problems. God will work in mysterious ways. God might be cruel and merciless. 

So even if we do succeed in preventing the creation of God, then that means we still need to do everything else EA is concerned about.  

The reason is that AI is at best a tool that could be used for good or bad, or at worst intrinsically misaligned against any human interests.

 Or alternatively AI just isn't solving any of our problems because AI will just be a mere extension of power of states and corporations. Whether moral problems are solved by AI is then up to the whim of corporate or state interests. AI just as well IS being used right now to conquer. The obvious military application has been explored in science fiction for decades. Reducing the cost of deployment of literal killer robots.

Obvious example, look how the profit motive is transforming OpenAI right now. Obvious example, look how AI is "solving" nefarious actors' abilities to create fake news and faked media. 

There is no theory that our glorious AI overlords are going to be effective altruists, or Buddhists, or Kantians, or utilitarians, or whatever else. As far as I'm aware AI may just as likely become a raging kill all humans fascist. 

In my opinion democracy is more likely to be utility maximizing compared to the alternative, oligarchy. In the status quo, the funders provide the moral weights. If your goal is utility maximization, small numbers of funders are more likely to have deviant moral weights compared to the median weights of the public. Their deviancy is less likely to capture maximally satisfactory policy. 

A membership-driven democracy is more likely to have moral weights aligned with the rest of the public. Membership implies multitudes and therefore diversity, which can be a huge advantage in decision making. Having dozens of decision makers is assuredly more diverse than the singular vision of a single funder. 

Like it or not, funders are also biased towards their self interest and may avoid otherwise effective policies. And perhaps this drives the other primary reason to use democracy - it is possible strategy towards building EA funding that relies less on the big funders and more on smaller donors. Democracy allows small donors to exert influence whereas the status quo is set up for large donors to exert influence. 

I think you forget the biggest reason to use democracy -- aligning the moral values of some entity with the moral values of its constituency. 

Whether you like it or not, people's moral values are different, even in EA. Some put much greater value on animal welfare than others for example. There is no universal or absolute way to say that yes, "my values system" is objectively better than yours. 

Democracy is a way to align an organization's moral values with its membership through aggregation. Democracy tends to satisfy more people than less through majority rule. 

Your perspective on democracy in contrast is more in tune to Madisonian or Schumpeter-ian justifications of democracy.