Policy

# 80

Cross-posted from Cold Takes

I'm interested in the topic of ideal governance: what kind of governance system should you set up, if you're starting from scratch and can do it however you want?

Here "you" could be a company, a nonprofit, an informal association, or a country. And "governance system" means a Constitution, charter, and/or bylaws answering questions like: "Who has the authority to make decisions (Congress, board of directors, etc.), and how are they selected, and what rules do they have to follow, and what's the process for changing those rules?"

I think this is a very different topic from something like "How does the US's Presidential system compare to the Parliamentary systems common in Europe?" The idea is not to look at today's most common systems and compare them, but rather to generate options for setting up systems radically different from what's common today.

I don't currently know of much literature on this topic (aside from the literature on social choice theory and especially voting methods, which covers only part of the topic). This post describes the general topic and why I care, partly in the hopes that people can point me to any literature I've missed. Whether or not I end up finding any, I'm likely to write more on this topic in the future.

Outline of the rest of the piece:

• I'll outline some common governance structures for countries and major organizations today, and highlight how much room there is to try different things that don't seem to be in wide use today. More
• A short-term, tangible need: over the last several years, I've spoken with several (more than 3) organizations that feel no traditional corporate governance structure is satisfactory, because the stakes of their business are too great and society-wide for shareholder control to make sense, yet they are too early-stage and niche (and in need of nimbleness) to be structured like a traditional government. An example would be an artificial intelligence company that could end up with a normal commercial product, or could end up bringing about the most important century of all time for humanity. I wish I could point them to someone who was like: "I've read all of, and written much of, the literature on what your options are. I can walk you through the pros and cons and help you pick a governance system that balances them for your needs."
• A small probability of a big future win. The world today has lots of governments, but they seem to mostly follow a very small number of basic governance templates. At some point, there will be new states with new Constitutions - maybe via space settlements, maybe via collapse of existing states, etc. - but I expect these moments to be few and far between. A significant literature and set of experts on "ideal governance" could lead to a radically different kind of state government, potentially with radically different policies that the rest of the world could learn from.
• A weird, out-of-left-field application. Some of my interest in this topic actually comes via my interest in moral uncertainty: the question of what it's ethical to do when one is struggling between more than one theory of ethics, with radically different implications. This is hard to explain, but I try below.
• I'll describe a bit more what I think literature on this question could look like (and what already exists that I know of), partly to guide readers who might be able to help me find more.

## Common governance structures today

All of these are simplified; I'm trying to illustrate the basic idea of what questions "ideal governance" is asking.

• A standard (e.g., public) corporation works like this: it has shareholders, assigned one vote per share (not per person), who elect a board of directors that governs by majority. The board generally appoints a CEO that it entrusts with day-to-day decisions. There is a "constitution" of sorts (the Articles of Incorporation and bylaws) and a lot more wrinkles in terms of how directors are selected, but that's the basic idea.
• A standard nonprofit is like a corporation, but entirely lacking the shareholder layer - it's governed directly by the board of directors. (I find something weird about a structure this simple - a simple board majority can do literally anything, even though the board of directors is often a somewhat random assortment of donors, advisors, etc.)
• The US federal government is a lot more complex. It splits authority between the House of Representatives, the Senate, the Presidency and the Supreme Court, all of which have specific appointment procedures, term limits, etc. and are meta-governed by a Constitution that requires special measures to change. There are lots of specific choices that were made in designing things this way, and lots of things that could've been set up differently in the 18th century that would probably still matter today.
• Other democracies tend to have governments that differ in a lot of ways (e.g.), while being based on broadly similar principles: voters elect representatives to more than one branch of government, which then divide up (and often can veto each other on) laws, expenditures, etc.
• When I was 13, the lunch table I sat at established a Constitution with some really strange properties that I can't remember. I think there was a near-dictatorial authority who rotated daily, with others able to veto their decisions by assembling supermajorities or maybe singing silly songs or something.

In addition to the design choices shown in the diagrams, there are a lot of others:

• Who votes, how often, and what voting system is used?
• How many representatives are there in each representative body? How are they divided up (one representative per geographic area, or party-list proportional representation, or something else)?
• What term limits exist for the different entities?
• Do particular kinds of decisions require supermajorities?
• Which restrictions are enshrined in a hard-to-change Constitution (and how hard is it to change), vs. being left to the people in power at the moment?

One way of thinking about the "ideal governance" question is: what kinds of designs could exist that aren't common today? And how should a new organization/country/etc. think about what design is going to be best for its purposes, beyond "doing what's usually done"?

For any new institution, it seems like the stakes are potentially high - in some important sense, picking a governance system is a "one-time thing" (any further changes have to be made using the rules of the existing system1).

Perhaps because of this, there doesn't seem to be much use of innovative governance designs in high-stakes settings. For example, here are a number of ideas I've seen floating around that seem cool and interesting, and ought to be considered if someone could set up a governance system however they wanted:

• Sortition, or choosing people randomly to have certain powers and responsibilities. An extreme version could be: "Instead of everyone voting for President, randomly select 1000 Americans; give them several months to consider their choice, perhaps paid so they can do so full-time; then have them vote."
• The idea is to pick a subset of people who are both (a) representative of the larger population (hence the randomness); (b) will have a stronger case for putting serious time and thought into their decisions (hence the small number).
• It's solving a similar problem that "representative democracy" (voters elect representatives) is trying to solve, but in a different way.
• Proportional decision-making. Currently, if Congress is deciding how to spend $1 trillion, a coalition controlling 51% of the votes can control all$1 trillion, whereas a coalition controlling 49% of the votes controls \$0. Proportional decision-making could be implemented as "Each representative controls an equal proportion of the spending," so a coalition with 20% of the votes controls 20% of the budget. It's less clear how to apply this idea to other sorts of bills (e.g., illegalizing an activity rather than spending money), but there are plenty of possibilities.2
• Quadratic voting, in which people vote on multiple things at once, and can cast more votes for things they care about more (with a "quadratic pricing rule" intended to make the number of votes an "honest signal" of how much someone cares).
• Reset/Jubilee: maybe it would be good for some organizations to periodically redo their governance mostly from scratch, subject only to the most basic principles. Constitutions could contain a provision like "Every N years, there shall be a new Constitution selected. The 10 candidate Constitutions with the most signatures shall be presented on a ballot; the Constitution receiving the most votes is the new Constitution, except that it may not contradict or nullify this provision. This provision can be prevented from occurring by [supermajority provision], and removed entirely by [stronger supermajority]."
• More examples in a footnote.3

If we were starting a country or company from scratch, which of the above ideas should we integrate with more traditional structures, and how, and what else should we have in our toolbox? That's the question of ideal governance.

## Why do I care?

I have one "short-term, tangible need" reason; one "small probability of a big future win" reason; and one "weird, out-of-left-field" reason.

A short-term, tangible need: companies developing AI, or otherwise aiming to be working with huge stakes. Say you're starting a new company for developing AI systems, and you believe that you could end up building AI with the potential to change the world forever.

• The standard governance setup for a corporation would hand power over all the decisions you're going to make to your shareholders, and by default most of your shares are going to end up held by people and firms that invested money in your company. Hopefully it's clear why this doesn't seem like the ideal setup for a company whose decisions could be world-changing. A number of AI companies have acknowledged the basic point that "Our ultimate mission should NOT just be: make money for shareholders," and that seems like a good thing.
• One alternative would be to set up like a nonprofit instead, with all power vested in a board of directors (no shareholder control). Some issues are that (a) this cuts shareholders out of the loop completely, which could make it pretty hard to raise money; (b) according to me at least, this is just a weird system of governance, for reasons that are not super easy to articulate concisely but I'll take a shot in a footnote4 (and possibly write more in the future).
• Another alternative is a setup that is somewhat common among tech companies: 1-2 founders hold enough shares to keep control forever, so you end up with essentially a dictatorship. This also ... leaves something to be desired.
• Or maybe a company like this should just set up more like a government from the get-go, offering everyone in the world a vote via some complex system of representation, checks and balances. But this seems poorly suited to at least the relatively early days of a company, when it's small and its work is not widely known or understood. But then, how does the company handle the transition from the latter to the former? And should the former be done exactly in the standard way, or is there room for innovation there?

Over the last several years, I've spoken with heads of several (more than 3) organizations that struggle between options like the above, and have at least strongly considered unusual governance setups. I wish I could point them to someone who was like: "I've read all of, and written much of, the literature on what your options are. I can walk you through the pros and cons and help you pick a governance system that balances them for your needs."

But right now, I can't, and I've seen a fair amount of this instead: "Let's just throw together the best system we can, based mostly on what's already common but with a few wrinkles, and hope that we figure this all out later." I think this is the right solution given how things stand, but I think it really does get continually harder to redesign one's governance as time goes on and more stakeholders enter the picture, so it makes me nervous.

Similar issues could apply to mega-corporations (e.g., FAANG) that are arguably more powerful than what the standard shareholder-centric company setup was designed for. Are there governance systems they could adopt that would make them more broadly accountable, without copying over all the pros and cons of full-blown representative democracy as implemented by countries like the US?

A small probability of a big future win: future new states. The world today has lots of governments, but they seem to mostly follow a very small number of basic governance templates (e.g., I believe you see almost none of the things I listed above), and probably relatedly, there seems to be remarkably little variety and experimentation with policy. Policies that many believe could be huge wins - such as dramatically expanded immigration, land value taxation, "consumer reports"-style medical approvals,5 drug decriminalization, and charter cities - don't seem to have gotten much of a trial anywhere in the world.

At some point, there will be new states with new Constitutions - maybe via space settlements, maybe via collapse of existing states, etc. - but I expect these moments to be few and far between.

By default I expect future Constitutions to resemble present ones an awful lot. But maybe, at some future date, there will be a large "ideal governance" literature and some points of expert consensus on innovative governance designs that somebody really ought to try. That could lead to a radically different kind of state government, potentially with radically different policies that the rest of the world could learn from.

An out-of-left-field application for "ideal governance." This is going to veer off the rails, so remember to skip to the next section if I lose you.

Some of my interest in this topic actually comes via my interest in moral uncertainty: the question of what it's ethical to do when one is struggling between more than one theory of ethics, with radically different implications.

For example, there are arguments that our ethical decisions should be dominated by concern for ensuring that as many people as possible will someday get to exist. I really go back and forth on how much I buy these arguments, but I'm definitely somewhere between 10% convinced and 50% convinced. So ... say I'm "20% convinced" of some view that says preventing human extinction6 is the overwhelmingly most important consideration for at least some dimensions of ethics (like where to donate), and "80% convinced" of some more common-sense view that says I should focus on some cause unrelated to human extinction.7 How do I put those two together and decide what this means for actual choices I'm making?

The closest thing I've seen to a reasonable-seeming answer is the idea of a moral parliament: I should act as though I'm run by a Parliament with 80 members who believe in "common-sense" ethics, and 20 members who believe in the "preventing extinction is overwhelmingly important" idea. But with default Parliament rules, this would just mean the 80 members can run the whole show, without any compromise with the 20.

And so, a paper on the "moral parliament" idea tries to make it work by ... introducing a completely new governance mechanism that I can't find any other sign of someone else ever talking about, "proportional chances voting" (spelled out in a footnote).8 I think this mechanism has its own issues,9 but it's an attempt to ensure something like "A coalition controlling 20% of the votes has 20% of the effective power, and has to be compromised with, instead of being subject to the tyranny of the majority."

My own view (which I expect to write more about in the future) is that governance is roughly the right metaphor for "moral uncertainty": I am torn by multiple different sides of myself, with different takes on what it means to be a good person, and the problem of getting these different sides of myself to reach a decision together is like the problem of getting different citizens (or shareholders) to reach a decision together. The more we can say about what ideal governance looks like, the more we can say about how this ought to work - and the better I expect this "moral parliament"-type idea to end up looking, compared to alternatives.10

## The literature I'm looking for

Ideal governance seems like the sort of topic for which there should be a "field" of "experts," studying it. What would such study look like? Three major categories come to mind:

Brainstorming ideas such as those I listed above - innovative potential ways of solving classic challenges of governance, such as reconciling "We want to represent all the voters" with "We want decisions to be grounded in expertise and high engagement, and voters are often non-expert and not engaged."

I've come across various assorted ideas in this category, including quadratic voting, futarchy, and proportional chances voting, without seeing much sign that these sit within a broader field that I can skim through to find all the ideas that are out there.

Economics-style theory in which one asks questions like: "If we make particular assumptions about who's voting, what information they have and lack, how much they suffer from bounded rationality, and how we define 'serving their interests' (see below), what kind of governance structure gets the best outcome?"

Social choice theory, including on voting methods, tackles the "how we define 'serving their interests'" part of this. But I'm not aware of people using similar approaches to ask questions like "Under what conditions would we want 1 chamber of Congress vs. 2, or 10? 100 Senators vs. 500, or 15? A constitution that can be modified by simple majority, vs. 2/3 majority vs. consensus? Term limits? Etc. etc. etc."

Empirical research (probably qualitative): Are there systematic reviews of unusual governance structures tried out by companies, and what the results have been? Of smaller-scale experiments at co-ops, group houses and lunch tables?

To be clear, I think the most useful version of this sort of research would probably be very qualitative - collecting reports of what problems did and didn't come up - rather than asking questions like "How does a particular board structure element affect company profits?"

One of the things I expect to be tricky about this sort of research is that I think a lot of governance comes down to things like "What sorts of people are in charge?" and "What are the culture, expectations, norms and habits?" A setup that is "officially" supposed to work one way could evolve into something quite different via informal practices and "soft power." However, I think the formal setup (including things like "what the constitution says about the principles each governance body is supposed to be upholding") can have big effects on how the "soft power" works.

If you know where to find research or experts along the lines of the above, please share them in the comments or using this form if you don't want them to be public.

I'll likely write about what I come across, and if I don't find anything new, I'll likely ramble some more about ideal governance. So either way, there will be more on this topic!

## Footnotes

1. Barring violent revolution in the case of countries.

2. An example would be the "proportional chances voting" idea described here

• Proxying/liquid democracy, or allowing voters to transfer their votes to other voters. (This is common for corporations, but not for governments.) This could be an alternative or complement to electing representatives, solving a similar problem (we want lightly-engaged voters to be represented, but we also want decisions ultimately made using heavy engagement and expertise). At first glance it may seem to pose a risk that people will be able to "buy votes," but I don't actually think this is necessarily an issue (proxying could be done anonymously and on set schedules, like other votes).
• Soft term limits: the more terms someone has served, the greater a supermajority they need to be re-elected. This could be used to strike a balance between the advantages of term limits (avoiding "effectively unaccountable" incumbents) and no-term-limits (allowing great representatives to keep serving).
• Formal technocracy/meritocracy: Using hard structures (rather than soft norms) to assign authority to people with particular expertise and qualifications. An extreme example would be futarchy, in which prediction markets directly control decisions. A simpler example would be structurally rewarding representatives (via more votes or other powers) based on assessments of their track records (of predictions or decisions), or factual understanding of a subject. This seems like a tough road to go down by default, as any mechanism for evaluating "track records" and "understanding" can itself be politicized, but there's a wide space of possible designs.
3. Most systems of government have a sort of funnel from "least engaged in day to day decisions, but most ultimately legitimate representatives of whom the institution is supposed to serve" (shareholders, voters) to "most engaged in day to day decisions, but ultimately accountable to someone else" (chief executive). A nonprofit structure is a very short funnel, and the board of directors tends to be a somewhat random assortment of funders, advisors, people who the founders just thought were cool, etc. I think they often end up not very accountable (to anyone) or engaged in what's going on, such that they have a hard time acting when they ought to, and the actions they do take are often kind of random.

I'm not saying there is a clearly better structure available for this purpose - I think the weirdness comes from the fact that it's so unclear who should go in the box normally reserved for "Shareholders" or "Voters." It's probably the best common structure for its purpose, but I think there's a lot of room for improvement, and the stakes seem high for certain organizations.

4. Context in this Marginal Revolution post, which links to this 2005 piece on a "consumer reports" model for the FDA

5. Or "existential catastrophe" - something that drastically curtails humanity's future, even if it doesn't drive us extinct.

6. This isn't actually where I'm at, because I think the leading existential risks are a big enough deal that I would want to focus on them even if I completely ignored the philosophical argument that the future is overwhelmingly important.

7. Let's say that 70% of the Parliament members vote for bill X, and 30% vote against. "Proportional chance voting" literally uses a weighted lottery to pass bill X with 70% probability, and reject it with 30% probability (you can think of this like rolling a 10-sided die, and passing the bill if it's 7 or under).

A key part of this is that the members are supposed to negotiate before voting and holding the lottery. For example, maybe 10 of the 30 members who are against bill X offer to switch to supporting it if some change is made. The nice property here is that rather than having a "tyranny of the majority" where the minority has no bargaining power, we have a situation where the 70-member coalition would still love to make a deal with folks in the minority, to further increase the probability that they get their way.

Quote from the paper that I am interpreting: "Under proportional chances voting, each delegate receives a single vote on each motion. Before they vote, there is a period during which delegates may negotiate: this could include trading votes on one motion for votes on another, introducing novel options for consideration within a given motion, or forming deals with others to vote for a compromise option that both consider to be acceptable. The delegates then cast their ballots for one particular option in each motion, just as they might in a plurality voting system. But rather than determining the winning option to be the one with the most votes, each option is given a chance of winning proportional to its share of the votes."

8. What stops someone who lost the randomized draw from just asking to hold the same vote again? Or asking to hold a highly similar/related vote that would get back a lot of what they lost? How does that affect the negotiated equilibrium?

9. Such as "maximize expected choice-worthiness," which I am not a fan of for reasons I'll get to in the future.

# 80

New Comment

I've been interested in the area of improving governance for a long time! On a societal level, there are some organizations and efforts in the space like One Project and RadicalXChange. Unfortunately I'm not aware of research that evaluates the efficacy of various governance models. I've been thinking about doing that for quite a few years. Doing that research is a high priority on Roote's backlog. We haven't tried getting funding for it yet, but it is somewhat related to our funded Civic Abundance and Web3 & Society initiatives. For example, for Web3 & Society, given the proliferation of alternative governance methods in Web3 as well as the transparent nature of decisions and performance results with DAOs, we can directly assess the efficacy of various DAO governance models.

Rhys (also from Roote) here. Agree with Brendon that there isn't too much literature evaluating the "efficacy of various governance models". Some links you may want to look into, Holden:

• Lots of governance experiments are happening with DAOs in crypto. See Vitalik's back and forth here: https://twitter.com/VitalikButerin/status/1442039126606311427
• Or my response here. I find it helpful to visualize these systems:  https://twitter.com/RhysLindmark/status/1446276859109335040 and https://www.rhyslindmark.com/popper-criterion-for-politics/ . Those pieces contain lots of political economy books like The Dictator's Handbook. https://www.goodreads.com/en/book/show/11612989
• More crypto stuff: https://gnosisguild.mirror.xyz/OuhG5s2X5uSVBx1EK4tKPhnUc91Wh9YM0fwSnC8UNcg. These are interchangeable "Modules" that DAOs can use like DeGov. https://otherinter.net/research/ is doing research on DAO governance as well.
• On the non-crypto side, Rob Reich has great thoughts on this. I found this convo between him and Stuart Russell re legitimacy and AI governance helpful. (49:30)
• Worth differentiating how much groups disagree on what should be (goals) vs. what is (current state). https://twitter.com/RhysLindmark/status/1294107741246517248
• This feels close to the work Ian David-Moss et al are doing here https://forum.effectivealtruism.org/tag/effective-institutions-project
• Many of the governance issues take the form of one of Meadow's "system traps" https://bytepawn.com/systems-thinking.html#:~:text=Thinking%20in%20Systems%2C%20written%20by,furnace%20to%20a%20social%20system.
• In the spirit of your final experimental point: Long term, I do think a lot of this will just be understood (and computationally modeled) as social groups (bounded by a Markov Blanket) abiding by the Free Energy Principle / Active Inference with Bayesian generative models, co-evolving into evolutionarily stable strategies. But we're not there yet! 🙂

Beyond social choice theory, not sure there's a better field you're looking for. Maybe Political Economy, Public Choice Theory, or Game Theory? ¯\_(ツ)_/¯

Anywho, good luck and excited to see what you unearth!

I think the problem is not so much to find the perfect governance system (which changes over time and with context) ...

... but how to get there from here?

In business schools this is addressed through the research category 'Management of Change'.

In politics, why it's easier in France is a perennial topic.

I believe that governance is a technology. Thus, while there may be no "perfect" governance system, humanity's knowledge on it will improve greatly over time. That will improve the default governance models used (right now, representative democracy is a very common default with company shareholders, most developed countries, etc.) as well as humanity's ability to customize governance models to particular situation. Since representative democracy is so commonplace, I think making default models better will produce most of the benefit, rather than the adapting to the context as you mention.

Regarding getting there, as indicated in Holden's article, governance can be applied to many human systems, not just a government. Governments change, of course, but organizations change faster and emerge at a higher rate. Take public benefit corporations (PBCs) for example. Delaware (the most popular state for incorporation), passed  PBC legislation in 2013 and we already have PBCs IPOing.

There are also very creative ways to influence governments with technology. For example, in Taiwan, while the governance model hasn't changed, the government is deploying technologies like Polis to improve democracy, using it to effectively come up with policy proposals for potentially contentious issues that improve society and enjoy high consensus. I think that developing "add-ons" to entrenched governance models is a decent strategy, and it's one of the routes that our Civic Abundance project is taking.

My credentials: I am an investor by profession and have experience negotiating governance structures. I have been a director of 3 private companies and a trustee of 4 non-profits.

Governance is often not ideal. That's because it is a weird confluence of fitting within the law (often modern laws layered over common laws that don't make much sense today), relationships and negotiation. For example, you pose the question about whether unusual governance structures have been tried by companies. In general, they haven't because they aren't legally possible.

In terms of structuring like a democracy, I don't think democracies deal well with technical and minor issues. I say "minor" issue because if everything is going well, people will probably not consider it important. It's also impossible for future people to participate in today's democracy by definition.

The final point I'd make is that time horizons are important here. Many organisations struggle to manage both the immediate term and the long-term in the same framework. Within a corporate, it is good organisational practice to divide those responsibilities to a certain extent.

On social choice theory, I think it's important to distinguish between decisions that have to be made (typically handled by the executive, e.g. there needs to be a new Chair of the Federal Reserve) and decisions about changes (typically handled by the legislature, e.g. we could improve the law on bank regulation). Budgets typically require approval of the legislature, but are really something that has to happen (the status quo of the government having no money used to be a reasonable option but is not in the modern day).

Some minor comments on the piece:

• I know you've tried to simplify things, but governance of for-profit corporations is a lot more complex than you make out. Board members are not as accountable to shareholders as you would expect, e.g. AGM votes often being non-binding, adoption of poison pills. There are also normally minority protections, e.g. takeover rules for public companies, investor vetoes in private companies. CEOs typically serve on the Board (which is different to non-profits), are sometimes also the Chair and can be the controlling shareholder, which adds a lot of additional dynamics. I think it's also very important to consider not just the legal governance but the practical governance, e.g. the Chairman has significantly more influence than other board members even though they all have 1 vote each. Soft power is very important.
• With non-profits, I have observed a significant difference between UK and US boards. UK boards are typically filled based on expertise, whereas many US board are filled with donors and fundraisers. This is not a legal difference, but does affect the dynamic a lot.
• Non-profits can also have members that act a bit like shareholders. This is most common for membership organisations, e.g. sports clubs, mutual interest societies, but it's also possible for non-profits to have another organisation as its sole member, i.e. a bit like a subsidiary.

This comment is really insightful. It is short but has a huge amount of content. It draws from experience, expertise and reality, maybe why it can be concise and still accurate. Thanks a lot.

One issue that I've rarely seem addressed directly is changing population sizes. Historically a lot of systems (e.g. one-man-one-vote democracy) have relied on the fact that adding voters is slow and expensive. But with reduced travel costs, artificial wombs, and eventually the possibility of digitally replicating people an arbitrary number of times, this assumption could cease to be the case. At that point dividing up rights on a per capita basis looks more like an invitation to abuse.

This issue has historically been addressed by corporations through allocating votes proportionally to shares, not people, or by coins through proof-of-[work/stake]. In the future it could be as easy to multiple people as it is to multiple legal entities, email addresses or wallets.

Can you explain how this viewpoint is substantively different than say, the Greek concept of "Aristocracy", which is seen as highly positive by the classical Greeks?

The point of my question is to understand the net contribution of this idea (other the mechanics of the additional step of reducing of vote weights not to zero, and putting this all in a spreadsheet). It also suggests we can just examine the related literature, which should be pretty large?

Hmm, I'm not sure I quite have the intuition for what 'Aristocracy' means in your link - it seems the Greek definition differs from the Hobbesian one for example.

But I think the answer is that I am outlining a problem, and there are many different potential solutions. To use the crypto example, both proof-of-stake and proof-of-work could be valid solutions, even though they are quite distinct. So while perhaps Aristocracy might be one solution I'm not sure it would be the only one, unless defined very broadly.

One challenge you might find with examining the literature in my space is a lack of prioritisation – in particular I think this leads to an overly strong focus on voting mechanisms above other issues.

To me it feels like how animal charities focus mostly on  pets*. Sure pets are the most obvious animals that we engage with in our daily life, but the vast majority of animal suffering happens in farms. Sure voting is the the most obvious part of the system that we engage with in our daily life, but the vast majority of system improvements are more behind the scenes.

I don’t think voting mechanisms are more important than other governance issues such as: constitutions, the ability to kick out corrupt leaders, judiciary independence, the way political leaders pick ministers, or about 100 other aspects of governance. I would be interested in someone making the case for voting mechanisms being more important than other aspects of governance but I have honestly never seen anyone even trying to prioritise along these lines.

* and donkeys. So much money donated to donkeys. I have never really understood why.

I have done a lot of reading related to governance in political science, organizational studies, and governance studies themselves, so lots I could mention, and happy to help.

The contemporary major reference on how "informal ties" beat "formal ties" is https://www.jstor.org/stable/2780199

If I had to mention just two current scholars on governance, considering what you described before, I would point you to https://www.law.columbia.edu/faculty/charles-f-sabel and https://polisci.berkeley.edu/people/person/christopher-ansell

From Economics, the best stuff in my view comes from Ostrom, Nobel 2009. I would start here https://www.jstor.org/stable/27871226, even if the deep end of the pool.

For "empirical research"

The thing I have found most useful is the work of the UK's Institute for Government. Both their reports and podcasts. I often find I pick up useful things on ideal system design like it may well be that a mix of private and public services are better than 100% one or the other as can compare and see which is working better and take best practice from both (this was from their empirical work on prisons). The caveat is that if you are not into UK policy there may be too much context to wade through to reach the interesting conclusions. But worth a look.

Also when looking into the ideal governance structures for AI companies I think I found it very useful to look at the nuclear system. Civil nuclear risk is surprisingly (compared to other areas of policy I have experience of) well managed at both a international level, a regulatory level (in the UK), and a company level. And it is a hard topic because the aim is to stop the one very bad and very unlikely scenario of a major meltdown. Nuclear is obviously more understood than AI alignment but interesting nevertheless. Not sure the best reading on this but perhaps guidance notes form the IAEA or the ONR.

[I have thoughts to add on brainstorming but might have to add that at another time]

A small probability of a big future win. The world today has lots of governments, but they seem to mostly follow a very small number of basic governance templates. At some point, there will be new states with new Constitutions - maybe via space settlements, maybe via collapse of existing states, etc. - but I expect these moments to be few and far between. A significant literature and set of experts on "ideal governance" could lead to a radically different kind of state government, potentially with radically different policies that the rest of the world could learn from.

I think you could rework this paragraph a bit and make it work for big online communities too. Mostly fairly static and undifferentiated, with occasional tectonic shifts representing opportunities for change, and important due to being upstream of lots of stuff. The nature of the governance problem is different, but I think there are many hypothetical approaches on a continuum between social media & states as they're governed now, and many ideas which are applicable to both (e.g. quadratic voting, prediction markets).

IMO the most interesting aspect of crypto (by a long shot) is the lab it creates for mechanism/incentive/governance design.  If there's any major innovation in governance in the near future, my bet is it comes out of crypto.  We already see  early signs of this with GitCoin using quadratic funding and Vitalik's collaboration with Glen Weyl and RadicalXChange (as Brendon mentioned).

Similarly, while  I could envision a lot of (really bad?) unintended consequences here, Balaji Srinivasan's idea of "network states" is interesting to the extent it leads to a competitive marketplace for governance structures.  Note especially his emphasis on the "right to exit".  Voting with your feet (migrating out) has a lot more impact than voting with votes or dollars.  He probably has a twitter thread that explains it succinctly but there's something mesmerizing about  listening to him in this 4 hour marathon podcast on Tim Ferris' show

Audrey Tang, the digital minister of Taiwan, is doing some very interesting experiments in governance: 80000 Hours has a great, recent podcast with her

RE:why different voting models gets all the attention, without having thought about this deeply, my gut instinct is "if you could solve one mechanism of governance that solves all the others" voting would be the leading candidate.

PS this is my first post on the EA Forum.  If I've broken any explicit rules, cultural norms, or even committed any minor faux pas, please let me know!  I'd be grateful

Empirical research (probably qualitative): Are there systematic reviews of unusual governance structures tried out by companies, and what the results have been? Of smaller-scale experiments at co-ops, group houses and lunch tables?

Check out the Community Rules governance toolkit. It's intended for communities (which probably leans towards the small side of the governance spectrum) and describes eight governance frameworks with three brief case studies of practioners, ranging from  Ancient Athens to the Facebook Oversight Board,  that have used each.  Four of the governance frameworks have already been described in the post:

• Elected board
• Self-appointed board
• Benevolent dictator (matches founder keeping control)
• Jury (matches sortition?)

And there are four others:

• Circles
• Do-ocracy
• Petition (basically governance  only using ballot initiatives)
• Consensus

I don't know much about the  groups that put together the toolkit, but they would probably be worth contacting to find further expertise:

CommunityRule is a project of the Media Enterprise Design Lab at the University of Colorado Boulder, in collaboration with the Metagovernance Project.

There are a lot of approaches in software engineering that are really answers to governance problems. These got started, perhaps, with https://agilemanifesto.org/.

While the Manifesto only states a few generic principles, there exist more applied approaches like Scrum which mandate specific roles, processes, and decision mechanisms.

I'm far from being an expert here, so I won't add many more links... ask your favorite "Agile Coach"; they will point you to a lot of research into which of these approaches work and which ones don't, which exists because questions of software engineering governance have direct impact on the success of organizations. Software engineering is also a space where there is a lot more change and innovation than in political governments.

Proportional Chances Voting is basically equivalent to a mechanism where one vote is selected at random to be the deciding vote, as Newberry and Ord register in a footnote (they refer to it as "Random Dictator"; I've also seen it described as "lottery voting"). Newberry and Ord do say that Proportional Chances is supposed to be different because of the negotiation period, but I don't see how Random Dictator is incompatible with negotiation.

Anyway, some of the literature on this mechanism may be of interest here, given footnotes 8-9. This paper proposes such a mechanism, defends its plausibility: Saunders, Ben. “Democracy, Political Equality, and Majority Rule.” Ethics 121, no. 1 (2010): 148–77. I haven't read any good papers which offer interesting critiques of Saunders, but the paper seems to be influential, so maybe someone else knows  of one?

As for calling new votes (footnote 9 of this post), votes could be scheduled by a separate body than that doing the voting, or could be scheduled by some regularised rule. For instance, in Kira's Dinner, the thought experiment in the Newberry and Ord paper, votes on what Kira should eat are scheduled according to the regular rhythm of Kira getting hungry. The voters take the votes as given -- I think there are usually similar ways to establish systems like this in real-world multi-person organizations.

Yudkowsky has been writing about how he'd set up a government in his story Mad Investor Chaos, which is, in many ways, a story about human coordination, and how the "obvious" way to do it is so completely different from anything Earth is doing.

This has nobody studying it AFAIK.

• I can try finding that part for you
• Also see this bounty.

My imperfect memory:

TL;DR: I don't think any of these suggestions will solve your problem for AI governance; Take this as "Yudkowsky has lots of unusual ideas about the topic of governance specifically"

# "Governance" for any org

The main principle is "every decision needs to have a specific person tagged as responsible for it".

There's a long rant about how if there are 2 (or more) people "responsible" for a decision or making it "together", then it's as if there are zero people responsible for it.

The responsible person needs

• "organizational eyes": visibility to all the relevant information
• "organizational fingers": the ability to do the required actions

If a strange/unexpected case comes up, one can "throw an exception" to the person one step higher up in the hierarchy. For example, if I got seemingly contradicting orders, or if I'm missing information, or skill for a complicated decision, or if for some other reason I am not willing to take responsibility for what I'm going to decide.

All decisions are logged. There's a process for reviewing the logs.

# "Governance" for a country specifically

Elections:

1. Delegative voting
2. With several "levels": citizens give their votes to delegates, who give their votes to higher-order-delegates , who vote for the people in government
3. Voting starts linar but at some point (around 200 votes?) becomes quodratic
4. A main principle driving this is that each person can talk directly to a representative who has an incentive to listen

Easy to overthrow:

• There are a ton of safety measures that ensure that the citizens are incredibly able and ready to overthrow the government, and that the government knows this. For example:
• There is a "oops let's overthrow the government" festival once a year
• The military can't grow stronger than what the citizens can beat by themselves

Slow

• The government is optimized for "not making mistakes" and absolutely not for "being able to move quickly and break things". For example:
• Every law needs someone from the government to officially take responsibility from it (meaning if something goes wrong with the intended consequences of the law - people complain to that person), including reading the law out loud. If nobody is willing to take that responsibility, the law is voided. If the law is too long, people will be more annoyed at having to read them.

Uses conditional prediction markets to chose policies.

Against [certain] corruption:

• If you serve any role in government, you can't serve any other role in the future. This makes sure that the "higher" elected officials can't offer "jobs" to the lower elected officials (like the delegates) - and so the lower elected officials get their power only from voters, and not from the existing centralized government.

It is described that nobody is actually happy with their solution for governance (which I interpret as "it's full of patches, it's not based on an elegant mathematical proof on how ideal governance must certainly look"), but it's the best they found so far, and they're ready to overthrow it over the tiniest thing (like something 1% as bad as lying (my words)), or even "just in case" if they're in a situation where they can't know for sure that there's no conspiracy going on.

See some more here.

Meta: It seems to me like Yudkowsky is mainly concerned in aligning the country's government's  incentives, NOT in optimizing for ideal decisions, or fast ones, or anything like that. Mainly it seems like he's concerned with the government not being able to abuse its power, or to become a monopoly that nobody can replace. I interpret his intent as "make it very easy for anyone with a better idea of how to run a government to replace the current government (if they are right), and everything will flow from there", similarly to how capitalism doesn't tell you how to run a company, but it does try making sure that if someone has a better idea than you - they can replace you.