All of MikkW's Comments + Replies

In this case "improve gradually over time" could take place over the course of a few days or even a few hours. So it's not actually antithetical to FOOM

1
titotal
1y
So, for natural selection to function, there has to be a selection pressure towards the outcome in question, and sufficient time for the selection to occur. Can you explain how a flat-earther AI designed to solve mathematical equations would lose that belief in a matter of hours? Where is the selection pressure? 

Natural selection is the ultimate selector. Any meta-system that faces this problem will be continually outcompeted and replaced by meta-systems that either have an architecture that can better mitigate this failure mode, or that have beliefs that are minimally detrimental to its unfolding when this failure mode occurs (in my evaluation, the first is much more likely than the second)

1
titotal
1y
Indeed, better programs will outcompete worse ones, so I expect AI to improve gradually over time. Which is the opposite of foom.  I don't quite understand what you mean by a "meta-system that faces this problem". Do you mean the problem of having imperfect beliefs, or not being perfectly rational? That's literally every system!

Given that much of existential risk is made up of human-caused events stemming from issues that can be mitigated by policy and better governance, this intervention is even more important.

This logic is also the core intuition behind why I've been so passionate about voting reform (and social choice reform in general). Some thoughts:

For better voting methods to translate into better risk policy, I assume that it'd be necessary for the public to strongly desire to fix the problem. In the case of climate change, we see a strong movement to address that [1],... (read more)

I'm curious how much good can be done by setting up / funding Public Goods Markets (that is, quadratic funding) for the residents of developing countries

UBI does not create value, it merely redistributes it. It does not guarantee that there is enough for everybody, it merely helps ensure that when there is already enough (thanks to the workings of the free market), everybody has access to some part of that value

A note: This was written mostly 8 months ago, and I no longer entirely endorse everything I say in this essay (though I broadly agree with the main thesis that PGMs are worth investing time and energy to promote and create). I have cut the most egregrious parts I no longer endorse, and have lightly edited some other parts, however I haven’t spent a significant amount of time reworking this, and some parts are left being less precise than I would desire; however, I feel that most of the points in this essay are worth making and reading, and I have other thi... (read more)

In practice, states decide how to vote in both congressional and presidential elections (Maine for example, uses ranked choice for both). It is true that getting rid of the electoral college requires a federal amendment, but the electoral college isn't actually that bad; the big problems can be solved within its framework

Tl;dr: We should argue before the election, and build a consensus when it's time to actually get stuff done

I don't think reducing (tribal) polarization is at odds with agonism. There's plenty of room for healthy debate about what direction we should go in, but when in comes to actually deciding who has power, we want to track the median of what people think, not sway back and forth between two extremes (which is a compromise in and of itself)

I think it's worth being aware of lock-in effects; while I agree that it's unproductive to put down alternative approaches, it's also important to do what we can to avoid locking in a suboptimal choice- if Ranked Choice gets locked in over Approval Voting, and if Approval is actually better, it may be very hard to change it in the future, leading to much long-term disutility, perhaps even more than if the shift away from FPTP takes a little longer in the process of getting it right.

But I do agree that bickering and putting alternatives down probably isn't the best way to mitigate lock-in

From a total view, I think this does outline a potentially compelling case against the climate change argument, but I don't think it's compelling from an average point of view. Even from an in-between perspective (which I think roughly represents my feelings) which evaluates overall welfare as the product of average quality of life times the square root of population, it seems that marginal hits to the climate may outweigh marginal gains in quality of life.

Even from a totalist POV, it's important to consider lives with negative value. It mat... (read more)

In short, I think the effectiveness of altruism is limited in a non-demopistic society.

My perspective is that markets are a key factor in enabling things to happen. EAs are focused on promoting the social good, but current markets only promote individual benefits, not public benefits. We can either work upstream against the structure of incentives that currently exists, or we can implement systems (such as Quadratic Funding) that align incentives with the public good, thereby making our work easier and motivating more people to work on projects that the EA community values.

2
EdoArad
4y
Thanks. I'd be very excited to see a full post considering this set of ideas as a cause area proposal, possibly using the ITN framework, if you or anyone else is up to it. I think that the discourse in EA is too thin on these topics, and that perhaps some posts exploring the basics while considering the effects of marginal contribution might be the way to see whether we should consider them worthwhile. I think this makes this post somewhat premature, although I appreciate the suggested terminology and the succinct but informative writing.

But we can decide what goes inside the machine, whereas with people we can only control outside circumstances. It seems to me that such a mechanism would be highly likely to be an internal mechanism, so wouldn't be applicable to people

1
Gordon Seidoh Worley
4y
We're in an analogous situation with AI. AI is too complex for us to fully understand what it does (by design), and this is also true of mundane, human-programmed software (asking any software engineer who has worked on something more than 1k lines long if their program ever did anything unexpected and I can promise you the answer is "yes"). Thus although we in theory have control of what goes on inside AI, that's much less the case than it seems at first, so much so that we often have better models of how humans decide to do things than we do for AI.