All of Jackson Wagner's Comments + Replies

Yeah, I wondered what threshold to set things at -- $10m is a pretty easy bar for some of these areas, since of course some of my listed cause areas are more niche / fringe than others. I figure that for the highest-probability markets, where $10m is considered all but certain, maybe I can follow up with a market asking about a $50m or $100m threshold.

I agree that $10m isn't "mainstream" in the sense of joining the pantheon alongside biosecurity, AI safety, farmed animal welfare, etc. But it would still be a big deal to me if, say, OpenPhil doubled their... (read more)

There are definitely a lot of examples of places where some rich people wanted to try to create a kinda dumb, socially-useless tax haven, and then they accomplished that goal, and then the resulting entity had either negative impact or close-to-zero impact on the surrounding area. (I don't know much about Monaco or the Cayman Islands, but these seem like potentially good examples?)  But there have also been times when political leaders have set out to create sustained, long-term, positive-sum economic growth, and this has also occasionally been achiev... (read more)

4
David T
1mo
I think that's an interesting and very open-minded reply. But I think the problem with the proposed model in practice isn't just that competition limits the ability to coordinate to prevent negative externalities, it's that the specific type of competition that "charter cities" are designed to stimulate (making undeveloped land in a poor country with severe governance problems into an attractive opportunity for business investment) is naturally geared towards lower taxes than elsewhere because it's one of the few things they can credibly commit to. Pretty much everyone agrees that "better institutions" matter to economic development, but good institutions are composed of people and culture and stability, and a startup entity with no existing infrastructure, limited sovereignty and a problematic parent country doesn't have any of that. What it can offer is "really low taxes". This also tends to be the carrot for the governments setting up the zones (the Honduran government doesn't really have an interest in demonstrating that it's doing a terrible job of regulating, but Prospera's plans to pay them a very small percentage of some expat millionaires' income sounded like a good deal at the time. Same logic as the tax havens). So the "race to the bottom" dynamics would exist even if the intellectual and financial core of the movement was considerably less libertarian. I agree there's plenty of potential value for humanity in studying what went well in Singapore and Dubai and the Guangdong SEZs that might actually be replicable, but one thing they definitely don't have in common is being set up by foreigners as greenfield projects to explore the regulations and city planning that most suit them. In that respect, Itana as a project set up by Nigerians with the stated goal of boosting Nigeria might be more interesting, but a quick glance suggests that generous tax exemptions are the real draw there too. And Prospera's low and regressive tax rates and corporate appointees'

Hyperbolic discounting, despite its reputation for being super-short-term and irrational, is actually better in this context, and doesn't run into the same absurd "value an extra meal in 10,000 years more than a thriving civilization in 20,000 years" problems of exponential discounting.

Here is a nice blog post arguing that hyperbolic discounting is actually more rational than exponential: hyperbolic discounting is what you get when you have uncertainty over what the correct discount rate should be.

Nice!  I like this a lot more than the chaotic multi-choice markets trying to figure out exactly why he was fired.

Very interested to find out some of the details here:

  • Why now?  Was there some specific act of wrongdoing that the board discovered (if so, what was it?), or was now an opportune time to make a move that the board members had secretly been considering for a while, or etc?
  • Was this a pro-AI-safety move that EAs should ultimately be happy about (ie, initiated by the most EA-sympathetic board members, with the intent of bringing in more x-risk-conscious leadership)?  Or is this a disaster that will end up installing someone much more focused on making
... (read more)

Side note: Greg held two roles: chair of the board, and president. It sounds like he was fired from the former and resigned from the latter role.

Definitely agree that empathy and other social feelings provide indirect evidence for self-awareness (ie, "modeling stuff about yourself" in your brain) in a way that optimism/pessimism or pain-avoidance doesn't.  (Although wouldn't a sophisticated-enough RL circuit, interacting with other RL circuits in some kind of virtual evolutionary landscape, also develop social emotions like loyalty, empathy, etc?  Even tiny mammals like mice/rats display sophisticated social behaviors...)

I tend to assume that some kind of panpsychism is true, so you don't... (read more)

1
MikhailSamin
4mo
I appreciate this comment. Qualia (IMO) certainly is "information processing": there are inputs and outputs. And it is a part of a larger information-processing thing, the brain. What I'm saying is that there's information processing happening outside of the qualia circuits, and some of the results of the information processing outside of the qualia circuits are inputs to our qualia.  Well, how do you know that visual information processing produces qualia? You can match when algorithms implemented by other humans' brains to algorithms implemented by your brain, because all of you talk about subjective experience; how do you, inside your neural circuitry, make an inference that a similar thing happens in neurons that just process visual information? You know you have subjective experience, self-evidently. You can match the computation run by the neural circuitry of your brain to the computation run by the neural circuitry of other humans: because since they talk about subjective experience, you can expect this to be caused by similar computation. This is valid. Thinking that visual information processing is part of what makes qualia (i.e., there's no way to replace a bunch of your neurons with something that outputs the same stuff without first seeing and processing something, such that you'll experience seeing as before) is something you can make theories about but is not a valid inference, you don't have a way of matching the computation of qualia to the whole of your brain. And, how can you match it to matrix multiplications that don't talk about qualia, did not have evolutionary reasons for experience, etc.? Do you think an untrained or a small convolutional neural network experiences images to some extent, or only large and trained? Where does that expectation come from? I'm not saying that qualia is solved. We don't yet know how to build it, and we can't yet scan brains and say which circuits implement it. But some people seem more confused than warranted

Why would showing that fish "feel empathy" prove that they have inner subjective experience?  It seems perfectly possible to build a totally mechanical, non-conscious system that nevertheless displays signs of empathy.  Couldn't fish just have some kind of built-in, not-necessarily-conscious instinct to protect other fish (for instance, by swimming together in a large school) in order to obtain some evolutionary benefit?

Conversely, isn't it possible for fish to have inner subjective experience but not feel empathy?  Fish are very simple crea... (read more)

3
MikhailSamin
4mo
Both (modeling stuff about others by reusing circuits for modeling stuff about yourself without having experience; and having experience without modelling others similarly to yourself) are possible, and the reason why I think the suggested experiment would provide indirect evidence is related to the evolutionary role I consider qualia to possibly play. It wouldn't be extremely strong evidence and certainly wouldn't be proof, but it'd be enough evidence for me to stop eating fish that has these things. The studies about optimistic/pessimistic behaviour tell us nothing about whether these things experience optimism/pessimism, as they are an adaptation an RL algorithm would implement without the need to implement circuits that would also experience these things, unless you can provide a story for why circuitry for experience is beneficial or a natural side effect of something beneficial. One of the points of the post is that any evidence we can have except for what we have about humans would be inderect, and people call things evidence for confused reasons. Pain-related behaviour is something you'd see in neural networks trained with RL, because it's good to avoid pain and you need a good explanation for how exactly it can be evidence for qualia.

April fools' day request:

I was reading the openai blog post "learning to summarize with human feedback" from the AI Safety Fundamentals course (https://openai.com/research/learning-to-summarize-with-human-feedback), especially the intriguing bit at the end about how if they try to fully optimize the model for maximum reward, they actually overfit and get lower-quality responses.

My ill-advised request is that I would just LOVE to see the EA Forum's "summaryBot" go similarly haywire for a day and start summarizing every post in the same repetitive / aggressi... (read more)

Here is a post of mine where I try to explore what a consaguinity-based intervention might look like, and what some of the benefits (cultural as well as health!) might be.

This is great; I've used it a few times over the past month and it's been interesting/helpful!

Here is a suggestion for a very similar tool: I would love to use some kind of "arbitrage calculator".  If I think that two markets with different prices have substantially the same criteria (for example, these three markets, which were priced at 24%, 34%, and 53% before I stepped in), obviously I can try to arbitrage them!  But there are many complications that I haven't been able to think through very clearly:

  • One market might be much smaller than the o
... (read more)

The framing of this does indeed sound like an accusation, and I kind of agree with Matthew Barnett that if you actually asked for "comment on the general trend", Caplan would just respond that he thinks he's right on all those things and that libertarianism is simply a good ideological lens.

But I totally agree that it would be great to ask for "examples of views he holds that are most inconvenient for his politics" -- this seems like a generally interesting/underrated interview question!

5
Devin Kalish
5mo
I think any question that attempts to get at the heart of the strongest objection to a public figure's worldview is going to sound like an accusation, because in a way it is, mostly I hope it's taken as an ultimately good natured, curious, and productive accusation. On the point of libertarianism being a "good lens", I mean libertarianism as a policy suggestion. I am voicing suspicion that there isn't a plausible lens behind this policy view that generalizes so well in both philosophy and the real world that it doesn't leave Caplan's slate of opinions looking suspicious, but for what it's worth my second question was basically asking him to propose one.

I would be interested in hearing more from Caplan about "stable totalitarianism", but not if it's just going to be a retread of the abstract concept that stable totalitarianism seems bad, what if Stalin had lived forever, etc.  Some questions I'd be interested in:

  • Is Caplan worried by any of the specific technologies that might make stable totalitarianism more likely?  Besides life-extension medical technology, we also have things like:
    • LLM-powered censorship
    • other AI-powered surveillance tech like facial recognition and gait tracking, or potential
... (read more)

Caplan has a lot of kids, wrote a book about why it's a good idea to have kids, and puts a lot of special effort into parenting (ie, teaching his children advanced economics).  To some extent this has been talked about in previous podcasts, but it would be interesting to hear some more from him about the ups and downs of parenting, what his advice would be to prospective parents, etc.

I would be interested to hear Bryan Caplan's take on Georgism (Tyler Cowen for instance thinks it's a bad idea) -- in general Caplan is opposed to pigouvian taxation, despite its appealing efficiency on paper, because Caplan thinks that it's all too easy for government to start calling anything it doesn't like a "negative externality", thus eroding peoples' freedoms.  I can see where he's coming from.  But maybe land value taxes could be a good idea even if we don't jump all the way to "tax everything we can think of that strikes us as a negative... (read more)

5
Matthew_Barnett
5mo
Bryan Caplan co-authored a paper critiquing Georgism in 2012. From the blog post explaining the critique,

Kind of a repetitive stream-of-consciousness response, but I found this both interesting as a philosophical idea and also annoying/cynical/bad-faith:

This is interesting but also, IMO, kind of a strawman -- what's being attacked is some very specific form of utilitarianism, wheras I think many/most "longtermists" are just interested in making sure that we get some kind of happy long-term future for humanity and are fuzzy about the details.  Torres says that "Longtermists would surely argue...", but I would like to see some real longtermists quoted as a... (read more)

1
alexherwix
6mo
Yeah, I totally agree with you. This writing style is kind of annoying/cynical/bad-faith. Still it really does raise an interesting point as you acknowledge. I just wish more of the EA community would be able to see both of these points, take the interesting point on board, and take the high road on the annoying/cynical/bad-faith aspect. For me the key insight in this last section is that utilitarianism as generally understood does not have an appreciation of time at all, it just cares about sums of value. Thus, the title of the book is indeed pretty ironic because the position presented in it does not really care about the "future" per se but about how to collect more containers of value. It just happens that we currently believe that many multitudes of those containers of value could theoretically still be brought into being in the future (i.e., the potential of humanity). Thus, it sketches an understanding of what is good in the world that is a little bit like pac man, where the goal is to just go for as many dots as possible without being eaten. What does pac man owe the future? I have never really thought about it this way before but I think that's not a totally unfair characterization. It reminds me of a similar critique of utilitarianism by Martha Nussbaum in her recent book "Justice for Animals". So at least to me, this argument does seem to have some force and made me think about the relationship between utilitarianism and longtermism in a new light. But I totally agree with you that we should all stop with this annoying/cynical/bad-faith style of arguing. We are all in this together. I think we can also all learn from each other. While I do fear that philosophy and worldviews can make actual differences and do significantly influence politics, this just makes it just all the more important that we start talking WITH rather than fight against each other. When I posted the link, I was hoping for the former and it looks like we made some of that happen afte

I think the downvotes are coming from the fact that Émile P. Torres has been making similar-ish critiques on the concept of longtermism for a while now.  (Plus, in some cases, closer to bad-faith attacks against the EA movement, like I think at one point saying that various EA leaders were trying to promote white supremacism or something?)  Thus, people might feel both that this kind of critique is "old news" since it's been made before, and they furthermore might feel opposed to highlighting more op-eds by Torres.

Some previous Torres content whi... (read more)

7
alexherwix
6mo
Yeah, I mean I understand that people don't really like Torres and this style of writing (it is pretty aggressive) but still there are some interesting points in the post, which I think deserve reflection and discussion. Just because "the other side" does not listen to the responses does not mean there is nothing to learn for oneself (or am I too naive in this belief?). So, I still think downvoting into oblivion is not the right move here.  Just to give an example, I think the end of the post is interesting to contemplate and cannot just be "dismissed" without at least some level of critical engagement.

Various "auto-GPT" schemes seem like a good demonstration of power-seeking behavior (and perhaps very limited forms of self-preservation or self-improvement), insofar as auto-GPT setups will often invent basic schemes like "I should try to find a way to earn some money in order to accomplish my goal of X", or "I should start a twitter account to gain some followers", or other similarly "agenty" actions/plans.

This might be a bit of a stretch, but to the extent that LLMs exhibit "sycophancy" (ie, telling people what they want to hear in response to stuff lik... (read more)

I'm definitely not deeply familiar with any kind of "official EA thinking" on this topic (ie, I don't know any EAs that specialize in nuclear security research / grantmaking / etc).  But here are some things I just thought up, which might possibly be involved:

  • Neglectedness in the classic sense.  Although not as crowded as climate change, there are other large organizations / institutions that address nuclear risk and have been working in this space since the early Cold War.  (Here I am thinking not just about charitable foundations, but also
... (read more)
4
Denkenberger
7mo
I agree that the nuclear risk field as a whole is less neglected than AGI safety (and probably than engineered pandemic), but I think that resilience to nuclear winter is more neglected. That's why I think overall cost-effectiveness of resilience is competitive with AGI safety.
6
Davidmanheim
7mo
This gets a lot of things right, but (knowing some of the EAs who did look into this or work on it now,) I would add a few: 1. Lindy effect and stability - we're 70 years in, and haven't had any post-first-use nuclear usage, so we expect it's somewhat stable - not very stable, but the risk from newer technologies under this type of estimation is higher, because we have less of a record.  2. The current inside-view stability of the nuclear situation, where strong norms exist against use, and are being reinforced already by large actors, with deep pockets. 3. There seems to be a pretty robust expert consensus about the problem, and it concludes that there is little to be done other than on the margin. Also, note that this was investigated as a cause area early on by Open Philanthropy, and then was looked at by Longview more recently. Both decided to have it as a small focus, rather than a key area. Edit (to correct a mistake): It was looked at by Longview more recently, and they have highlighted the topic significantly more, especially in the wake of other funders withdrawing support.

reposting a reply by Omnizoid from Lesswrong:

"Philosophy is pretty much the only subject that I'm very informed about.  So as a consequence, I can confidently say Eliezer is eggregiously wrong about most of the controversial views I can fact check him on.  That's . . . worrying."

And my reply to that:

Some other potentially controversial views that a philosopher might be able to fact-check Eliezer on, based on skimming through an index of the sequences:

  • Assorted confident statements about the obvious supremacy of Bayesian probability theory and how
... (read more)

I suggest maybe re-titling this post to:
"I strongly disagree with Eliezer Yudkowsky about the philosophy of consciousness and decision theory, and so do lots of other academic philosophers"

or maybe:
"Eliezer Yudkowsky is Frequently, Confidently, Egregiously Wrong, About Metaphysics"

or consider:
"Eliezer's ideas about Zombies, Decision Theory, and Animal Consciousness, seem crazy"

Otherwise it seems pretty misleading / clickbaity (and indeed overconfident) to extrapolate from these beliefs, to other notable beliefs of Eliezer's -- such as cryonics, quantum mec... (read more)

6
Jackson Wagner
7mo
reposting a reply by Omnizoid from Lesswrong: "Philosophy is pretty much the only subject that I'm very informed about.  So as a consequence, I can confidently say Eliezer is eggregiously wrong about most of the controversial views I can fact check him on.  That's . . . worrying." And my reply to that: Some other potentially controversial views that a philosopher might be able to fact-check Eliezer on, based on skimming through an index of the sequences: * Assorted confident statements about the obvious supremacy of Bayesian probability theory and how Frequentists are obviously wrong/crazy/confused/etc.  (IMO he's right about this stuff.  But idk if this counts as controversial enough within academia?) * Probably a lot of assorted philosophy-of-science stuff about the nature of evidence, the idea that high-caliber rationality ought to operate "faster than science", etc.  (IMO he's right about the big picture here, although this topic covers a lot of ground so if you looked closely you could probably find some quibbles.) * The claim / implication that talk of "emergence" or the study of "complexity science" is basically bunk.  (Not sure but seems like he's probably right?  Good chance the ultimate resolution would probably be "emergence/complexity is a much less helpful concept than its fans think, but more helpful than zero".) * A lot of assorted references to cognitive and evolutionary psychology, including probably a number of studies that haven't replicated -- I think Eliezer has expressed regret at some of this and said he would write the sequences differently today.  But there are probably a bunch of somewhat-controversial psychology factoids that Eliezer would still confidently stand by.  (IMO you could probably nail him on some stuff here.) * Maybe some assorted claims about the nature of evolution?  What it's optimizing for, what it produces ("adaptation-executors, not fitness-maximizers"), where the logic can & can't be extended (can corporations b

I agree, and I think your point applies equally well to the original Eliezer Zombie discussion, as to this very post.  In both cases, trying to extrapolate from "I totally disagree with this person on [some metaphysical philosophical questions]" to "these people are idiots who are wrong all the time, even on more practical questions", seems pretty tenuous.

5
Brad West
7mo
To be fair to the OP, I don't think that he was saying you should not consider the views of Yudkowsky- in fact he admits that Yudkowsky has some great thoughts and that he is an innovator. OP observes that he himself for a long time reflexively deferred to Yudkowksy. I think his objective with his post was to point out some questions on which he thought Yudkowsky was  pretty clearly wrong (although it is not clear that he accomplished this). His goal was not to urge people not to read or consider Yudkowsky, but rather to urge people not to reflexively defer to him.

But all three parts of this "takedown" are about questions of philosophy / metaphysics?  How do you suggest that I "follow the actual evidence" and avoid "first principles reasoning" when we are trying to learn about the nature of consciousness or the optimal way to make decisions??

9
oivavoi
7mo
I realize that my comment was somewhat poorly worded. I do not mean that you can follow the evidence in an absolute and empirical sense when forming a belief about the nature of consciousness. What you can do, however, and which Eliezer doesn't do, is to pay attention to what the philosophers who spend their lives working on this question are saying, and take their arguments seriously. The first principle approach is kind of "I have an idea about consciousness which I think is right so I will not spend too much time looking at what actual philosophers are saying".  (I did a master's degree in philosophy before turning to a career in social science, so at least I know enough about contemporary analytic philosophy to know what I don't know) My comment "just follow the actual evidence" was not regarding consciousness or metaphysics, but regarding broader epistemic tendencies in the EA community. This tendency is very much Eliezer-ish in style: An idea that one knows best, because one is smart. If one has a set of "priors" one thinks are reasonably well-founded one doesn't need to look too much at empirical evidence, arguments among researchers or best practices in relevant communities outside of EA.  A case in point that comes to mind was some time ago when EAs debated whether it is a good idea that close colleagues in EA orgs have sex with each other. Some people pointed out that this is broadly frowned upon in most high-risk or high-responsibility work settings. Eliezer and other EAs thought they knew better, because, hey - first principles and we know ethics and we are smart! So the question then becomes: who should we trust on this, Eliezer and some young EAs in their early twenties or thirties, or high-powered financial firms and intelligence agencies who have fine-tuned their organizational practices over decades? Hm, tough one. There are obviously huge differences between metaphysics, empirical evidence on various social issues, and sexual ethics in organiza

For what it's worth, I didn't mean to come off as being hostile to the idea that EA should pay more attention to the exploitation of poor countries. I really just don't know much about this area -- for example I've never before heard the idea that France has an extractive relationship with former colonies that use the French currency. Maybe to you, "neoliberalism" is a very clear-cut concept. But personally, I would have a hard time telling apart which loans are good and which are ill-intentioned debt traps. (Is the IMF mostly good and Belt-And-Road mo... (read more)

"what about Haiti? How many people are in poverty there that can easily be solved, how many lives saved?"

I agree that there is probably a lot more that rich western countries should be doing to make life better for ordinary Haitians. Unfortunately, Step One in almost every conceivable plan is "help establish some kind of functional government in Haiti, to end the ongoing gang-fueled anarchy and violence." And that would involve sending western soldiers to take temporary control of the island, which (justly or not) would be derided by the press and public... (read more)

-14
AlanGreenspan
7mo

Some discussion of the "career security" angle, in the form of me asking how 1-year-grant recipients are conceiving of their longer-term career arcs: https://forum.effectivealtruism.org/posts/KdhEAu6pFnfPwA5hf/grantees-how-do-you-structure-your-finances-and-career

Radical life extension is IMO a big part of the rationalist worldview, if not the EA movement.  (Although recent progress in AI has taken attention away from anti-aging, on the grounds that if we get AI alignment wrong, we're all dead, and if we get alignment right, the superintelligent AI will easily solve aging for us.)

One of the problems with radical life extension as an EA cause area is that it seems like other people ought to be rationally self-interested in funding anti-aging research, so it's not clear why EA should foot the bill:

Health interve... (read more)

5
Matt_Sharp
8mo
I'd also note that hundreds of billions of dollars are spent on biomedical research generally each year. While most of this isn't targeted at anti-aging specifically, there will be a fair amount of spillover that benefits anti-aging research, in terms of increased understanding of genes, proteins, cell biology etc.

Roll to disbelieve?  50-100 words is only, like, a couple of tweets, so it is really not much time to communicate many new ideas.  Consider some of the most influential tweets you've ever read (either influential on you personally, or on societal discourse / world events / etc).  I think that the most impactful/influential tweets, are gonna be pretty close to the limit of what's possible when you are just blasting out a standard message to everyone -- even with a superintelligence, I doubt there is much room for improvement.

Now, if you were ... (read more)

6
titotal
8mo
It could probably pull this this off for a subsection of highly suggestible people, but I'm skeptical that even a superintelligence could convince most people to change deeply held values with a mere tweet thread.  Yes, there are a lot of potential word combinations out there, but the computer only has a finite amount of time to search through them, and is relying on an unavoidably imperfect model of the target person and the outer world (because of incomplete information and finite computing power).  I think it all comes down to how difficult the attempted persuasion is: I'm sure an AI could convince me to buy a 50$ product, but I don't see any universe where it can convince me to commit an act of violence against a loved one. 
1
trevor1
8mo
I think this goes a long way to completely shut down my argument. So, basically, case closed.  In-person communication is generally better anyway due to continuous feedback, and figuring out how to make those conversations 1) go as well as possible and 2) allow the presentations to remain effective after many people down the telephone chain. It's definitely important to get bullet pointed lists going, but you also need to present it so that it's not interpreted as threatening or a show of dominance, it's harder than you'd intuitively think to make someone accept a message as controversial as Wild Animal Welfare.

Seems like this might be a cause that "Canning What We Give" is well-positioned to address! https://forum.effectivealtruism.org/posts/ozyQ7PwhRgP7D9b2w/new-org-canning-what-we-give

(seriously though, this is a really cool post and I enjoyed reading it)

2
NickLaing
8mo
Nice one :) think you have solved the problem conclusively. If we just can all of the beans and put them in supermarkets then we will hardly need to cook them at all. A far more elegant solution ;)

I think the main response would be to try and scale up alternative food supplies such as those being explored by https://allfed.info/, as well as trying to scale up cultivation of whatever plants are immune to the virus (it would presumably be difficult to make a single disease which could wipe out all plants, so if perhaps wheat crops start failing worldwide, maybe we start re-seeding those farms with potatoes/corn/rice or whatever is climactically appropriate).

In general, I think out-of-the-box GCBR ideas are not discussed much in public due to the infoh... (read more)

The following is way more speculative and wacky than the proven benefits of family planning that you point out above, but I think it's interesting that there is some evidence that changing family norms around marriage / children / etc might have large downstream effects on culture, in a way that potentially suggests "discouraging cousin marriage" as an intervention to increase openness / individualism / societal-level trust: https://forum.effectivealtruism.org/posts/h8iqvzGQJ9HiRTRja/new-cause-radio-ads-against-cousin-marriage-in-lmic

1
alamo 2914
8mo
Thanks, that proposal is indeed very interesting!

Well, I wrote this script and not the other one, and I think the idea of publishing draft scripts on the Forum just never occurred to RationalAnimations before? (Even though it seemed like a natural step to me, and indeed it has been helpful.) So naturally I will advocate doing it with future scripts. We already have a couple other ways of getting feedback including internal feedback, working with relevant organizations (eg for this Charter Cities script we got some comments from people associated with the charter cities institute, etc), but the more the merrier when you are trying to get something finalized before animating it for a large audience.

1
alamo 2914
8mo
I agree, it seems like a very good idea to post it here. I assume it's seen by dozen times more people than in your team. Also, the EA forum is definitely less biased about any thing than organizations that work for that thing. Just the way humans work. 

Yes, I think RationalAnimations is actually planning to do an episode on GiveDirectly sometime soon, which is why I nod towards the idea of interventions like cash transfers and bednets at the very beginning of the script -- the GiveDirectly video will probably come out first, so then in the beginning of the Charter Cities video we'll be able to make some visual allusions and put an on-screen title-card link to the GiveDirectly video.

1
alamo 2914
8mo
That's neat. May I ask why have you published this script, but not the script for the GiveDirectly episode?
  • Since "Why The West Rules" is pretty big on understanding history through quantifying trends and then trying to understand when/why/how important trends sometimes hit a ceiling or reverse, it might be interesting to ask some abstract questions about how much we can really infer from trend extrapolation, what dangers to watch out for when doing such analysis, etc.  Or as Slate Star Codex entertainingly put it, "Does Reality Drive Straight Lines On Graphs, Or Do Straight Lines On Graphs Drive Reality?"  How often are historical events overdetermine
... (read more)

@MvK and @titotal , here is the new section about political tractability:

"A bigger problem is political feasibility.  The whole concept of giving a city the ability to write its own rules is to make reform easier, but in order to get that ball rolling, you first need to find a nation willing to give away lots of their own regulation-writing authority in order to enable your charter city project.  This isn’t completely unheard of -- in many ways, charter cities are just a bigger and bolder version of “Special Economic Zones”, where a port might be

... (read more)

Yes, in response to MvK's comment, I am reworking the script to add a section (in-between "objection: why whole new cities?" and "wider benefits") about political feasibility, where I will talk about how Paul Romer abandoned the idea after delays and failed projects in Honduras and Madagascar.  I'll add another comment here when I update this Forum post with the new draft.

Do you have any suggestions as to which parts of the draft could be cut or made shorter?  The current post is already getting a little long compared to our ideal video length of 10-15 mins.

As I mention in my reply to MvK above, I agree that I don't think charter city efforts should literally be funded by EA megadonors or ranked as top charities by GiveWell; I just think they are a potentially helpful idea that the EA movement should support when convenient / be friendly towards.  Instead, since charter cities double as a "lets all get rich" thing, they can be mostly funded by investors (just like how investors already fund lots of international development projects -- factories, etc).

Also agree that the benefit vs tractability of charte... (read more)

Thanks for this feedback!  For more context, the tone of the video is intended to be a kind of middle ground between persuasion and EA-Forum-style information, which I'd describe as "introducing people to a cool and intriguing new idea, as food for thought".  (I also see this video as "making sure RationalAnimations has enough interesting variety to keep pulling in new viewers, even though many of our upcoming videos are going to be about more-technical AI alignment topics".)  So, the video is definitely trying to be informative and somewhat... (read more)

 So, the video is definitely trying to be informative and somewhat evenhanded rather than a purely persuasive advertisement for charter cities.

If this is your goal, I'm afraid to say you have not succeeded. I apologise if the following sounds harsh, but you have a platform and a commensurate responsibility towards accuracy. 

Ask yourself the question: "Is the viewer coming away from this video with a broadly accurate picture of the facts and the most relevant expert opinions on this topic"? I think the answer is a clear no. If an audience member i... (read more)

Just brainstorming here, I have zero experience with actual psychology research:

- It might be interesting to try and identify some psychological traits that lead people to becoming EAs / becoming alignment researchers, in order to aid future recruitment/education/community-building efforts.

- This is a medium-term concern rather than about alignment itself, but I would be interested to get a clearer picture on how "botpocalypse" concerns will play out.  (See this ACX article for more detail, as well as the relevant recurring section of theZvi's AI news... (read more)

4
Geoffrey Miller
8mo
Jackson - great ideas; thanks so much for your thoughtful and creative suggestions here!

Yeah, as a previous top-three winner of the EA Forum Creative Writing Contest (see my story here) and of Future of Life Institute's AI Worldbuilding contest (here), I agree that it seems like the default outcome is that even the winning stories don't get a huge amount of circulation.  The real impact would come from writing the one story that actually does go viral beyond the EA community.  But this seems pretty hard to do; perhaps better to pick something that has already gone viral (perhaps an existing story like one of the Yudkowsky essays, or... (read more)

Yeah, huge fan of this concept; I'm eager to see that people are trying it out!

This post felt vague and confusing to me.  What is meant by a "game board" -- are you referring to the world's geopolitical situation, or the governance structure of the United States, or the social dynamics of elites like politicians and researchers, or some kind of ethereum-esque crypto protocol, or internal company policies at Google and Microsoft, or US AI regulations, or what?

How do we get a "new board"?  No matter what kind of change you want, you will have to get there starting from the current situation that the world is in right now.&nbs... (read more)

3
Prometheus
9mo
Thank you for taking the time to read and critique this idea. I think this is very important, and I appreciate your thoughtful response. Regarding how to get current systems to implement/agree to it, I don't think that will be relevant longterm. The mechanisms current institutions use for control I don't think can keep up with AI proliferation. I imagine most existing institutions will still exist, but won't have the capacity to do much once AI really takes off. My guess is, if AI kills us, it will happen after a slow-motion coup. Not any kind of intentional coup by AIs, but from humans just coup'ing themselves because AIs will just be more useful. My idea wouldn't be removing or replacing any institutions, but they just wouldn't be extremely relevant to it. Some governments might try to actively ban use of it, but these would probably be fleeting, if the network actually was superior in collective intelligence to any individual AI. If it made work economically more useful for them, they would want to use it. It doesn't involve removing them, or doing much to directly interfere with things they are doing. Think of it this way, recommendation algorithms on social media have an enormous influence on society, institutions, etc. Some try to ban or control them, but most can still access them if they want to, and no entity really controls them. But no one incorporates the "will of twitter" into their constitution. The game board isn't any of the things you mention. All the things you mention I don't think have the capacity to do much to change the board. The current board is fundamentally adversarial, where interacting with it increases the power of other players. We've seen this with OpenAI, Anthropic, etc. The new board would be cooperative, at least at a higher level. How do we make the new board more useful than the current one? My best guess would be economic advantage of decentralized compute. We've seen how fast the OpenSource community has been able to make pro

I think at the very least, I'd expect non-neglected AI safety to look like the global campaigns against climate change, or the US military-industrial complex:

  • something that governments spend hundreds of billions of dollars on
  • a vast ecosystem of nonprofits, think tanks, etc, with potentially tens of thousands of people just thinking about strategy, crafting laws, etc
  • legions more people working on specific technologies that will help with various niche aspects of the problem, like more efficient solar panels or etc
  • lots of people who don't think of thems
... (read more)

I think it is widely acknowledged that virtue ethics is perhaps easier to live by / more motivating / produces better incentives / etc, on an individual level, than trying to be a hardcore utilitarian in all your daily-life actions.  And I agree with Stefan Schubert's linked posts.

But when people look at morality from the perspective of what works best on an individual level, they miss some of the most advantageous things about utilitarianism as it pertains to EA:

  • Utilitarianism is a more legible framework that makes it easier for many people to debate
... (read more)
2
Stefan_Schubert
10mo
Nice comment, you make several good points. Fwiw, I don't think our paper is conflict with anything you say here.
2
Severin
10mo
Agreed!

Community notes seem like a genuinely helpful improvement on the margin -- but coming back to this post a year later, I would say that on net I am disappointed.  (Disclaimer -- I don't use twitter much myself, so I can't evaluate people's claims of whether twitter's culture has noticeably changed in a more free-speech direction or etc.  From my point of view just occasionally reading others' tweets, I don't notice any change.)

During the lead-up to the purchase, people were speculating about all kinds of ways that Twitter could try to change its s... (read more)

For totally selfish, non-historical reasons, I feel like May 8 is a better date:

  • December 9 is too close to other rationalist/EA holidays, like Solstice, Giving Tuesday, and Petrov Day.

  • December 9 is right at the START of the typical cold/flu season, when infectious diseases are the worst. (Although idk if smallpox, plague, typhus, etc, were also seasonal in this way?) Maybe this makes it thematically resonant? But personally, like how Christians celebrate Easter at the end of winter, I feel like Smallpox eradication is a good seasonal match as a sp

... (read more)
1
Timothy Currie
1y
One other counterargument is that rinderpest eradication day is on 25 May. Just two weeks after May 8.

I agree with the idea that nuclear wars, whether small or large, would probably push human civilization in a bad, slower-growth, more zero-sum and hateful, more-warlike direction.  And thus, the idea of civilizational recovery is not as bright a silver lining as it seems (although it is still worth something).

I disagree that this means that we should "try to develop AGI as soon as possible", which connotes to me "tech companies racing to deploy more and more powerful systems without much attention paid to alignment concerns, and spurred on by a sense ... (read more)

I agree that hoping for ideal societies is a bit of a pipe dream.  But there is some reason for hope.  China and Russia, for instance, were both essentially forced to abandon centrally-planned economies and adopt some form of capitalism in order to stay competitive with a faster-growing western world.  Unfortunately, the advantages of democracy vs authoritarianism (although there are many) don't seem quite as overwhelming as the advantages of capitalism vs central planning.  (Also, if you are the authoritarian in charge, maybe you don't... (read more)

1
Arturo Macias
1y
I agree with all your political positions! Let’s run for office together! Now, more seriously, thank you very much for the links.

Yes, it is definitely a little confusing how EA and AI safety often organize themselves via online blog posts instead of papers / books / etc like other fields!  Here are two papers that seek to give a comprehensive overview of the problem:

... (read more)

Hi!  Some assorted thoughts on this post:

  • You say that "my opinion is that Nuclear War is among the most likely causes of death for a person my age in the Northern Hemisphere".  I think I agree with this in a literal sense, but your most of your post strikes me as more pessimistic than this statistic alone.  Based on actuarial tables, the risk of dying at 45 years old (from accidents, disease, etc) is about 0.5% for the average person.  So, in order to be the biggest single risk of death, the odds of dying in a nuclear war probably need
... (read more)
5
Arturo Macias
1y
Dear Mr. Wagner, Do you have any canonical referece for AI aligment research? I have read Eliezer Yudkowsky FAQ and I have been surprised of how little technical details are commented. His arguments are very much "we are building alien squids and they will eat us all". But they are not squids, and we have not trained them to prey on mammals, but to navigate across symbols. The IAs we are training are not as alien as giant a squid, but far more: they are not even trained for self-preservation.  MR suggests that there is not peer reviewed literature on AI risk: https://marginalrevolution.com/marginalrevolution/2023/04/from-the-comments-on-ai-safety.html "The only peer-reviewed paper making the case for AI risk that I know of is: https://onlinelibrary.wiley.com/doi/10.1002/aaai.12064. Though note that my paper (the second you linked) is currently under review at a top ML conference." But perhaps I can read something comprehensive (a pdf, if possible), and not depend on navigating posts, FAQs and similar stuff. Currently my understanding of AI risk is based in technical knowledge about Reinforcement Learning for games and multiagent systems. I have no knowledge nor intuition on other kind of systems, and I want to engage with the "state of the art" (in compact format) before I make a post focused on the AI alignement side.
Load more