Question from a newbie. I am constantly seeing negative references to the gutting of US foreign aid. It seems pretty clear that global development-focused EAs generally view the change in policy to be a bad thing. But I do not think I have once seen any discussion at all about how to reverse this state of affairs. Building on a running theme as of recently, it seems like political giving may have an outsize effectiveness, due to the relatively sparse funding in the space. So, naively, it would seem like you probably could get a great rate of return on efforts to reinstate USAID. I understand that political coalition building and organizing is not easy, etc. I'm not someone with those skills, just a rando. But I'm a little surprised that I don't think I've ever seen it taken up here when it seems like the downstream effects are making our goals harder to achieve. Basically, not only is it at face value cost effective, but also, we are collectively burning a lot of human capital working around this problem. Why not confront it head on?

I agree with you two. I don't have any delusions about avoiding all risk of causing harm or that harm avoidance is straightforwardly more important than providing benefits. I guess what I am saying is that risk of causing harm is distinct from risk this is ineffective, and it would be nice to see these broken out (as someone who works in data analytics and engineering, I realize the real world is not so simple). It seems to me that actively causing harm is a bit of a different thing than just being ineffective, and you would ideally reason about it specifically, rather than bundling it all together into one big effectiveness metric.

Especially since, while I don't think we should try to avoid all harm, people may have different moral weights about causing harm. For some people, they may be much more indifferent about causing harm relative to providing some benefit, whereas others may have a stronger bias towards "first, do no harm." Given that the tradeoff between these is something each individual must determine, it is better to separate it out in your model and allow people to discount the effectiveness according to their own priorities.

Numerous EA-adjacent orgs arriving at the same conclusion about some issue may also be the result of re-circulating the same people. After one year of observing EA online, my impression is that EA is not that large or diverse a group of people in the grand scheme of things. Many people seem to be pretty tightly interconnected together, even to the point of being family with each other!

I actually think EA does a pretty good job of avoiding group-think relative to its homogeneity (see posts like OPs), but given the social dynamics of a smaller and tightly interconnected movement, it is important to constantly reinforce truth-seeking behaviors, including by direct questioning of established orthodoxy.

Maybe my characterization of EA is wrong though. I'm not someone who would know.

I think what really bothers me about OPs post, is precisely the possibility that my donations are actually worsening animal welfare. That stings. I think its important for recommenders of interventions to carefully consider how they are going to communicate about such things. I would want them to break out not only their general uncertainty about the effectiveness of the intervention, but the specific uncertainty that it actually causes harm. For those of us who are concerned a lot about harm reduction, seeing that as its own line item would be helpful.

Sorry to turn this into an infinitely extended thread, but I wanted to post yet more data, namely CloudFlare's recent write up of their chance to work with Project Glasswing.

They do not provide numbers on bugs/vulnerabilities found, but they do provide some interesting commentary. Like others, they note that where Mythos stands out is in its ability to put together working exploits, and they elaborate on the value of this: proof-of-concept exploits are obviously worth reviewing; they are far less likely to be false positives.

They talk about the inadequacy of simply pointing a model at a codebase, and advocate for building pipelines and harnesses that enable the model to stay on task and counteract some of its reward-seeking behaviors. In an aside, they do make a passing comparison of Mythos to other frontier LLMs.

> When we ran other frontier models through the same harness, they found a fair number of the same underlying bugs, and in some cases they got further than we expected on the reasoning side too. Where they fell short was at the point of stitching the pieces together. A model would identify an interesting bug, write a thoughtful description of why it mattered, and then stop, leaving the actual chain unfinished and the question of exploitability open. What changed with Mythos Preview is that a model can now take those low-severity bugs (which would traditionally sit invisible in a backlog) and chain them into a single, more severe exploit. 

To me, this write up is only so valuable. On the one hand, it is evidence for what I've been saying—that an important part of unlocking model capabilities in cybersecurity is the development of adequate harnesses. Raw model intelligence is not enough for such a complex task. On the other hand, as is evident in the write up, the model intelligence is the foundation of capabilities, and without an adequate supply of it, the task is impossible.

So, I don't know that this really moves the needle on our broader discussion of whether "Mythos is overhyped," though I do think it supports some of my intermediate claims.

You are rightly grasping that we disagree, but I don't think you are understanding my view (and to be clear, reasonable people can disagree about this).

My wife and I are debating whether we will have more children or not. Having another child is desirable to us. So much so that she's willing to undergo the relatively risky process of child birth to have another one. However, failing to have another child is significantly less bad than losing one of our existing children, IMO. I'd even say that, failing to have 100 more children is significantly less bad than losing one of our existing children. The reason why is that the child who never existed is not sentient and so does not experience any deprivation. They do not suffer. And my suffering of that abstract loss is not nearly as bad as would be the suffering I would experience losing a living child who I know.

Now you may disagree with that, and mourn all the lost utility, and that is a reasonable perspective, but its not mine, and as you can see, this is a deeper philosophical difference and not some sort of misunderstanding about expected utility or something like that.

FYI, about this sentence: "X risks aren't especially bad because of all the utility lost ... they're bad because after they happen there's never any utility again." I don't really see a difference between these two statements.

One thing I didn't consider in my revised answer is that I didn't actually do the math. Taking an existential event as literally causing the end of earth-originating life, the question is whether the difference in probability multiplied by the immediate mass extinction itself would represent more death and suffering than the avertible death and suffering occurring over a 100-year period. I just don't know. It seems unlikely that the avertible death and suffering amounts to as much as the amount caused by the mass-extinction event itself, but after multiplying by the difference in probability and acknowledging the ambiguity of the timeline proposed in this question, things become less clear. However, let's say that the probability-adjusted, undetermined-timing mass-extinction event does cause more suffering and death and I change my answer to 50% agree. I don't think this is what most people would interpret 50% agree to express.

I should also be clear that I'm taking the question to mean literally ending earth-originating life in more-or-less one, fell swoop. Obviously, traditional x-risks actually have a spectrum of severity, so this is not so straightforward to apply to real-world resource allocation.

Initially I just calculated a naive expected value function and put 100% agree, but then I realized that I don't value realizing potential lives nearly as much as I value improving existing ones. While I do value realizing potential lives, the loss of them is not experienced by anyone other than present-day people like myself who think about them abstractly, which seems to me in sum to be less bad than the suffering otherwise avertible due to technological progress in the next 100 years. But I obviously haven't thought about this enough or I wouldn't have made my initial mistake.

I wanted to follow up on this thread and bring in some additional evidence on the whole AISLE thing. It isn't definitive or anything, but both AISLE and Mythos have been used to scan curl and the results are interesting.

- AISLE identified five CVEs and 24 bugs (plus two more CVEs in a dependency).
- Mythos identified 1 CVE and potentially 20 bugs.

Now, Mythos scanned after AISLE. We don't know what would have happened if Mythos had come first. But here's some quotes by the maintainer of curl about the Mythos results:

> curl is certainly getting better thanks to this report, but counted by the volume of issues found, all the previous AI tools we have used have resulted in larger bugfix amounts.

> I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in code analyzing.

> Any project that has not scanned their source code with AI powered tooling will likely find huge number of flaws, bugs and possible vulnerabilities with this new generation of tools. Mythos will, and so will many of the others.

Quotes taken from here: https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-vulnerability/

Daniel's blog post has some not great English, and also his tone is a bit less than objective. And of course, this is not a rigorous scientific comparison. But he is an expert in software and writing secure software, and he has demonstrated a willingness to change his mind on AI for cybersecurity,^[1] so I think he's worth listening to.

AISLE wrote a little news release about their findings vis-a-vis Mythos in the curl project: https://aisle.com/blog/curl-adopts-aisle-after-its-ai-agents-discovered-5-cves 

AISLE also wrote up another blog post building on the earlier one. In this case, they showcase a simple pipeline that is able to recreate a Mythos result without any steering towards the relevant snippet of code: https://aisle.com/blog/system-over-model-zero-day-discovery-at-the-jagged-frontier 

Interestingly, the same harness is not as successful at recreating AISLE's own results, but this is all a bit selective.

To my earlier claim that harnesses and pipelines enabling Mythos like results can and will be built in the near future, AISLE open-sourced nano-analyzer, the harness used in this blog to discover maybe as many as 40 bugs in FreeBSD, so it would appear that my prediction was fulfilled at the time of writing...

AISLE quotes $100 in spend to find the 40 bugs in FreeBSD and to recreate a Mythos result. That is presumably going to be 1/100th or less of compute spend by Anthropic. To me, this causes me to think that the compute spend by Anthropic is playing an important role in the whole assessment of the Mythos results. My belief now is that Anthropic's high compute spend is probably due to an inefficient pipeline, which is somewhat of a reversal of my earlier belief that the pipeline was the key determinant in model capabilities. However, this doesn't mean that I think model capabilities are hugely improved—it just means that I think that with a better pipeline, they would have spent less to get these results, given that AISLE was able to find some of these results for $100 with dumber models.
 

1. ^{^}

   Here's Daniel's highly negative blog post on AI cybersecurity work from last Summer: https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/ Here's where he changed his mind just a few months later after ZeroPath started to systematically find bugs using AI: https://daniel.haxx.se/blog/2025/10/10/a-new-breed-of-analyzers/ 

Has anyone done any thinking about posting the actual conversations for LLM ideation and other things? I'd be kind of interested. Most authors don't include the revision history of their papers when they publish them, but I feel like this could be helpful. Especially thinking about the case with the non-native English speaker using an LLM to translate their thoughts. I think that actually for us English primary-language speakers, funnily enough, it might be helpful to be able to see the full history.

Thank you for writing this survey of the evidence. I initially assumed from the title that you were going to present evidence that the attitudes of the general public are changing towards AI, rather than arguments intended to effect a change in their attitudes.

I feel compelled to note that Anthropic and OpenAI report ARR differently, making direct comparisons difficult. So, that chart could be misleading. For the purposes of this discussion, it is probably fine, as it captures the acceleration of growth of these companies, and we aren't trying to directly compare them to each other.

I do think that current-generation AI capabilities are already at the point where they could drive significant growth in the economy with an adequate inference infrastructure and time to develop workflows. Basically, what I'm trying to say is that the revenue growth of these companies may not be direct evidence that AGI is imminent in the technical sense. It seems possible to me that AGI could be stalled by technical challenges even as current-generation and similar AIs drive significant economic growth.