All of Davidmanheim's Comments + Replies

Yes, I see a strong argument for the claim that the companies are in the best position to shoulder the harms that will inevitably come along, and pass that risk onto their customers through higher prices - but the other critical part is that this also changes incentives because liability insurers will demand the firms mitigate the risks. (And this is approaching the GCR argument, from a different side.)

I think that the use of insurance for moderate harms is often a commercial boondoggle for insurers, a la health insurance, which breaks incentives in many ways an leads to cost disease. And typical insurance regimes shift burden of proof about injury in damaging ways because insurers have deep pockets to deny claims in court and fight cases that establish precedents. I also don't think that it matters for tail risks - unless explicitly mandating unlimited coverage, firms will have caps in the millions of dollars, and will ignore tail risks that will bankru... (read more)

I would be interested in understanding whether you think that joint-and-several liability among model training, model developers, application developers, and users would address many of the criticisms you point out against civil liability. As I said last year, "joint-and-several liability for developers, application providers, and users for misuse, copyright violation, and illegal discrimination would be a useful initial band-aid; among other things, this provides motive for companies to help craft regulation to provide clear rules about what is needed to ... (read more)

If you find anyone who quotes that as an excuse where a modern Halachik authority would rule that they don't have too much money for that to apply to them, I'll agree they are just fine only giving 20%. (On the other hand, my personal conclusion is less generous.) But DINKs or single people  making $100k+ each who comprise most of the earning to give crowd certainly don't have the same excuse!

It was actually quoting the first bit; "The amount of charity one should give is that if you can afford to, give as much as is needed. Under ordinary circumstances, a fifth of one's property is most laudable. To give one-tenth is normal. To give less than one-tenth is stingy."

To ruin the joke, cf. Taanis 9a and even more, Yoreh Deah 249:
שיעור נתינתה אם ידו משגת יתן כפי צורך העניים ואם אין ידו משגת כל כך יתן עד חומש נכסיו מצוה מן המובחר ואחד מעשרה מדה בינונית פחות מכאן עין רעה

That's much too Meta.

Steinsaltz: "development" - This is the mitzvah of "Tikkun Olam"

Commentaries on the Mishnah of Rabbi Ord:

Rashi: "three permissible cause areas" - cause areas, not fathers.
Tosfos: "cause areas" - If the mishna calls these cause areas not fathers, per [Rashi's] notebooks, why does Rabbi bar bar Hana call them categories? Clearly, these must be categories. How, then, do we explain the words of the master [Rashi]? Perhaps the Mishna was careful not to use the word "category" because parent categories requires a listing of child categories, but the child categories are subject to an extensive dispute between charity evaluat... (read more)

Perhaps worth noting that very long term discounting is even more obviously wrong because of light-speed limits and the mass available to us that limits long term available wealth - at which point discounting should be based on polynomial growth (cubic) rather than exponential growth. And around 100,000-200,000 years, it gets far worse, once we've saturated the Milky Way.

The reason it seems reasonable to view the future 1,000,010 years as almost exactly as uncertain as 1,000,000 years is mostly myopia. To analogize, is the ground 1,000 miles west of me more or less uneven than the ground 10 miles west of me? Maybe, maybe not - but I have a better idea of what the near-surroundings are, so it seems more known. For the long term future, we don't have much confidence in our projections of either a million or a million an ten years, but it seems hard to understand why all the relevant uncertainties will simply go away, other than simply not being able to have any degree of resolution due to distance. (Unless we're extinct, in which case, yeah.)

To embrace this as a conclusion, you also need to fairly strongly buy total utilitarianism across the future light cone, as opposed to any understanding of the future, and the present, that assumes that humanity as a species doesn't change much in value just because there are more people. (Not that I think either view is obviously wrong - but it is so generally assumed in EA that it's often unnoticed, but it's very much not a widely shared view among philosophers or the public.)

I misunderstood, perhaps. Audit rates are primarily a function of funding - so marginal funding goes directly to those audits, because they are the most important. But if the US government wasn't being insane, it would fund audits until the marginal societal cost of the audit was roughly equal to the income by the state.
The reason I thought this disagreed with you point is because I thought you were disagreeing with the earlier claim that "This is going to lead to billionaires' actions being surveilled more and thus gone after for crimes more often than th... (read more)

Data seems to indicate otherwise.
https://www.gao.gov/products/gao-22-104960
https://www.usatoday.com/story/money/2023/11/09/irs-uses-funding-to-audit-wealthy/71486513007/

First, to your second point, I agree that they aren't comparable, so I don't want to respond to your discussion. I was not, in this specific post, arguing that anything about safety in the two domains is comparable. The claim, which you agree to in the final paragraph, is that there is an underlying fallacy which is present both places. 

However, returning to your first tangential point, the claim that the acceleration versus deceleration debate is theoretical and academic seems hard to support. Domains where everyone is dedicated to minimizing regulat... (read more)

"The analogies establish almost nothing of importance about the behavior and workings of real AIs"

You seem to be saying that there is some alternative that establishes something about "real AIs," but then you admit these real AIs don't exist yet, and you're discussing "expectations of the future" by proxy. I'd like to push back, and say that I think you're not really proposing an alternative, or that to the extent you are, you're not actually defending that alternative clearly.

I agree that arguing by analogy to discuss current LLM behavior is less us... (read more)

I do too!

I agree that the properties are somewhat simplified, but a key problem here is that the intuition and knowledge we have about how to make software better fails for deep learning. Current procedures for developing debugging software work less well for neural networks doing text prediction than psychology does. And at that point, from the point of view of actually interacting with the systems, it seems worse to group software and AI than to group AI and humans. Obviously, however, calling current AI humanlike is mostly wrong. But that just shows that we don't want to use these categories!

One the first point, I think most technical people would agree with the claim: "AI is a very different type of thing that qualifies as software given a broad definition, but that's not how to think about it."

And given that, I'm saying that we don't say " a videoconference meeting is different to other kinds of software in important ways," or "photography is different to other kinds of software in important ways" because we think of those as a different thing, where the fact that it's run on software is incidental. And my claim is that we should be doing that with AI.

No, the title wasn't a definitional claim, it was pointing out that we're using the word "software" as hidden inference, in ways that are counterproductive, and so I argued that that we should stop assuming it's similar to software.
 

Also, no, AI models aren't executing code line by line, they are using software to encode the input, then doing matrix math, and feeding the result into software that provides this as human-readable output. The software bits are perfectly understandable, it's the actual model that isn't software which I'm trying to discuss.
 

Appealing to definitions seems like a bad way to argue about whether the conceptual model is useful or not. The operation of a computer system and the "software" used for  digital photography, or videoconferencing, or essay writing, is not typically considered software.  Do you think those should be called software, given that they fit into the definitions given?

I'm claiming that AI is distinct in many ways from everything else we typically think of as software, not that it doesn't fit a poorly scoped definition. Amd the examples of "collection[s... (read more)

Yeah, I don't think there's a ton of benefit it trading hypotheticals and counterfactuals here, especially because I don't think much of anyone's intutions will be conveyed clearly, but I do think it's worth noting that it's not obvious to me that the convention didn't have a large counterfactual impact over the past 50 years.

First, thanks for writing this up. I mostly agree with the points being made, but I disgree with some of the conclusions.

Yes, the BWC is dysfunctional, and yes, it's unlikely to materially reduce bioweapon risk below the (currently low) levels they are at today. But, as you say, it's better for it to exist than not. I think the counterfactual here is critical. If we look at the BWC compared to other weapon bans, it's somewhat weaker than the IAEA, though arguably stronger than the NPT, and it's weaker than the CWC, though chemical weapons are used more. On... (read more)

Thank you for noting this, and I obviously missed it, and have retracted the comment due to you pointing this out and other replies that changed my view. 

That said, I think that housing policy is the most obvious place for c(4) donations, given the local political nature of the work, but I'm frustrated that there's currently so little similar direct work on the national level in the US, for example, in farm animal welfare, in biosecurity and global health, and in AI policy. I think that more political engagement in each area by individual donors small... (read more)

(Original comment retracted, however,) I would think that staff members could recommend funding political campaigns without endorsing specific candidates. I think almost all of the hesitation of recommending doing so is the FTX fiasco and the impact on almost all of the political work that had been done in EA, which SBF was funding a large portion of - but I think that's a really bad reason not to pursue this type of work, albeit obviously not doing so with dubious campaign finance ethics, much less stolen customer funds.

I both like and deeply respect people at Rethink, and I think they are certainly worth supporting, but supporting their work is much more a form of pursuing value of information and viewpoint diversity than directly benefitting the world. (Also, I do not believe that they represent a consensus about what a cause-neutral viewpoint would be, as opposed to a principled but idiosyncratic view.)

If I were giving tens of millions of dollars, I would view non-trivial investment into VoI as a priority, but as an individual small dollar donor, I do not think that I ... (read more)

Thanks for engaging. Despite the fact that I don't expect this to sound convincing or fully lay out the case for anything, I'll provide a brief partial sketch of some of my tentative views in order to provide at least a degree of transparency around my reasoning - and I'm unlikely to expand on this much futher here, as I think this takes a lot of time and intensive discussion to transmit, and I have other things I should be working on.

First, I think you're accidentally assuming utilitarianism in my decisionmaking. I view my charitable giving from more of a... (read more)

Agreed. I was very excited, a few years ago, that a friend was able to talk to someone about having the local Tomchei Shabbos offer to pay for job certifications and similar training for people out of work and living off of those types of charity, in order to help them find jobs - as the Rambam says, this is the highest form of charity. So I think that concrete steps like this are absolutely possible, and worth pursuing if and when you can find them.

First, I'm not opposed to other drawing inspiration from my views!

Second, every time there is a disaster, I try to remind people that disaster risk mitigation is 3-10x as effective as disaster response. If you really want to give, I'd say you could support work like IPRED, rather than the red cross response teams. I don't know where specifically to give for that type of work, however, and I'd love for someone to do a deep dive on the most effective risk mitigation for disaster opportunities.

So if there is some technology which makes invading easier than defending or info-sec easier than hacking, it might not change the balance of power much because each actor needs to do both. If offense and defense are complements instead of substitutes then the balance between them isn’t as important.

This seems reasonable to explain many past data points, but it's not at all reassuring for bioweapons, which is a critical reason to be concerned about offense-defense balance of future technologies, and one where there really is a clear asymmetry. So to ... (read more)

This seems reasonable, but I'm always nervous about any program that is using carbon offsets, because there are significant financial incentives to game the system and take credit for reductions, instead of just reducing emissions. It's better than doing nothing, but if you're interested, there's an easy way to set up recurring monthly donations to the fund Vox recommended, here: https://www.givingwhatwecan.org/en-US/charities/founders-pledge-climate-change-fund

This is a super interesting point, and I'm completely unsure what it should imply for what I actually do, especially since returns are uncertain and prepaying at a discount under possible bankruptcy / extinction risk at an uncertain rate is hard - all of which (probably unfortunately) means I'm just going to keep doing the naive thing I've done so far.

That's a really interesting question, but I don't invest my charitable giving, though I do tithe my investment income, once gains are realized. My personal best guess is that in non-extinction scenarios, humanity's wealth increases in the long-term, and opportunities to do good should in general become more expensive, so it's better to put money towards the present.

There's a fair amount of discussion in AI alignment about what outer alignment requires, and how it's not just pursuing goals of a single person who is supposed to be in control.

As a few examples, you might be interested in some of these:
https://www.alignmentforum.org/posts/Cty2rSMut483QgBQ2/what-should-ai-owe-to-us-accountable-and-aligned-ai-systems

https://www.cs.utexas.edu/~pstone/Papers/bib2html-links/ASIMOV2021-REUTH.pdf 

https://www.lesswrong.com/posts/Tmvvvx3buP4Gj3nZK/learning-societal-values-from-law-as-part-of-an-agi 

It was crossposted via the alignment forum

I've found that if a funder or donor asks, (and they are known in the community,) most funders are happy to privately respond about whether they decided against funding someone, and often why, or at least that they think it is not a good idea and they are opposed rather than just not interested.

I don't necessarily care about the concept of personal identity over time, but I think there's a very strong decision-making foundation for considering uncertainty about future states. In one framing, I buy insurance because in some future states it is very valuable, and in other future states it was not. I am effectively transferring money from one future version of myself to another. That's sticking with a numerical identity view of my self, but it's critical to consider different futures despite not having a complex view of what makes me "the same perso... (read more)

I don't see a coherent view of people that doesn't have some version of this. My firstborn child was not a specific person until he was conceived, even when I was planning with my wife to have a child. As a child, who he is and who he will be is still very much being developed over time. But who I will be in 20 years is also still very much being determined - and I hope people reason about their contractualist obligations in ways that are consistent with considering that people change over time in ways that aren't fully predictable in advance.

More to the p... (read more)

Since any given future person only has an infinitesimally small chance of coming into existence, they have an infinitesimally weak claim to aid.

I think this is confused. Imagine we consider each person different over time, a la personites, and consider the distribution of possible people I will be next year. There are an incredibly large number of possible changes which could occur which would change my mental state, and depending on what I eat, the physical composition of my body. Does each of these future me have only an infinitesimal claim, and th... (read more)

See my reply to Yovel, but preferring violence to the status quo is very different than not wanting peace. And given that they keep getting bombed by Israel, it makes sense that they don't want to simply lay down arms - but per the last link, a majority supported continuing the ceasefire.

First, election of Hamas and Hamas affiliated seats is very different than support for Hamas. These were local representative elections, not a national party election like Israel. So the 58% number seems misleading. And the reason there have not been elections since 2006 has much more to do with Hamas being unwilling to have elections than you seem to think

Second, I think there is a critical difference between support for a political group and support for violence. Most Gazans did not want violence, and a majority of Palestinians, when polled, would accept... (read more)

The problem is that there is no endgame. As I said two days ago, we're repeating what the US did after 9/11, mistakes and all.

Strongly disagree.

Most people are the silent majority, who just want peace and don't have leverage to make it happen. That applies to the Palestinians and Israelis, most of whom don't know what to do, but want things to be peaceful and allow them to live happy lives. Unfortunately, political leaders on both sides have incentives to ensure that the easiest path to their near-term goals of staying in power involve continuing the cycle of violence. 

The critical path in my mind is prosperity - if Palestinians had a median income of Jordanians and Iraqis, ... (read more)

In the near term, misuse via bio doesn't pose existential risks, because synthetic bio is fundamentally harder than people seem to assume. Making a bioweapon is very hard, making one significantly worse than what previous natural diseases and bioweapons were capable of is even harder, and the critical path isn't addressed with most of the capabilities that narrow AI I expect is possible before AGI could plausibly do.

After that, I think that the risk from powerful systems is disjunctive, and any of a large number of different things could allow a malign act... (read more)

As someone who does lots of biorisk work, I disagree that this is the only likely catastrophic risk, as I note in my response, but event more strongly disagree that this is actually a direct extinction risk - designed diseases that kill everyone in the face of humans actually trying to stop them aren't obviously possible, much less findable by near-human or human level AI systems. 

Of course, combined with systemic fragility, intentional disinformation, and other attack modes enabled by AI, it seems plausible that a determined adversary with tremendous... (read more)

Just commenting to note that I saw this, and don't know that much about the different systems that exist - there are trade-offs, but those are probably project specific, and I don't use any of them heavily enough, or have enough background in project management, to offer any useful opinions.

I didn't say "blanket ban on ML," I said "a blanket ban on very large models."

Why? Because I have not seen, and don't think anyone can make, a clear criterion for "too high risk of doom," both because there are value judgements, and because we don't know how capabilities arise - so there needs to be some cutoff past which we ban everything, until we have an actual regulatory structure in place to evaluate. Is that different than the "conditional ban" you're advocating? If so, how?

Thanks, fixed!