NickLaing's Quick takes

A lot of modern training data isn’t stolen, though. There are organisations which recruit people to do their jobs normally and screen share, or provide worked-through examples of their work, and this is increasingly making up the bulk of the data that’s used to pull frontier models ahead of others on work benchmarks. People are being paid for this and do it willingly, usually with knowledge of where their labour outputs are going!

So really, the problem is a subset of workers in each field are ‘defecting’ (to use a rat term I kinda loathe). How do you create solidarity among groups of workers to prevent a small number of them from putting the others out of work? Or, if technological progress is to be necessary, how do those groups of workers politically agitate for a welfare state and good ongoing education?

The left solved this problem two hundred years ago, but I suspect EA won’t like the solution…

In general it's okay for a person to look at a dozen different paintings and then make a new painting that's kind of like those paintings. This seems pretty analogous to what AI is doing and I'm not sure why it becomes not OK if it's done by a corporation training an AI model. Perhaps there are specific violations of IP laws, and those can be discussed (and some of them are being adjudicated in court as we speak), and of course there is a separate question of whether those IP laws are just. However, what AI models doing to me seems mostly like it's the sort of behavior we would be OK with individual people doing: i.e closer to the remixing/synthesizing end of the spectrum than the copying/"stealing" end.

It would be quite funny for my intern to be the highest-paid person at my org if they keep that salary 😭

Dean Ball's commentary on this refamed the issue for me https://www.hyperdimensional.co/p/clawed

The big difference, however, is that Anthropic is essentially using the contractual vehicle to impose what feel less like technical constraints and more like policy constraints on the military. Think of the difference between “this fighter jet is not certified for flight above such-and-such an altitude, and if you fly above that altitude, you’ve breached your warranty,” and “you may not fly this jet above such-and-such an altitude”). It is probably the case that the military should not agree to terms like this, and private firms should not try to set them.

But the Biden Administration did agree to those terms, and so did the Trump Administration, until it changed its mind. That alone should make one thing clear: terms like this are not some ridiculous violation of the norms of defense contracting...

The contract was not illegal, just perhaps unwise, and even that probably only in retrospect. Note that this is true even if you agree with the underlying substance of the limitations. You can support restrictions on mass domestic surveillance and lethal autonomous weapons, but disagree that a defense contract is the optimal vehicle to achieve that policy outcome. The way you achieve new policy outcomes, under the usual rules of our republic, is to pass a law...

I agree that there's something iffy/non-democratic in theory about putting that kind of constraint around the Pentagon, and that it would have been prudent for them to decline it in the first place. An analogy I read on Substack: if an epidural manufacturer told a government hospital "you're welcome to use our drug so long as you don't use it in any abortions," it would probably be prudent to decline that contract (too much overhead). 

 Anyway this reframing put one sentence in particular by Dario into a new light: "To the extent that such surveillance is currently legal, this is only because the law has not yet caught up with the rapidly growing capabilities of AI." In other words, because we know what the law should be and what it's probably going to be, we should implement that policy today. I think many of us can think of examples where we'd be uncomfortable with a billionaire tech CEO saying that. 

That is encouraging! Scott's post linking to various prediction markets for Antrhopic's implied valuations was also heartening.

100% agree with this!

Given your expertise is in global health, I do think it's likely that you're less well-calibrated on how reasonable your animal welfare comments are relative to your global health ones! So you may think it's a reasonable critique but someone who is a die-hard animal person may have already thought about your comment and know there is a common counterpoint that negates it (which you haven't heard yet). Obviously, the inverse could be true for global health comments.

But I agree that this shouldn't have been downvoted on karma grounds!

(Also, sometimes your comments do give me "I am sceptical of most things animal welfare" vibes, so people might be reacting to a real or perceived difference in values about how much animals matter).

haha I can confirm I did not karma knock you and I was kind of surprised you had gotten so downvoted! I actually upvoted when I saw that to counteract. 

One random thought I'll add is that since you are most experienced (afaict?) in ghd, I'd expect your arguments to be at their best in that context, so you getting upvoted on GHD and downvoted on AW is at least consistent with having more expertise in one than the other, so not necessarily evidence that AW folks are more sensitive. Although I'm not ruling that out!

The other thing I'm not sure I understand is how much weight a single individual's downvote can have - is there any chance that a few AW people have a ton of karma here, so that just a few people downvoting can take you negative in a way that wouldn't happen as much in GHD? 

Just wanted to say, as someone from the animal side of things, that if this is indeed the case (no strong opinion either way), I'm sorry this happened to you and I appreciate that you're pointing this out. Separating agreement from post quality is something I deeply value in the discourse here, and we should police the correct use of voting.

As a (wild) animal welfare person, I am disappointed to see this. Your comment was thoughtful and well-intentioned.

It doesn't apply here, but in general I expect animal welfare people are more likely to disapprove of certain views, or take more of a combative attitude to public debates, because so much of normal discourse sneaks in speciesist assumptions and is actively harmful to animals. But I don't think that's the explanation here - I largely agree with your comment.

To respond to your original comment, I think with a bit of creativity you will be able to find politically tractable interventions. For example, people tend to view humane management of animals in cities quite positively. There's also a growing movement for compassionate conservation. It's more focused on doing no harm than actively helping wild animals, but at least it is a movement towards thinking about the welfare of wild animals. I do think that there will often be a tradeoff between effectiveness and political tractability though, and it may be worth pursuing sub-optimal interventions for a while in order to gain greater political momentum towards helping wild animals.

I have no idea how much this kind of voting behaviour varies from discussions about one cause area to another, but I can confirm that this happens a lot in various discussions on multiple different topics. People seem keen to karma downvote/strongly downvote posts or comments they disagree with, regardless of quality. I surmise this is probably just people being mad and having a knee-jerk reaction, in a lot of cases.

I recently had an experience where someone was using a technical term incorrectly, and I left two comments trying to correct the use of this term. These comments didn't get responded to, but they did get strongly downvoted. It was enough that it triggered me getting rate limited from commenting on the forum! This just seems like a case where someone felt spiteful that I pointed out they made a mistake.

That's one example, but people do this all the time. Another recent example is here, where I didn't notice or complain about the downvoting, but someone else noticed it and complained about it on my behalf. 

People in this community are only human, and we're affected by the same impulses as anybody else. We have the same cognitive/psychological biases as anyone else and the same problems with emotional self-regulation as anyone else. It takes a certain amount of discipline for me to remain judicious about my downvotes (or, on rare occasions, quickly undo a downvote I made in a moment of irritation).

To the extent it's a real problem, it's a problem where it's not a one-off spar between two people (or a few), but where it's systemic around certain ideas or topics. I get the impression people sort of knee-jerk/undiscerningly upvote posts and comments that support the narrative that near-term AGI is likely, and downvote posts and comments that challenge this narrative, even when the actual content is narrow in scope, e.g. pointing out a mistake in something that promotes a near-term AGI narrative.

If you want to express mainstream North American liberal/progressive views on social justice-related topics, forget about it. The downvotes will fall like a cloud of daggers. (The Overton window of the EA Forum is a bit skewed in that regard — and it's extremely skewed for the LessWrong users who come over here — where some people will defend views they concede are or might be offensive and wrong, such as scientific racism or white nationalism, on the grounds of free speech or neutrality or intellectual freedom or whatever, but do not extend the same defense to the sort of bog standard social justice views that mainstream liberal or progressive politicians or academics generally affirm. I think Kamala Harris or Barack Obama might get downvoted on the EA Forum talking about systemic racism, sexism, or trans rights in the ways they already do in public appearances, and an authoritarian white nationalist like Curtis Yarvin — who many people on LessWrong seem to be a fan of — might get upvoted for some edgy take about Black-on-Black crime or race and IQ or something.)

I find the ways that I see people on the EA Forum circle the wagons around near-term AGI and social justice quite demented — sorry to be so blunt. Someone can make a post that says we should try to contact aliens to warn them about AGI and that gets +27 karma, but if you point out someone used a term from machine learning incorrectly that gets 0 or negative karma, and if you get into arguments about social justice you will get negative karma about the half the time, seemingly regardless of what you say (at least, I can't figure out the pattern, if there is one). It just seems like if you are willing to entertain really strange ideas like the aliens thing, you should also be willing to entertain mainstream ideas that many experts endorse — I mean specifically things that don't go against EA's core principles and haven't been discussed to death already, about which there isn't already a consensus within the community.

I guess it doesn't surprise me that people might knee-jerk downvote entirely reasonable comments about the limited market potential for humane insecticides or things of that nature, since that's similar. Maybe it's an example of in-group polarization, where people want to fiercely defend an opinion that's extreme relative to the general population or out-group. 

I also thought your comment didn't deserve to get downvoted :'(, even though I disagreed and thought it partly missed my point (I ofc didn't downvote it, tho). Even the number of upvotes of Mal's comment responding to yours feels a bit violent, actually. I think people should maybe hold off from upvoting when it's not necessary. They can just agree-vote.

I think Mal's, James', and Tristan's potential explanations for why this happened are pretty plausible.

But, also, as I suggest in response to Mal, it's probably just one single person, so :shrug:, I guess. :)

I broadly agree with this! 

At the same time, I'd flag that I'm not quite sure how to frame this.

If I were a donor to 80k, I'd see this action as less "80k did something nice for the EA community that they themselves didn't benefit from" and more "80k did something that was a good bet in terms of expected value." In some ways, this latter thing can be viewed as more noble, even though it might be seen as less warm.

Basically, I think that traditional understandings of "being thankful" sort of break down when organizations are making intentional investments that optimize for expected value.

I'm not at all saying that this means that these posts are less valuable or noble or whatever. Just that I'd hope we could argue that they make sense strictly through the lens of EV optimization, and thus don't need to rely as much on the language of appreciation.

(I've been thinking about this with other discussions) 

Change in strategic direction?

It's true that meat eating is closer to what we actually care about, but it's worth singling out causal pathways from saving lives and increasing incomes/wealth, as potential backfire effects. "Meat eating problem" seems likely to be understood too generally as the problem of animal consumption, without explanation. I'd prefer a more unique expression to isolate the specific causal pathways.

Some other ideas:

1. meat eating backfire (problem)
2. more meat backfire/problem
3. meat backfire (problem)
4. (more) animal product backfire (problem)

(Eggs and other animal products besides meat matter, too.)

When we were talking about this in 2012 we called it the "poor meat-eater problem", which I think is clearer.

I think 'meat-eating problem' > 'meat-eater problem' came in my comment and associated discussion here, but possibly somewhere else.^[1]

1. ^{^}

   (I still stand by the comment, and I don't think it's contradictory with my current vote placement on the debate week question)

I think it's totally fair name of the problem, as its "unfairness" comes from the problem statement, not its name. "I think its pretty unfair to lump the label of a serious philosophical problem on the poorest people on earth" here for example, it's meat eater problem being morally icky, not its name.

1. Accuracy: I don’t think the core problem actually the people who’s lives we are saving, its that they then eat meat and cause suffering. I think its important to separate the people from the core problem as this better helps us consider possible solutions

The main takeaway of the 'meat eater problem' (sorry!) is to reassess the cost-effectiveness of saving human lives, not necessarily to argue that we should focus on reducing animal consumption in lower-income countries. While reducing animal consumption is important, that’s not typically the central takeaway from this specific 'problem'.

In this sense, the saving lives aspect is more central to the problem than the meat consumption aspect, though both are pivotal. So, in a purely logical sense, the term 'meat eater problem' might actually be more accurate.

Thanks so much for reading, Nick.

It really is nearly impossible as an individual consumer to figure out which of the many products we interact with everyday are safe, and which might be contaminated, unfortunately... so hoping we can collectively make progress on these things at a systems level soon!

On a related topic, I'm curious whether you see any geophagia (soil consumption) among pregnant women, or other people, in Northern Uganda? It's fairly common in Kenya and Malawi and we've unfortunately seen that the soils (which are often compacted, so they look like small stones) frequently contain lead levels well above what you'd want to see in something that's being directly consumed.

-- Isabel Arjmand (cofounder)

This is incredibly impressive for a first video on a new channel. Great work 80kh!

In ordinary language, I wouldn't generally consider something that gets 1% of resources to be a "priority." Applying your reasoning above, that would create a theoretical maximum of 100 "priorities" and a more realistic range of perhaps 10-40. As we move beyond the low teens, the idea of a "priority" gets pretty watered down in my book.

Does this mean you recently had a baby? If so, congratulations! 

I feel mixed about ai-writing detection for a few reasons. I have very few issues with someone putting the bullet points of their argument into ai and then reading/editing/discussing a few times the response and letting the ai write it. I also think there is value in just putting your messy thoughts out as you have them and not having everything polished, but it depends on the situation.

Also separately I'm worried ai-writing detector proliferation will just speed up "immunity".  I don't think there is something deep and fundamental that stops ai from writing e.g. exactly what I have written to this point.  You can already download all your writing, ask ai to summarize it, make a text file that precisely describes your style and then ask the ai to write something in your voice. I've done this and yes they still have a bit of that vanilla llm feel but if there is actual market demand for solutions it doesn't seem like this is an insurmountable problem. 

I think people should say when they used ai and to what degree and there should be an expectation that just because polished writing is cheaper than it used to be you will not pollute the forum with things that you have not thought an appropriate amount about. 

I think folks should be free to choose what level of AI use they're comfortable with, without fear of being shamed or outed inherently for using AI. Those who don't approve are free to downvote or 'x' to show their discontent. If enough people feel the same way then the incentives will do their job. 

As I said on LinkedIn, you are still engaging with a human... a human that's using AI. 

I see AI as an interesting leveller of the playing field. It gives people less room to dismiss a point of view for being conveyed haphazardly in writing. 

Writing well is a skill, built on the existential privilege of intelligence. If you don't have it, does it mean you have less right to be heard? 

We're actually currently working on updating our policy on AI-generated content, so this thread (and follow up poll) is helpful! :)

I disagree pretty strongly with this. 

Although there are tradeoffs associated with AI writing, mostly being able to produce content that can appear polished and well-considered when it is not, I think AI's enabling the proliferation of good thoughts and ideas that would otherwise just never happen far outweighs this. 

Going back and forth with AI, reviewing, and drafting can turn a writing process that might take several days to a week or more, into an hour or two, or less. This enables me, and I'm sure others, to share content and ideas that otherwise we would not be able to.

Removing the barriers to people sharing their thoughts quickly and effectively is probably how we get more new and impactful ideas out there. I've been pretty sad at the sort of witch-huntery I've been seeing about AI generated content.

my first post on LessWrong was scrapped because they identified it as AI written

I'm surprised to read this, can you check your post on https://www.pangram.com/ ?

Have made a related poll  :).

https://forum.effectivealtruism.org/posts/evTZ2mguA7eg4Kvmm/how-much-of-a-post-are-you-comfortable-for-ai-to-write
 

Yes it's quite exciting! One quick thought: I don't think SMA/Bregman would be very pleased if EAs regularly started using SMA's videos and ideas in their outreach (because they want to keep themselves quite separate). Maybe he's changed his view on that (they're organising an event with Singer and GWWC shortly), but it was certainly the case a year or so ago. If anyone's considering doing this on a significant scale, I'd suggest checking with them first. 

Worth noting: Peter Singer and Rutger Bregman’s School for Moral Ambition are co-hosting a Profit for Good conference in Amsterdam on 11 June—a concrete EA-adjacent collaboration that channels Bregman’s “moral ambition” into effective-charity business models. Another good touchpoint for anyone looking to ride this wave.

https://www.moralambition.org/profit-for-good-conference-live-stream

I think for most people applying to their fellowships would be the best way to collaborate with SMA to do good (as he mentions in the video)

I'm also very keen to get more global health content! One thing people can do to help is suggest bloggers/ substackers etc... who write on global health topics relevant to EA. I can talk to them and offer them automated crossposting and support. 

Hoping to post some shallow takes on global health and development this year!

The CE of redirecting money is simply (dollars raised per dollar spent) * (difference in CE between your use of the money vs counterfactual use). So if GD raises $10 from climate mitigation for every $1 it spent, and that money would have otherwise been neutral, then that's a cost-effectiveness of 10x in GiveWell units.

There's nothing complicated about estimating the value of leverage. The problem is actually doing leverage. Everyone is trying to leverage everyone else. When there is money to be had, there are a bunch of organizations trying to influence how it is spent. Melinda French Gates is likely deluged with organizations trying to pitch her for money. The CEAP shutdown post you mentioned puts it perfectly:

The core thesis of our charity fell prey to the 1% fallacy. Within any country, much of the development budget is fixed and difficult to move. For example, most countries will have made binding commitments spanning several years to fund various projects and institutions. Another large chunk is going to be spent on political priorities (funding Ukraine, taking in refugees, etc.) which is also difficult for an outsider to influence.

What is left is fought over by hundreds, if not thousands of NGOs all looking for funding. I can’t think of any other government budget with as many entities fighting over as small a budget. The NGOs which survive in this space, are those which were best at getting grants. Like other industries dependent on government subsidies, they fight tooth and nail to ensure those subsidies stay put.

This doesn't mean that leverage is impossible. It just means that leverage opportunities tend to be specific and limited. We have to take them on opportunistically, rather than making leverage a theory of impact.

On the subject of redirecting streams of money from less impactful causes to EA causes, I feel I need to beat my drum regarding the potential of Profit for Good businesses (businesses with charities in all or almost all of the shareholder position). In such cases, to the extent EA PFGs profits displace those of normal businesses, funds are diverted from the average shareholder to an effective charity. 

So when a business like Humanitix (PFG helping projects in the developing world, $4mil AUD to The Life You Can Save) displaces the marketshare of Ticketmaster, funds are diverted not from charities, but from the funds of the business's competitors. This method of diversion seems less difficult because the operative actors (consumers, employees, business partners) are not deciding between a strong non-EA charity often optimized for warm fuzzies and marketing, but rather choosing between products with similar value propositions, but where engaging with one - in addition to the other value proposition - implies helping fight malaria or something instead of enriching a random investor. 

If you're interested in learning more about Profit for Good, here is a reading list on the subject.

"AIDS related deaths in the next five years will increase by 6.3 million" if funding is not restored, UNAIDS executive director Winnie Byanyima said.

This is a quote from a BBC news article, mainly about US political and legal developments. We don't know what the actual statement from the ED said, but I don't think there's enough here to infer fault on her part. 

For all we know, the original quote could have been something like predicting that deaths will increase by 6.3 million if we can't get this work funded -- which sounds like a reasonable position to take. Space considerations being what they are, I could easily see a somewhat more nuanced quote being turned into something that sounded unaware of counterfactual considerations.

There's also an inherent limit to how much fidelity can be communicated through a one-sentence channel to a general audience. We can communicate somewhat more in a single sentence here on the Forum, but the ability to make assumptions about what the reader knows helps. For example, in the specific context here, I'd be concerned that many generalist readers would implicitly adjust for other funders picking up some of the slack, which could lead to double-counting of those effects. And in a world that often doesn't think counterfactually, other readers' points of comparison will be with counterfactually-unadjusted numbers. Finally, a fair assessment of counterfactual impact would require the reader to understand DALYs or something similar, because at least a fair portion of the mitigation is likely to come from pulling public-health resources away from conditions that do not kill so much as they disable.

So while I would disagree with a statement from UNAIDS that actually said if the U.S. doesn't fund PEPFAR, 6.3MM more will die, I think there would be significant drawbacks and/or limitations to other ways of quantifying the problem in this context, and think using the 6.3MM number in a public statement could be appropriate if the actual statement were worded appropriately. 

The only scenario where this could happen is if all of these people went completely untreated, which means that no local government would come in at any stage. This scenario is impossible

Can you elaborate why this is impossible, or at least unlikely?

This is huge. TRIP has a very wide listenership, particularly of UK Politics nerds, politicians, journalists, and policy workers.

Who is considering Altman and Hassabis thought leaders in AI safety? I wouldn't even consider Altman a thought leader in AI - his extraordinary skill seems mostly social and organizational. There's maybe an argument for Amodei, as Anthropic is currently the only one of the company whose commitment to safety over scaling is at least reasonably plausible.

I think thoughtleader sometimes means "has thoughts at the leading edge" and sometimes mean "leads the thoughts of the herd on a subject" and that there is sometimes a deliberate ambiguity between the two.

I'm reading "OK" as "morally permissible" rather than "not harmful". E.g., I think it's also "OK" to eat meat, even though I think it's causing harm.

(Not saying you should clarify the poll, it's clear enough and will probably produce interesting results either way!)