All of Gavin's Comments + Replies

Gavin
7d95

Vouching for this, it's a wonderful place to work and also to hang out.

Gavin
20d30

A successor project is live here, takes all comers.

1
Tristan Williams
20d
Ah cool, thanks. Would probably include that at the top of the post for others who may be interested.
Gavin
1mo60

Scottish degrees let you pick 3 very different subjects in first year and drop 1 or 2 in second year. This seems better to me than American forced generalism and English narrowness.

2
Yonatan Cale
1mo
Maybe one day a university will let students study any topic they want from the internet, that would be rad
Gavin
1mo30

Thanks: you can apply here.

I've edited the post to link to the successor project.

Gavin
3mo61

I dream of getting a couple questions added onto a big conference's attendee application form. But probably not possible unless you're incredibly well-connected.

Gavin
3mo20

Oh that is annoying, thanks for pointing it out. I've just tried to use the new column width feature to fix it, but no luck.

Here's a slightly more readable gdoc.

Gavin
3mo40

it is good to omit doing what might perhaps bring some profit to the living, when we have in view the accomplishment of other ends that will be of much greater advantage to posterity.

 

- Descartes (1637)

Gavin
4mo20

Yes, if I was using the same implicature each time I should have said "MacAskill" for Guzey. Being associated with Thiel in any way is a scandal to some people, even though his far-right turn was after this talk.

It's not normative, it's descriptive - "shameable", not "ought to be ashamed".

Gavin
5mo102

I really think egoism strains to fit the data. From a comment on a deleted post:

[in response to someone saying that self-sacrifice is necessarily about showing off and is thus selfish]:

How does this reduction [to selfishness] account for the many historical examples of people who defied local social incentives, with little hope of gain and sometimes even destruction? 

(Off the top of my head: Ignaz Semmelweis, Irena Sendler, Sophie Scholl.)

We can always invent sufficiently strange posthoc preferences to "explain" any behaviour: but what do you gain in

... (read more)
2
Wei_Dai
5mo
Pure selfishness can't work, since if everyone is selfish, why would anyone believe anyone else's PR? I guess there has to be some amount of real altruism mixed in, just that when push comes to shove, people who will make decisions truly aligned with altruism (e.g., try hard to find flaws in one's supposedly altruistic plans, give up power after you've gained power for supposedly temporary purposes, forgo hidden bets that have positive selfish EV but negative altruistic EV) may be few and far between. This is just a reasonable decision (from a selfish perspective) that went badly, right? I mean if you have empirical evidence that hand-washing greatly reduced mortality, it seems pretty reasonable that you might be able to convince the medical establishment of this fact, and as a result gain a great deal of status/influence (which could eventually be turned into power/money). The other two examples seem like real altruism to me, at least at first glance. Question is, is there a better explanation than this?
Gavin
5mo92

This is a great question and I'm sorry I don't have anything really probative for you. Puzzle pieces:

  • "If hell then good intentions" isn't what you mean. You also don't mean "if good intentions then hell". So you presumably mean some surprisingly strong correlation. But still weaker than that of bad intentions. We'd have to haggle over what number counts as surprising. r = 0.1?
     
  • Nearly everyone has something they would call good intentions. But most people don't exploit others on any scale worth mentioning. So the correlation can't be too high.
     
  • Goo
... (read more)
Gavin
5mo102

I really think egoism strains to fit the data. From a comment on a deleted post:

[in response to someone saying that self-sacrifice is necessarily about showing off and is thus selfish]:

How does this reduction [to selfishness] account for the many historical examples of people who defied local social incentives, with little hope of gain and sometimes even destruction? 

(Off the top of my head: Ignaz Semmelweis, Irena Sendler, Sophie Scholl.)

We can always invent sufficiently strange posthoc preferences to "explain" any behaviour: but what do you gain in

... (read more)
Gavin
5mo53

I'm mostly not talking about infighting, it's self-flagellation - but glad you haven't seen the suffering I have, and I envy your chill.

You're missing a key fact about SBF, which is that he didn't "show up" from crypto. He started in EA and went into crypto. This dynamic raises other questions, even as it makes the EA leadership failure less simple / silly.

Agree that we will be fine, which is another point of the list above.

1
Michael Simm
5mo
Ah, thank you this does add good context. If I was an EA with any background in finance, I'd probably be very upset at myself about not catching on (a lot) earlier. Since he'd been involved in EA for so long,  I wonder if he never truly subscribed to EA principles and has simply been 'playing the long game'. I've seen plenty of examples of SBF being a master at this dumb game we woke westerners play where we say all the right shibboleths and so everyone likes us [https://unusualwhales.com/news/sbf-dms-with-vox-exposed].  I had heard of him only a few times before the crash, and mostly in the context of youtube clips where he basically described a Ponzi scheme, then said that it was 'reasonable' [https://www.bloomberg.com/news/articles/2022-04-25/sam-bankman-fried-described-yield-farming-and-left-matt-levine-stunned]. The unfortunate thing is that FTX's exchange business model wasn't inherently fraudulent. There was likely no way for anyone outside the company to know he was lending out users' money against their own terms of service (apart from demanding a comprehensive audit).  Ultimately it doesn't look like he's going to get away with it, but it's good to be much more cautious with funders (especially those connected to an operating non-public company) going forward. 
Gavin
5mo130

got karma to burn baby

Gavin
5mo30

Just shameable. 

1
Jeroen Willems
5mo
That makes things even clearer, thank you!
Gavin
5mo103

Thanks to Nina and Noah there's now a 2x2 of compromises which I've numbered:

The above post is a blend of all four.

Gavin
5mo10

Maybe people just aren't expecting emotional concerns to be the point of a Forum article? In which case I broke kayfabe, pardon.

Gavin
5mo30

Yeah it's not fully analysed. See these comments for the point.

The first list of examples is to show that universal shame is a common feature of ideologies (descriptive).

The second list of examples is to show that most very well-regarded things are nonetheless extremely compromised, in a bid to shift your reference class, in a bid to get you to not attack yourself excessively, in a bid to prevent unhelpful pain and overreaction. 

1
Jeroen Willems
5mo
The first comment you linked makes things a lot clearer, thanks. But I'm still curious how exactly you define "being compromised".
Gavin
5mo31

Good analysis. This post is mostly about the reaction of others to your actions (or rather, the pain and demotivation you feel in response) rather than your action's impact. I add a limp note that the two are correlated.

The point is to reset people's reference class and so salve their excess pain. People start out assuming that innocence (not-being-compromised) is the average state, but this isn't true, and if you assume this, you suffer excessively when you eventually get shamed / cause harm, and you might even pack it in.

"Bite it" = "everyone eventually ... (read more)

1
Noah Scales
5mo
Oh, I see. So by "benign" you mean shaming from folks holding common-sense but wrong conclusions, while by "deserved" you mean shaming from folks holding correct conclusions about consequences of EA actions. And "compromise" is in this sense, about being a source of harm.
Gavin
5mo110

There's some therapeutic intent. I'm walking the line, saying people should attack themselves only a proportionate amount, against this better reference class: "everyone screws up". I've seen a lot of over the top stuff lately from people (mostly young) who are used to feeling innocent and aren't handling their first shaming well.

Yes, that would make a good followup post.

Gavin
5mo30

We're not disagreeing.

1
Noah Scales
5mo
It could be that EA folks: 1. risk criticism for all actions. Any organization risks criticism for public actions. 2. deserve criticism for any immoral actions. Immoral actions deserve criticism. 3. risk criticism with risky actions whose failure has unethical consequences and public attention. EA has drawn criticism for using expected value calculations to make moral judgments. Is that the compromise you're alluding to when you write: SBF claimed that, if events had gone differently, FTX would have recovered enough funds to carry on. In that hypothetical scenario, FTX's illegal dealing with Alameda would have gone unnoticed and would have had no adverse financial consequences. Then the risk-taking is still unethical but does not inspire criticism. There is a difference between maximizing potential benefits and minimizing potential harms. It's not correct to say that minimizing unavoidable harms from one's actions has negative consequences for others and therefore those actions are immoral options, unless all one means by an immoral action is that the action had negative consequences for others. I don't think there's unanimity about whether actions should be taken to minimize harms, maximize benefits, or some combination. If all it means to "bite it" is that one takes actions with harmful consequences, then sure, everyone bites the bullet. However, that doesn't speak to intention or morality or decision-making. There's no relief from the angst of limited altruistic options in my knowing that I've caused harm before. If anything, honest appraisal of that harm yields the opposite result. I have more to dislike about my own attempts at altruism. In that way, I am compromised. But that's hardly a motive for successful altruism. Is that your point?
Gavin
5mo158

Good point thanks (though I am way less sure of the EU's sign). That list of examples is serving two purposes, which were blended in my head til your comment:

  1. examples of net-positive organisations with terrible mistakes (not a good list for this)
  2. examples of very well-regarded things which are nonetheless extremely compromised (good list for this)

You seem to be using compromised to mean "good but flawed", where I'm using it to mean "looks bad" without necessarily evaluating the EV.

Yet another lesson about me needing to write out my arguments explicitly.

9
Habryka
5mo
Yeah, to be clear, my estimates of EU impact have pretty huge variance, so I also wouldn't describe myself as confident (though I do think these days the expected value seems more solidly in the negative).  And yeah, that makes sense. 
Gavin
5mo109

yes. The fire is in an entirely different room.

5
Alexander Briand
5mo
bold to post memes on the EA forum. 
5
gavento
5mo
Or did you mean ...?  
Gavin
5mo42

Title: The long reflection as the great stagnation 

Author: Larks

URL: https://forum.effectivealtruism.org/posts/o5Q8dXfnHTozW9jkY/the-long-reflection-as-the-great-stagnation 

Why it's good: Powerful attack on a cherished institution. I don't necessarily agree on the first order, but on the second order people will act up and ruin the Reflection.

Gavin
5mo20

Title: Forecasting Newsletter: April 2222

Author: Nuno

URL: https://forum.effectivealtruism.org/posts/xnPhkLrfjSjooxnmM/forecasting-newsletter-april-2222 

Why it's good: Incredible density of gags. Some of the in-jokes are so clever that I had to think all day to get them; some are so niche that no one except Nuno and the target could possibly laugh.

Gavin
5mo20

Good question. Everyone feel free to have it in this thread

Gavin
5mo40

I take it the authors weren't anonymised? Not actually that important though.

1
Vael Gates
5mo
The authors were not anonymized, no.
Gavin
5mo40

https://twitter.com/sir_deenicus/status/1606360611524206592

Gavin
5mo20

Agree about the contest. Something was submitted but it wasn't about blowup risk and didn't rise to the top.

Gavin
5mo106

I personally only offer paid work trials, and this is the norm in the orgs I've seen (e.g. OpenPhil). I hope the answer is that the ones you experienced actually can't afford to do this (but I'm sure some could).

Gavin
5mo2418

Your reasoning in footnote 4 is sound, but note that practitioners often complain that OPT is much worse than GPT-3 (or even  GPT-NeoX) in qualitative / practical terms. Benchmark goodharting is real.

(Even so, this might be goalpost shifting, since GPT3!2022 is a very different thing from GPT3!2020.)

3
Ben Cottier
5mo
That's a good point, but I think goalpost shifting is likely not significant in this case, which supports your original point. The OPT paper [https://arxiv.org/pdf/2205.01068.pdf] compares to "GPT-3" (or "GPT" in the plots, as shorthand I guess) for the prompting and few-shot evaluations (section 3). It says on p.3: Also on p.3 they refer to "numbers reported by Brown et al. (2020)" But p.3 also mentions It sounds to me like they used the original results from Brown et al. (2020) where available, but evaluated using the Davinci API as a cross-check or fallback. In contrast, the paper talks about "Davinci" for the evaluations in subsequent sections, so this is presumably the API version of GPT-3 that was available at the time. It says on p.5 that "We compare primarily against GPT-3 Davinci, as these benchmarks were not yet available to be included in Brown et al. (2020)." I didn't include these other evaluations (e.g. Bias and Toxicity) in my analysis; I'm just pointing this out to support my guess that the evaluations in section 3 are comparing to the original GPT-3.
3
Ben Cottier
5mo
Thanks for raising this. On reflection, I think if I had started this project now (including re-considering my definition of "successful replication") I probably would not have classed OPT-175B as a successful replication. I probably should flag this clearly in the post. As noted in point 2(d) of the final section of the post, I was more-or-less sitting on this report for a few months. I made significant revisions during that period, but I was paying less attention to new evidence than before, so I missed some evidence that was important to update on. 
4
Gavin
5mo
https://twitter.com/sir_deenicus/status/1606360611524206592
Habryka
5mo102

I also wanted to say that from talking to a bunch of people and reading ML blogs/reddits/Twitter that my impression was that OPT is much worse than GPT-3, despite similar performance on some of the benchmarks, so I think this comparison is pretty off. 

Gavin
6mo180

Looks like we have a cost-saving way to prevent 7 billion male chick cullings a year.

I snipe at accelerationist anti-welfarists in the thread, but it's an empirical question whether removing horrifying parts of the horrifying system ends up delaying abolition and being net-harmful. It seems extremely unlikely (and assumes that one-shot abolition is possible) but I haven't modelled it.

Gavin
6mo70

Greg and crew: Thanks for all your work, happy to see you made it through the pandemic.

Others: If you want a third-party opinion on what it's like, DM me.

Gavin
6mo258

I like all of your suggested actions. Two thoughts:


1) EA is a both a set of strong claims about causes + an intellectual framework which can be applied to any cause. One explanation for what's happening is that we grew a lot recently, and new people find the precooked causes easier to engage with (and the all-important status gradient of the community points firmly towards them). It takes a lot of experience and boldness to investigate and intervene on a new cause.

I suspect you won't agree with this framing but: one way of viewing the play between these tw... (read more)

3
Siobhan_M
6mo
Thanks for the comment - this and the other comments around cause neutrality have given me a lot to think about! My thoughts on cause neutrality (especially around where the pressure points are for me in theory vs. practice) are not fully formed; it's something I'm planning to focus a lot on in the next few weeks, in which time I might have a better response. 
Gavin
6mo40

On AI quietism. Distinguish four things:

  1. Not believing in AGI takeover.
  2. Not believing that AGI takeover is near. (Ng)
  3. Believing in AGI takeover, but thinking it'll be fine for humans. (Schmidhuber)
  4. Believing that AGI will extinguish humanity, but this is fine. 
    1. because the new thing is superior (maybe by definition, if it outcompetes us). 
    2. because scientific discovery is the main thing

(4) is not a rational lack of concern about an uncertain or far-off risk: it's lack of caring, conditional on the risk being real.

Can there really be anyone in category (... (read more)

2
RyanCarey
6mo
(4) was definitely the story with Ben Goertzen and his "Cosmism". I expect some "a/acc" libertarian types will also go for it. But it is and will stay pretty fringe imo.
Gavin
6mo50

Great work, and I was just about to ask for the code.

I think including personal fit (with say a 5 or 6 OOM range) will flip the sign on this though. Would also be good to show the intervals.

Gavin
6mo30

Seems like it would contribute to the profitability and feasibility of factory farming.

2
Denise_Melchin
6mo
That makes sense! I failed to think of non-human applications. Edit: "economically crucial" should have been a hint.
Gavin
7mo20

Yep ta, even says so on page 1. 

Gavin
7mo20

Ord's undergrad thesis is a tight argument in favour of enlightened argmax: search over decision procedures and motivations and pick the best of those instead of acts or rules.

6
jh
7mo
Interesting thesis! Though, it's his doctoral thesis, not from one of his bachelor's degrees, right?
0
Doomed
4mo
Someone at spaceX is taking meaningful action to mitigate this, thankfully. https://www.reuters.com/business/aerospace-defense/spacex-curbed-ukraines-use-starlink-internet-drones-company-president-2023-02-09/ [https://www.reuters.com/business/aerospace-defense/spacex-curbed-ukraines-use-starlink-internet-drones-company-president-2023-02-09/] Maybe seeing the Russian sat throw debris is what it took to ask the 'so...about our constellation' question: https://www.businessinsider.com/russian-satellite-breaks-up-orbit-space-debris-could-last-century-2023-2?utm_source=reddit.com [https://www.businessinsider.com/russian-satellite-breaks-up-orbit-space-debris-could-last-century-2023-2?utm_source=reddit.com] Thanks for the downvotes everyone!
Gavin
7mo20

3. Tarsney suggests one other plausible reason moral uncertainty is relevant: nonunique solutions leaving some choices undetermined.  But I'm not clear on this.

Gavin
7mo20

Excellent comment, thanks! 

Yes, wasn't trying to endorse all of those (and should have put numbers on their dodginess).

1. Interesting. I disagree for now but would love to see what persuaded you of this. Fully agree that softmax implies long shots.

2. Yes, new causes and also new interventions within causes.

3.  Yes, I really should have expanded this, but was lazy / didn't want to disturb the pleasant brevity. It's only "moral" uncertainty about how much risk aversion you should have that changes anything. (à la this.)

4. Agree.

5. Agree.

6. I'm usin... (read more)

Gavin
7mo20

Not in this post, we just link to this one. By "principled" I just mean "not arbitrary, has a nice short derivation starting with something fundamental (like the entropy)".

Yeah, the Gittins stuff would be pitched at a similar level of handwaving.

Gavin
7mo70

Looking back two weeks later, this post really needs

  • to discuss of the cost of prioritisation (we use softmax because we are boundedly rational) and the Price of Anarchy;
  • to have separate sections for individual prioritisation and collective prioritisation;
  • to at least mention bandits and the Gittins index, which is optimal where softmax is highly principled suboptimal cope.
2
MichaelStJules
7mo
FWIW, I didn't get the impression there's a very principled justification for softmax in this post, if that's what you intended by "highly principled". That it might work better than naive argmax in practice on some counts isn't really enough, and there wasn't really much comparison to enlightened argmax, which is optimal in theory. I'd probably require being provably (approximately) optimal for a principled justification. Quickly checking bandits and the Gittins index on Wikipedia, bandits are general problems and the Gittins index is just the value of the aggregate reward. I guess you could say "maximize Gittins index" (use the Gittins index policy), but that's, imo, just a formal characterization of what enlightened argmax should be under certain problem assumptions, and doesn't provide much useful guidance on its own. Like what procedure should we follow to maximize the Gittins index? Is it just calculate really hard? Also, according to the Wikipedia page, the Gittins index policy is optimal if the projects are independent, but not necessarily if they aren't, and the problem is NP-hard in general if they can be dependent.
Gavin
7mo20

Yeah could be terrible. As such risks go it's relatively* well-covered by the military-astronomical complex, though events continue to reveal the inadequacy of our monitoring. It's on our Other list.

* This is not saying much: on the absolute scale of "known about" + "theoretical and technological preparedness" + "predictability" + "degree of financial and political support" it's still firmly mediocre.

-2
Doomed
7mo
Russian arms control officials have now made public statements suggesting that commercial space infrastructure that is used to support the conflict may be a legitimate target. EA did the analysis on alienating billionaires, so nobody is going to mock a US billionaire who wants to colonize space, but deployed a commercial sat swarm that is now being talked about as a valid military target. I'm guessing nobody funded by EA is putting the work in from an engineering standpoint to see if there's an existential risk there. There are no new physics required, just engineering analysis. An engineer at a relevant firm could answer the questions. What breaks their system, how much debris does that course of action generate, is their constellation equipped to avoid cascading failure due to debris, what would be the impact on launch windows for high orbits of the worst case scenario? I guess it has been done already and everything is totally fine, let's focus on other stuff, no need to call this an emergency.
Gavin
7mo60

We will activate for things besides x-risks. Besides the direct help we render, this is to learn about parts of the world it's difficult to learn about any other time.

Yeah, we have a whole top-level stream on things besides AI, bio, nukes. I am a drama queen so I want to call it "Anomalies" but it will end up being called "Other".

Gavin
8mo20

We're not really adding to the existing group chat / Samotsvety / Swift Centre infra at present, because we're still spinning up. 

My impression is that Great Power stuff is unusually hard to influence from the outside with mere research and data. We could maybe help with individual behaviour recommendations (turning the smooth forecast distributions of others into expected values and go / no-go advice).

-1
Doomed
7mo
Anyone thinking about this? A kessler syndrome of sufficient severity prevents spacecraft from leaving Earth for, depending on its' duration, centuries to millennia. A kessler cascade will eventually result in such a syndrome, it's only a question of time, and this can be estimated by looking at the slope on the graph of the rate of debris increase from collisions. This slope is easy to increase and hard to decrease. Starlink has a lot of small satellites in orbit. Starlink is carrying communications for a party to a terrestrial conflict, these may include military communications. A different party to the conflict, wishing to deny use of the constellation for the military communications of its' enemy, may take actions to degrade or destroy the constellation in the course of the war. Are there classes of action that would sufficiently degrade Starlink, such that it is no longer suitable for use as a communications platform for the party to the conflict, which would lead to a near term kessler syndrome? Optimistic: No, there's no risk to manned spaceflight of any action that could be taken against the Starlink constellation, including kinetic destruction of its' spacecraft in their current locations. Pessimistic: any damage to the constellation or its' control systems results in an immediate kessler syndrome, which prevents manned spacecraft from ascending to the high (or escape) orbits required to colonize the solar system. SpaceX engineers should be able to definitively answer this question. In the most pessimistic case, the kessler syndrome will outlive terrestrial energy resources and/or climate reserve, so the human race will end starving, buried in our waste.
Gavin
8mo20

Got you! Pardon the delay, am leaving confirmations to the director we eventually hire.

Gavin
8mo20

Been trying! the editor doesn't load for some reason.

2
peterhartree
8mo
Maybe a client-side content blocker on your end? Works fine for me today.
Gavin
8mo20

Yeah we're not planning on doing humanitarian work or moving much physical plant around. Highly recommend ALLFED, SHELTER, and help.ngo for that though.

3
Charles He
8mo
Your comment isn't a reply and reduced clarity.  This is bad, since it's already hard to see the nature of the org suggested in my parent comment and this further muddies it. Answering your comment by going through the orgs is laborious and requires researching individual orgs and knocking them down, which seems unreasonable. Finally, it seems like your org is taking up space for this org.   ALLFED has a specific mission that doesn't resemble the org in the parent comment. SHELTER isn't an EA org, it provides accommodations for UK people? It's doubtful help.ngo or its class of orgs occupy the niche—looking at the COVID-19 response gives some sense of how a clueful org would be valuable even in well resourced situations.  To be concrete, for what the parent org would do, we could imagine maintaining a list of crises and contingent problems in each of them, and build up institutional knowledge in those regions, and  preparing a range of strategies that coordinate local and outside resources. It would be amazing if this niche is even partially well served or these things are done well in past crises. Because it uses existing interest/resources, and EA money might just pay for admin, the cost effectiveness could be very high. This sophistication would be impressive to the public and is healthy for the EA ecosystem. It would also be a "Task-Y" and on ramp talent to EA, who can be impressive and non-diluting.    It takes great literacy and knowledge to make these orgs work, instead of deploying money or networking with EAs, it looks outward and brings resources to EA and makes EA more impressive.   Earlier this year, I didn't writeup or describe the org I mentioned (mostly because writing is costly and climbing the hills/winning the games involved uses effort that is limited and fungible), but also because your post existed and it would be great if something came out of it. I asked what an AI safety alert org would look like. As we both know, the answer is that n
Gavin
8mo50

Yeah, we're still looking for someone on the geopolitics side. Also, Covid was a biorisk. 

1
Guy Raveh
7mo
Cool! Yes, but it wasn't an existential biorisk. And I assume once you include risks which are catastrophic but not existential, you also get things which aren't AI/pandemics. So that's what I was trying to say.
Load more