All of Yitz's Comments + Replies

that's a fair point, I'm reconsidering my original take.

oof

This was a really fun read; thanks for helping put it together!!

+1 from me.

 I was talking about the whole situation with my parents, and they mentioned that their local synagogue experienced a very similar catastrophe, with the community's largest funder turning out to be a con-man. Everybody impacted had a lot of soul-searching to do, but ultimately in retrospect, there was really nothing they could or should have done differently—it was a black-swan event that hasn't repeated in the quarter of a century or so since it happened, and there were no obvious red flags until it was too late. Yes, we can always find de... (read more)

Same here, this is really helping me understand the (at least perceived) narrative flow of events

Thank you for sharing, I can understand why you might be feeling burnt out!! I've been in a workplace environment that reminds me of this, and especially if you care about the people and projects there...it's painful.

Strongly agree that moving forward we should steer away from such organizational structures; much better that something bad is aired publicly before it has a chance to become malignant

this seems probable to me, thanks for sharing a good-faith explanation

+1 to this, can attest I've done the same, and immediately regretted it lol

Curious what you think about screenshots like this one, which I've now seen in a few different places.

This is a fair critique imo, I'm updating against SBF using EA for sociopathic reasons. That being said, only slightly updating towards him using EA ideology as his main motivator to commit fraud, as that still may very well not be the case.

I'll be honest, I've been putting judgement based on his (apparent) lifestyle on hold, as I've seen some anecdotes/memes floating around twitter suggesting that he may not have been honest about his veganism/other lifestyle choices. I don't know enough about that situation to distinguish the actual truth of the matter, so it's possible I've been subject to misinformation there (also I scrolled past it quickly on Twitter and it's possible it was some meta-ironic meme or something). If there is (legitimate) evidence he was actually faking it, that would make me update strongly in the other direction, of course.

I got a free textbook  (that cost some insane amount of money on Amazon) once from a professor when I asked if he could share a copy of his work for reference in a video game I was making at the time. I don't know if that counts, but seems worth mentioning.

It certainly isn't a good outcome for EA either way, and I don't want us prematurely absolving ourselves of any responsibility we may end up holding. I just want to be as clear-thinking about this as possible, so we can best mend ourselves moving forward.

Thanks for this; it's a nicely compact summary of a really messy situation that I can quickly share if necessary.

I have a sticker by my bed reading 'What Would SBF Do?' (from EAG SF 2022) (I should probably remove that)

Maybe don't remove that—this seems emblematic of a category of mistakes worth remembering , if only so we don't repeat it.

+1 on this. It is painfully clear that we need to radically improve our practices relating to due diligence moving forward.

This is really interesting—thanks for sharing!

This has created a potentially dangerous mismatch in public perception between what the more serious AI safety researchers think they're doing (e.g. reducing X risk from AGI), and what the public thinks AI safety is doing (e.g. developing methods to automate partisan censorship, to embed woke values into AI systems, and to create new methods for mass-customized propaganda).

This is the crux of the problem, yes. I don’t think this is because of a “conservative vs liberal” political rift though; the left is just as frustrated by, say, censorship of sex e... (read more)

Yes, the consequences are probably less severe in this context, which is why I wouldn't consider this a particularly strong argument. Imo, it's more important to understand this line of thinking for the purpose of modeling outsider's reactions to potential censorship, as this seems to be how people irl are responding to OpenAI, et al's policy decisions.

I would also like to emphasize again that sometimes regulation is necessary, and I am not against it on principle, though I do believe it should be used with caution; this post is critiquing the details of how we are implementing censorship in large models, not so much its use in the first place.

There's nothing in this section about why censoring model outputs to be diverse/not use slurs/not target individuals or create violent speech is actually a bad idea.

The argument in that section was not actually  an object-level one, but rather an argument from history and folk deontological philosophy (in the sense that "censorship is bad" is a useful, if not perfect, heuristic used in most modern Western societies). Nonetheless, here's a few  reasons why what you mentioned could be a bad idea: Goodhart's law, the Scunthorpe Problem, and the gene... (read more)

Came across this post today—I assume the bounty has been long-closed by now?

Thanks, I think I somehow missed some of those!

Thanks for the clarification! I might try to do something on the Orthogonality thesis if I get the chance, since I think that tends to be glossed over in a lot of popular introductions.

Question—is $20,000 awarded to every entry which qualifies under the rules, or is there one winner selected among the pool of all who submit an entry?

This is really exciting! I’m glad there are so many talented people on the case, and hope the good news will only grow from here :)

I strongly agree with you on points one and two, though I’m not super confident on three. For me the biggest takeaway is we should be putting more effort into attempts to instill “false” beliefs which are safety-promoting and self-stable.

Thanks for making this document public, it’s an interesting model! I am slightly concerned this could lead to reduced effectiveness within the organization due to reduced communication, which could plausibly cause more net harm in EV than the increased risk of infohazard leakage. I assume you’ve done that cost/benefit analysis already of course, but thought it’s worth mentioning just in case.

We are in the process of reaching out to individuals and we will include them after they confirm. If you have suggestions for individuals to include please add a com

There is a very severe potential downside if many funders think in this manner, which is that it will discourage people from writing about potentially important ideas. I’m strongly in favor of putting more effort and funding into PR (disclaimer that I’ve worked in media relations in the past), but if we refuse to fund people with diverse, potentially provocative takes, that’s not a worthwhile trade-off, imo. I want EA to be capable of supporting an intellectual environment where we can ask about and discuss hard questions publicly without worrying about being excluded as a result. If that means bad-faith journalists have slightly more material to work with, than so be it.

Not a bad idea! I’d love to try to actually test this hypothesis—my hunch is that it will do worse at prediction in most areas, but there may be some scenarios where thinking things through from a narrative perspective could provide otherwise hard-to-reach insight

I was personally unaware of the situation until reading this comment thread, so can confirm

My brother was recently very freaked out when I asked him to pose a set of questions that he thinks an AI wouldn’t be able to answer, and GPT-3 gave excellent-sounding responses to his prompts.

Seconding this—I’m definitely personally curious what such a chart would look like!

I don’t think that would imply that nothing really matters, since reducing suffering and maximizing happiness (as well as good ol’ “care about other human beings while they live”) could still be valid sources of meaning. In fact, insuring that we do not become extinct too early would be extremely important to insure the best possible fate of the universe (that being a quick and painless destruction or whatever), so just doing what feels best at the moment probably would not be a great strategy for a True Believer in this hypothetical.

I’m really exited about this, and look forward to participating! Some questions—how will you determine which submissions count as “ Winners” vs “runners up” vs “honorable mentions”? I’m confused what the criteria for differentiating categories are. Also, are there any limits as to how many submissions can make each category?

I didn't focus on it in this post, but I genuinely think that the most helpful thing to do involves showing proficiency in achieving near-term goals, as that both allows us to troubleshoot potential practical issues, and allows outsiders to evaluate our track record. Part of showing integrity is showing transparency (assuming that we want outside support), and working on neartermist causes allows us to more easily do that.

Fair enough; I didn’t mean to imply that $100M is exactly the amount that needs to be spent, though I would expect it to be near a lower bound he would have to spend (on projects with clear measurable results) if he wants to because known as “that effective altruism guy” rather than “that cryptocurrency guy”

Within the domain of politics (and to a lesser degree, global health), PR impact makes an extremely large difference in how effective you’re able to be at the end of the day. If you want, I’d be happy to provide data on that, but my guess is you’d agree with me there (please let me know if that isn’t the case). As such, if you care about results, you should care about PR as well. I suspect that your unease mostly lies in the second half of your response—we should do things for “direct, non-reputational reasons,” and actions done for reputational reasons wo... (read more)

Other than the donations towards helping Ukraine, I’m not sure there’s any significant charity on the linked page that will have really noticeable effects within a year or two. For what I’m talking about, there needs to be an obvious difference made quickly—it also doesn’t help that those are all pre-existing charities under other people’s names, which makes it hard to say for sure that it was SBF’s work that made the crucial difference even if one of them does significantly impact the world in the short term.

If it was just me (and maybe a few other similar-minded people) in the universe however, and if I was reasonably certain it would actually do what it said in the label, then I may very well press it. What about you, for the version I presented for your philosophy?

Excellent question! I wouldn’t, but only because of epistemic humility—I would probably end up consulting with as many philosophers as possible and see how close we can come to a consensus decision regarding what to practically do with the button.

Love this newsletter, thanks for making it :)

One possible “fun” implication of following this line of thought to its extreme conclusion would be that we should strive to stay alive and improve science to the point at which we are able to fully destroy the universe (maybe by purposefully paperclipping, or instigating vacuum decay?). Idk what to do with this thought, just think it’s interesting.

Thanks for the post—It was really amazing talking with you at the conference :)

We already know that we can create net positive lives for individuals

Do we know this? Thomas Ligotti would argue that even most well-off humans live in suffering, and it’s only through self-delusion that we think otherwise (not that I fully agree with him, but his case is surprisingly strong)

If you could push a button and all life in the universe would immediately, painlessly, and permanently halt, would you push it?

I think it’s okay to come off as a bit insulting in the name of better feedback, especially when you’re unlikely to be working with them long-term.

my best guess is that more time delving into specific grants will only rarely actually change the final funding decision in practice

Has anyone actually tested this? It might be worthwhile to record your initial impressions on a set number of grants, then deliberately spend x amount of time researching them further, and calculating the ratio of how often further research makes you change your mind.