All of NicholasKross's Comments + Replies

A Quick List of Some Problems in AI Alignment As A Field

Point 1: I said "Different from MIRI but correlated with each other". You're right that I should've done a better job of explaining that. Basically, "Yudkowksy approaches (MIRI) vs Christiano approaches (my incomplete read of most of the non-MIRI orgs). I concede 60% of this point.

Point 2: !!! Big if true, thank you! I read most of johnswentworths' guide to being an independent researcher, and the discussion of grants was promising. I'm getting a visceral sense of this from seeing (and entering) more contests, bounties, prizes, etc. for alignment work. I'm... (read more)

A Quick List of Some Problems in AI Alignment As A Field

Good point about the secrecy, I hadn't heard of the ABC thing. The secrecy is "understandable" to the extent that AI safety is analogous to the Manhattan Project, but less useful to the extent that AIS is analogous to... well, the development of theoretical physics.

The Many Faces of Effective Altruism

Not sure how relevant, but this reminds me of stories from inside Valve, the noted semi-anarchistly-organized game developer. People can move to any project they want, and there are few/no formal position titles. However, some employees have basically said that, because decision-making is sorta by consensus and some people have seniority and people can organize informally anyway, the result is a "shadow clique/cabal" that has disproportionate power. Which, come to think of it, would probably happen in the average anarchist commune of sufficient size.

TLDR just because the cliques don't exist formally, doesn't mean they don't exist.

The Many Faces of Effective Altruism

Oh yeah, there's clustering networks showing mutual followers of e.g. Twitch streamers, it shouldn't be too hard to make this for the EA sphere on twitter.

Flimsy Pet Theories, Enormous Initiatives

Somebody ought to start an independent organization specifically dedicated to red-teaming other people and groups' ideas.

I could start this after I graduate in the Fall, or potentially during the summer.

DM me if you want to discuss organization / funding.

Flimsy Pet Theories, Enormous Initiatives

The Gates documentary was part of what pushed me towards "okay, earning-to-give is unlikely to be my best path, because there seems to be a shortage in people smart enough to run massive (or even midsized) projects well." I guess the lack of red-teaming is a subset of constrainedness (although is it more cognitive bias on the funders, vs lack of "people / orgs who can independently red-team ideas"? Prolly both).

Flimsy Pet Theories, Enormous Initiatives

FWIW, Elon Musk famously kiiiiiiinda had a theory-of-change/impact before starting SpaceX. In the biography (and the WaitButWhy posts about him), it notes how he thought about funding a smaller mission of sending mice to Mars, and used a material cost spreadsheet to estimate the adequacy of existing space travel technology. He also aggressively reached out to experts in the field to look for the "catch", or whether he was missing something.

This is still nowhere near good red-teaming/proving-his-hunch-wrong, though. He also didn't seem to do nearly as much talking-to-experts knowledge-base-building for his other projects (e.g. Neuralink).

And most groups don't even do that.

Why 80 000 hours should recommend more people become drug lords

Find your old student's house, catch them escaping out a window during a drug bust, recruit them into your RV.

Brief Thoughts on "Justice Creep" and Effective Altruism

Related thought: people having different definitions of "justice", where that word points to overlapping-but-not-identical clusters of moral intuitions.

Animal welfare maps best on a cluster like "concern for the least-well-off" or "power for the powerless" or "the Rawls thing where if you imagined it happening to you, you'd hate it and want to escape it" or "ending suffering caused by the whims of other agents." That last one is particularly noticeable, since we usually have a moral intuition that suffering caused by other agents is basically preventable-thus-more-tragic.

The Bioethicists are (Mostly) Alright

We'll also mirror this on our collaborative blog TMB soon.

The Bioethicists are (Mostly) Alright

Agreed (I shoulda done that when editing it :P

You can still edit the post to include them! I agree with Khorton that you'll probably get more engagement that way.

How should Effective Altruists think about Leftist Ethics?

Thank you for putting this (and solutions) in clear words

December 2021 monthly meme post

Imho some kind of /r/EffectiveMemes would be the best bet

I could maybe help moderate an EA meme subreddit
Perhaps, with effective moderation. If it doesn't work here, that's a good place to go. But I think people would just think of it as another Dank EA memes - instead of something like a "tougher environment to increase memetic fitness"
A Red-Team Against the Impact of Small Donations

I am naturally an angsty person, and I don't carry much reputational risk Relate! Although you're anonymous, I'm just ADD.

Point 1 is interesting to me:

  • longtermist/AI safety orgs could require a diverse ecosystem of groups working based on different approaches. This would mean the "current state of under-funded-ness" is in flux, uncertain, and leaning towards "some lesser-known group(s) need money".
  • lots of smaller donations could indicate/signal interest from lots of people, which could help evaluators or larger donors with something.

Another point: ... (read more)

December 2021 monthly meme post

I post one article by a friend about memes, look away for 5 seconds, and now this!

I’ll pay you a $1,000 bounty for coming up with a good bounty (x-risk related)

BOUNTY IDEA (also sent in the form): Exploring Human Value Codification.

Offered to a paper or study that demonstrates a mathematical (or otherwise engineering-ready) framework to measure human's real preference-ordering directly. Basically a neuroscience experiment or proposal thereof.

End goal: Using this framework / results from experiment(s) done based on it, you can generate novel stimuli that seem similar to each other, and reliably predict which ones human subjects will prefer more. (Gradients of pleasure, of course, no harm being done). And, of cours... (read more)

Effective Altruism, Before the Memes Started

Devin's reply:

“Thanks for the response, reading your posts was one of the biggest inspirations for me writing this, its overall demeanor reminded me of what I see as this older strain of EA public interface in a way I hadn’t thought of in a while. On the point of MacAskill responding, I think the information you’ve given is helpful, but I do think there would have been some value in public commentary even if Torres personally wasn’t going to change his mind because of it, for instance it would have addressed concerns the piece gave outsiders who read it, a... (read more)

I mostly agree, but the revision of the Longtermism white paper [] from the original "work in progress" version seems like exactly the type of response to some of the early claims you're requesting - see the discussion on fanaticism. And given how recent all of this is, further responses could still be forthcoming, as these types of conversations take time.
Effective Altruism, Before the Memes Started

Devin's reponse:

“Yeah, I was wondering when that might come up. I have a general resistance to making extraneous accounts, especially if they are anything like social media accounts. I find it stressful and think I would over-obsessively check/use them in a way that would wind up being harmful. Even just having this post up and the ability to respond through Nick has occupied my attention and anxiety a good deal the last few days, or I might do more cross-posts/enable comments on our blog. That said, I did consider it. EA forum seems like it would not be s... (read more)

An update in favor of trying to make tens of billions of dollars

I mostly agree with the AI risk worldview described in footnote 5, but this is certainly an interesting analysis! (Although not super-useful for someone in a non-MIT/non-Jane-Street/not-elite-skilled reference class, but I still wonder about the flexibility of that...)

1Mathieu Putz10mo
Thanks a lot for saying this! Yeah, I wonder about the flexibility as well. At least, "I have good reason to think I could've gone to MIT/ Jane Street..." should go a long way (if you're not delusional).
Effective Altruism, Before the Memes Started

Devin's response:

“The white supremacy part doesn’t have this effect for me. Yes there is a use of this word to refer to overt, horrible bigotry, but there is also a use of this word meaning something closer to ‘structures that empower, or maintain the power, of white people disproportionately in prominent decision-making positions’. It is reasonable to say that this latter definition may be a bad way of wording things, you could even argue a terrible way, but since this use has both academic, and more recently some mainstream, usage, it hardly seems fair t... (read more)

Effective Altruism, Before the Memes Started

Devin's response: “I would be careful about calling this a bad faith attack. It may seem low quality or biased, but low quality is very different from bad faith and bias is probably something most of our defenders are guilty of to a decent degree as well. I’m not an expert on this case, but my own understanding is basically that Torres wrote a more academic, EA-targeted version of this before, got no responses or engagement he found adequate, despite reaching out to try to get it, and decided to take his case to a broader audience. I think there’s a ton wr... (read more)

I’m not an expert on this case, but my own understanding is basically that Torres wrote a more academic, EA-targeted version of this before, got no responses or engagement he found adequate

He got a very lengthy response here - far more detailed than most people would get.

I see every indication he was trying to criticize in good faith.

In contemporary western society, 'white supremacist' is one of the most harmful accusations you can make about someone, and should not be done without serious evidence, yet Phil flings the slur around with abandon.  Indee... (read more)

Effective Altruism, Before the Memes Started

Devin's response (also to DavidNash): “Sorry, there might be a misunderstanding here. The William MacAskill example is supposed to be more a framing device and specific case I’ve been thinking about, not any sort of proof that there’s a problem. As I mention in my epistemic status section, the overall claims I make about EA aren’t defended here, I rely on readers to just share this same impression of current fatigue with critics relative to early EA on reflection. If you don’t, that’s fine, but this piece isn’t going to try to convince you otherwise. On Ma... (read more)

See my response about the specific reason I think Will and others have not responded - and why I think they are right not to do so directly. (And I'm still very much on speaking terms with Phil, and understand why he feels aggrieved, even though I don't agree with him either about his current approach, or the substantive criticisms, as I noted in the piece you linked.)
Effective Altruism, Before the Memes Started

The LOTR analogy was intriguing to me, thank you!

Is it crunch time yet? If so, who can help?

People vastly overestimate the stability of their motivation and mental life. "...even when you take into account Hofstadter's Law." Seems very likely in my case.

The rest was helpfully calibrating, thank you

Effective Altruism, Before the Memes Started

Devin's reply/summary:

“Thanks for the comments. Sorry, I wrote a good deal of this stream of conscious so it isn’t really structured as an argument. More a way of me to connect some personal thoughts/experiences together in a hopefully productive way. I can see how that wouldn’t be super accessible. The basic argument embedded in it though is:

  1. Effective Altruism, like many idealistic movements, started out taking critics very very seriously and trying to reach out to/be charitable to them as much as possible, which is a good thing

  2. Effective Altruism, l

... (read more)
Re: 4, I think this has a reasonably high chance of being correct. I feel somewhat guilty here, as I have indeed made memes critical of some of the criticisms of EA/longtermism, and perhaps this is bad for my soul or the soul of the movement or something. (Though in my partial defense I am publicly critical [] of the midwit memes specifically)
Thank you, I believe that it would be helpful to have this intro on the top of the post.
Getting started independently in AI Safety

I feel both held back and out of my depth in this, so this and the comments have helped my perspective. Thank you for writing this!

Getting started independently in AI Safety

I feel like I'm on both sides of this, so I'll take the course and then immediately jump into whatever seems interesting in PyTorch