Principles are great!  I call them "stone-tips".  My latest one is:

Look out for wumps and woozles!

It's one of my favorite. ^^  It basically very-sorta translates to bikeshedding (idionym: "margin-fuzzing"), procrastination paradox (idionym: "marginal-choice trap" + attention selection history + LDT), and information cascades / short-circuits / double-counting of evidence…  but a lot gets lost in translation.  Especially the cuteness.

The stone-tip closest to my heart, however, is:

I wanna help others, but like a lot for real!

I think EA is basically sorta that… but a lot gets confusing in implementation.

My latest tragic belief is that in order to improve my ability to think (so as to help others more competently) I ought to gradually isolate myself from all sources of misaligned social motivation.  And that nearly all my social motivation is misaligned relative to the motivations I can (learn to) generate within myself.  So I aim to extinguish all communication before the year ends (with exception for Maria).

I'm posting this comment in order to redirect some of this social motivation into the project of isolation itself.  Well, that, plus I notice that part of my motivation comes from wanting to realify an interesting narrative about myself; and partly in order to publicify an excuse for why I've ceased (and aim to cease more) writing/communicating.

I appreciate the point, but I also think EA is unique among morality-movements. We hold to principles like rationality, cause prioritization, and asymmetric weapons, so I think it'd be prudent to exclude people if their behaviour overly threatens to loosen our commitment to them.

And I say this as a insect-mindfwl vegan who consistently and harshly denounces animal experimentation whenever I write about neuroscience^[1] (frequently) in otherwise neutral contexts. Especially in neutral contexts.

1. ^{^}

   Evil experiments are ubiquitous in neuroscience btw.

Thank you for appreciating! 🕊️

Alas, I'm unlikely to prioritize writing except when I lose control of my motivations and I can't help it.^[1]  But there's nothing stopping someone else extracting what they learn from my other comments¹ ² ³ re deference and making post(s) from it, no attribution required.

(Arguably it's often more educational to learn something from somebody who's freshly gone through the process of learning it.  Knowledge-of-transition can supplement knowledge-of-target-state.)

Haphazardly selected additional points on deference:

1. Succinctly, the difference between Equal-Weight deference and Bayes
   1. "They say $O\left(H\right)=1:2$.  |  Then I can infer that they updated from $1:6$ to $1:2$ by multiplying with a likelihood ratio of $3:1$. And because C and D, I can update on that likelihood ratio in order to end up with a posterior of $O\left(H\right)=6:1$.  |  The equal weight view would have me adjust down, whereas Bayes tells me to adjust up."
2. Paradox of Expert Opinion
   1. "Ask the experts. They're likely the most informed on the issue. Unfortunately, they're also among the groups most heavily selected for belief in the hypothesis."

1. ^{^}

   It's sort of paradoxical. As a result of my investigations into social epistemology 2 years ago, I came away with the conclusion that I ought to focus ~all my learning-efforts on trying to (recursively) improve my own cognition, with ~no consideration for my ability to teach anyone anything of what I learn.  My motivation to share my ideas is an impurity that I've been trying hard to extinguish.  Writing is not useless, but progress toward my goal is much faster when I automatically think in the language I construct purely to communicate with myself.

To clarify, I'm mainly just sceptical that water-scarcity is a significant consideration wrt the trajectory of transformative AI. I'm not here arguing against water-scarcity (or data poisoning) as an important cause to focus altruistic efforts on.

Hunches/reasons that I'm sceptical of water as a consideration for transformative AI:

• I doubt water will be a bottleneck to scaling
  • My doubt here mainly just stems from a poorly-argued & uncertain intuition about other factors being more relevant. If I were to look into this more, I would try to find some basic numbers about:
    • How much water goes into the maintenance of data centers relative to other things fungible water-sources are used for?
    • What proportion of a data center's total expenditures are used to purchase water?
    • I'm not sure how these things work, so don't take my own scepticism as grounds to distrust your own (perhaps-better-informed) model of these things.
• Assuming scaling is bottlenecked by water, I think great-power conflict are unlikely to be caused by it
• Assuming conflicts happen due to water-bottleneck, I don't think this will significantly influence the long-term outcome of transformative AI

Note: I'll read if you respond, but I'm unlikely to respond in turn, since I'm trying to prioritize other things atm. Either way, thanks for an idea I hadn't considered before! : )

I think this is 100% wrong, but 100% the correct^[1] way to reason about it!

I'm pretty sure water scarcity is a distraction wrt modelling AI futures; but it's best to just assert a model to begin with, and take it seriously as a generator of your own plans/actions, just so you have something to iterate on. If you don't have an evidentially-sensitive thing inside your head that actually generates your behaviours relevant to X, then you can't learn to generate better behaviours wrt to X.

Similarly: To do binary search, you must start by planting your flag at the exact middle of the possibility-range. You don't have a sensor wrt to the evidence unless you plant your flag down.

1. ^{^}

   One plausible process-level critique is that… perhaps this was not actually your best effort, even within the constraints of producing a quick comment? It's important to be willing to risk thinking&saying dumb things, but it's also important that the mistakes are honest consequences of your best effort.

   A failure-mode I've commonly inhabited in the past is to semi-consciously handicap myself with visible excuses-to-fail, so that if I fail or end up thinking/saying/doing something dumb, I always have the backup-plan of relying on the excuse / crutch. Eg,

   • While playing chess, I would be extremely eager to sacrifice material in order to create open tactical games; and when I lost, I reminded myself that "ah well, I only lost because I deliberately have an unusual playstyle; not because I'm bad or anything."

I first learned this lesson in my youth when, after climbing to the top of a leaderboard in a puzzle game I'd invested >2k hours into, I was surpassed so hard by my nemesis that I had to reflect on what I was doing.  Thing is, they didn't just surpass me and everybody else, but instead continued to break their own records several times over.

Slightly embarrassed by having congratulated myself for my merely-best performance, I had to ask "how does one become like that?"

My problem was that I'd always just been trying to get better than the people around me, whereas their target was the inanimate structure of the problem itself.  When I had broken a record, I said "finally!" and considered myself complete.  But when they did the same, they said "cool!", and then kept going.  The only way to defeat them, would be by not trying to defeat them, and instead focus on fighting the perceived limits of the game itself.

To some extent, I am what I am today, because I at one point aspired to be better than Aisi.

More wisdom from Eliezer (from a quote I found via Nevin's comment):

"But how does Nemamel grow up to be Nemamel?  She was better than all her living competitors, there was nobody she could imitate to become that good.  There are no gods in dath ilan.  Then who does Nemamel look up to, to become herself?"

… If she'd ever stopped to congratulate herself on being better than everyone else, wouldn't she then have stopped?  Or that's what I remember her being quoted as saying.  Which frankly doesn't make that much sense to me?  To me it seems you could reach the Better Than Everybody key milestone, celebrate that, and then keep going?  But I am not Nemamel and maybe there's something in there that I haven't understood yet."

… It was the pride of the very smart people who are smarter than the other people, that they look around themselves, and even if they aren't the best in the world yet, there's still nobody in it who seems worthy to be their competitor, even the people who are still better than them, aren't enough better.  So they set their eyes somewhere on the far horizon where no people are, and walk towards it knowing they'll never reach it."

… And that's why when people congratulated her on being better than everybody, she was all, 'stop that, you only like me that much because you're thinking about it all wrong', compared to some greater vision of Civilization that was only in her own imagination."

It took me 24 minutes from when I decided to start (half-way through reading this post), but that could be reduced if I had these tips at the start.

1. The website requires you to enable cookies. If you (like me) select "no" before you realize this, you can enable them again like this.^[1] (Safe to ignore otherwise.)
2. Yes, you can simply copy-paste Ben's suggestions above into sections Policy proposal, Animal Welfare, and Further comments.
3. The only question I wrote anything in is Q29 in the About you section, selecting "Other (please specify)":

1. ^{^}

   On Chrome:
    1) to the right of the URL bar, select Cookies and site data → Manage on-device site data
    2) delete both cookies there
    3) refresh page, and you will now be prompted again

   

And a follow-up on why I encourage the use of jargon.

• Mutation-rate ↦ "jargon-rate"
• I tend to deliberately use jargon-dense language because I think that's usually a good thing. Something we discuss in the chat.
  • I also just personally seem to learn much faster by reading jargon-dense stuff.
  • As long as the jargon is apt, it highlights the importance of a concept ("oh, it's so generally-applicable that it's got a name of its own?").
  • If it's a new idea expressed in normal words, the meaning may (ill-advisably) snap into some old framework I have, and I fail to notice that there's something new to grok about it. Otoh, if it's a new word, I'll definitely notice when I don't know it.
  • I prefer a jargon-dump which forces me to look things up, compared to fluent text where I can't quickly scan for things I don't already know.
  • I don't feel the need to understand everything in a text in order to benefit from it. If I'm reading something with a 100% hit-rate wrt what I manage to understand, that's not gonna translate to a very high learning-rate.

To clarify: By "jargon" I didn't mean to imply anything negative. I just mean "new words for concepts". They're often the most significant mutations in the meme pool, and are necessary to make progress. If anything, the EA community should consider upping the rate at which they invent jargon, to facilitate specialization of concepts and put existing terms (competing over the same niches) under more selection-pressure.

I suspect the problems people have with jargon is mostly that they are *unable* to change them even if they're anti-helpfwl. So they get the sense that "darn, these jargonisms are bad, but they're stuck in social equilibrium, so I can't change them—it would be better if hadn't created them in the first place." The conclusion is premature, however, since you can improve things either by disincentivizing the creation of bad jargon, *or* increasing people's willingness to create them, so that bad terms get replaced at a higher rate.

That said, if people still insist on trying to learn all the jargon created everywhere because they'll feel embarrassed being caught unaware, increasing the jargon-rate could cause problems (including spending too much time on the forum!). But, again, this is a problem largely caused by impostor syndrome, and pluralistic ignorance/overestimation about how much their peers know. The appropriate solution isn't to reduce memetic mutation-rate, but rather to make people feel safer revealing their ignorance (and thereby also increasing the rate of learning-opportunities).

Naive solutions like "let's reduce jargon" are based on partial-equilibrium analysis. It can be compared to a "second-best theory" which is only good on the margin because the system is stuck in a local optimum and people aren't searching for solutions which require U-shaped jumps^[1] (slack) or changing multiple variables at once. And as always when you optimize complex social problems (or manually nudge conditions on partial differential equations): "solve for the general equilibrium".

1. ^{^}

   A "U-shaped jump" is required for everything with activation costs/switching costs.