Over the course of 2009, discussion on the Felicifia forum (archive) started to feel like an early EA community. For example, see thread on charity choice and the applied ethics and philanthropy boards.

It's really interesting to see, in the thread on charity choice, EmbraceUnity describing their "Utility, Attainability, and Obscurity" framework (see also this blog post from 2008) four years before Holden Karnofsky wrote about the Importance, Tractability and Neglectedness framework. I guess this is a sign that, for some reason, many of the key pieces of EA just fell into place in different locations at around the same time.

Here's what I understand happened:

1. FLI considers the grant to SND.
2. FLI does some initial vetting and fails to discover important facts about Nya Dagbladet.
3. FLI informs SND of its intent to issue the grant (because it does intend to do so).
4. SND asks for a letter of intent in writing (possibly to include it in the filing to register as an official foundation), which FLI agrees to.
5. FLI does fresh due diligence, uncovering issues.
6. FLI informs SND they won't issue the grant after all.

This doesn't seem like lying to me -- the only issue I see is that the vetting at stage (2) wasn't good enough, and this seems like something FLI will work on improving. FLI's telling SND it intended to issue a grant would only seem like lying, I think, if FLI actually suspected it would discover issues with SND later on, or otherwise suspected they wouldn't actually end up issuing the grant. But presumably the vast majority of grants pass due diligence just fine, and FLI wouldn't have communicated an intent to issue the grant in the first place if it thought it wouldn't actually go through with it.

Is that really the way you see an "intent to transfer the grant amount promptly"? Sanjay, tell me with a straight face that there is no attempt to mislead here.

This seems needlessly combative to me. One of the norms of this forum is to be kind, and I think this sounds unkind.

Note that the English page was created in January of this year. The stuff on the Swedish page about Nordiska motståndsrörelsen and vaccination scepticisim and pseudoscience was added on September 14, after FLI signed the letter of intent.

I'm not an authority here, but from scanning the front page yesterday and today I see quite a lot of anti-vax/covid-19 conspiracy sentiment, some pro-Russian/anti-Ukraine sentiment, some anti-immigration/anti-globalism sentiment, and I didn't see anything suggestive of Holocaust denial, neo-Nazism or replacement theory but that doesn't mean it doesn't exist. (There was one article critical of the Israeli government but I don't think that counts as anti-Semitic.) There's also a lot of culture war and freedom of speech stuff. There was a 9/11 truther article on the front page though it's 6 years old. (I didn't read any opinion pieces.)

As a counterpoint, there's one mostly sane article about the invasion of the Brazilian Congress (except for referring to the Capitol Hill attack as happening under "mysterious circumstances", which sounds pretty conspiratorial). There are also a bunch of articles that seem basically harmless, like this one about 165K chicken being killed due to risk of salmonella.

Thanks!

I had a look at the Nya Dagbladet website. My quick impression is that it looks like a somewhat milder and less sensationalist version of Breitbart News. The top stories were (and I only read the headlines and leads): (1) the newspaper itself being close to bankruptcy due to its bank account having been frozen, (2) an ex-CEO of Barclays' being connected to Jeffrey Epstein, (3) high levels of shoplifting in Sweden, (4) record number of calls to a national child abuse hotline in Sweden, (5) a Swedish pediatrician's having been subjected to hate due to a study of his which suggested that the risk of children needing emergency care for COVID-19 was low, (6) more on the frozen bank account, (7) Kiersten Hening getting a $100K settlement after BLM kneeling controversy and (8) EU and NATO collaborating more closely.

I can't imagine a good rationale for giving a grant to Nya Dagbladet or associated ventures, and can only assume that FLI agreed to give the grant based either on material provided by Nya Dagbladet itself and/or other people (but without doing any independent review).

Tegmark's brother, Per, seems to be affiliated with the Populist, anti-vax right in Sweden (note,this is only after a very cursory Google search). The reasons this seems to be relevant is that Per has been a contributing writer for Nya Dagbladet in recent years.

Do you have a source for this? I wasn't able to find anything myself with a quick search.

I wrote something about CICERO, Meta's new Diplomacy-playing AI. The summary:

• CICERO is a new AI developed by Meta AI that achieves good performance at the board game Diplomacy. Diplomacy involves tactical and strategic reasoning as well as natural language communication: players must negotiate, cooperate and occasionally deceive in order to win.
  • CICERO comprises (1) a strategic model deciding which moves to make on the board and (2) a dialogue model communicating with the other players.
  • CICERO is honest in the sense that the dialogue model, when it communicates, always tries to communicate the strategy model's actual intent; however, it can omit information and change its mind in the middle of a conversation, meaning it can behave deceptively or treacherously.
• Some who are concerned with risks from advanced AI think the CICERO research project is unusually bad or risky.
  • It has at least three potentially-concerning aspects:
    1. It may present an advancement in AIs' strategic and/or tactical capabilities.
    2. It may present an advancement in AIs' deception and/or persuasion capabilities.
    3. It may be illustrative of cultural issues in AI labs like Meta's.
  • My low-confidence take is that (1) and (2) are false because CICERO doesn't seem to contain any new insights that markedly advance either of these areas of study. Those capabilities are mostly the product of using reinforcement learning to master a game where tactics, strategy, deception and persuasion are useful, and I think there's nothing surprising or technologically novel about this.
  • I think, with low confidence, that (3) may be true, but perhaps no more true than of any other AI project of that scale.
• Neural networks using reinforcement learning are always (?) trained in simulated worlds. Chess presents a very simple world; Diplomacy, with its negotiation phase, is a substantially more complex world. Scaling up AIs to transformative and/or general heights using the reinforcement learning paradigm may require more complex and/or detailed simulations.
  • Simulation could be a bottleneck in creating AGI because (1) an accurate enough simulation may already give you the answers you want, (2) an accurate and/or complex enough simulation may be AI-complete and/or (3) extremely costly.
  • Simulation could also not be a bottleneck because, following Ajeya Cotra's bio-anchors report, (1) we may get a lot of mileage out of simpler simulated worlds, (2) environments can contain or present problems that are easy to generate and simulate but hard to solve, (3) we may be able to automate simulation and/or (4) people will likely be willing to spend a lot of money on simulation in the future, if that leads to AGI.
  • CICERO does not seem like an example of a more complex or detailed simulation, since instances of CICERO didn't actually communicate with one another during self-play. (Generating messages was apparently too computationally expensive.)

The post is written in a personal capacity and doesn't necessarily reflect the views of my employer (Rethink Priorities).

I think that, yes, you did misunderstand rationality, though it's hard to tell for sure since you never define it. I'll use two commonly used definitions:

• Rationality means having accurate beliefs.
• Rationality means making good decisions, i.e. decisions that help one achieve one's goals.

You write:

Our reliance on the scientific method and empirical evidence as the sole means of acquiring knowledge is problematic in its narrow, positivist approach, which ignores the invaluable insights that can be gleaned from our innate intuitive faculties. [...] I always thought I had to act rational and pay no attention (or less attention) to my own intuition as rationality suggests.

But rationality doesn't tell us that we should ignore intuition. Intuition is evidence, sometimes useful, sometimes not. Intuition can help us be rational if and to the extent that it helps us have accurate beliefs and/or make decisions that help us achieve our goals.

Effective altruists and rationalists do emphasise using evidence and reason, but that is not because those are inherently good, it's because they're instrumentally good -- because (we think) they help you have more true beliefs and make better decisions.

Thanks, that sounds quite bad.

commons plural noun [treated as singular] land or resources belonging to or affecting the whole of a community

The reputation of effective altruism is a commons. Each effective altruist can benefit from and be harmed by it (it can support or impede one's efforts to help others), and each effective altruist is capable of improving and damaging it.

I don't know whether actions that may cause substantial harm to a commons should be decided upon collectively. I don't know whether a community can come up with rules and guidelines governing them. But I do think, at minimum, in the absence of rules and guidelines, that one should inform the community when planning a possibly-commons-harming action, so that the community can at least critique one's plan.

I think purchasing Wytham Abbey (which may have made sense, even factoring in the reputational effects -- I'm not sure) was a possibly-commons-harming action, and this sort of action should probably be announced before it’s carried out in future.