The new EA Forum frontpage layout is unserious. My concern is simple: where is the list of the latest posts? The screen is so overwhelmed by "featured posts" and "comments" and "quick takes" that the centerpiece of the whole thing is completely gone. Now I can't see what is going on in EA, or whether there is something that I want to read, which defeats the point of visiting the website. And moreover, hiding such key information gives the appearance of a forum that doesn't care about supporting serious discussion, it sets the wrong tone for the community.

You can see a comparison to LessWrong and Hacker News below.

Current EA forum

Current Lesswrong

Current Hacker News

Hey this is super cool!

I think this could be a great intervention, but I disagree on the RCT front. I strongly believe that in the medium to long-term, this intervention should, and will only go to scale after high quality RCTs.

Yours and RP's reasons for not doing it aren't compelling to me. I'm almost a bit confused.. This might actually be one of the easier RCTs to do in the development world. If I was a funder, I would be asking why you aren't in the middle of a big RCT at the moment, or at least why you aren't getting started on one. I agree with the funders.

"Funders often tell us to come back with an RCT, which we estimate would need around 72 hospitals across three countries, US$5 million and three to four years." If this is what's needed, then yeah do it. You can get the 5 million - have you asked GiveWell?  But I'm skeptical you really need 72 hospitals. Mortality rates are high enough in massive hospitals I would imagine if you are in national referral hospitals in capital cities, or similar, then this number would drastically drop. Maybe you are considering a bunch of smaller hospitals? 

From your article
"conventional RCT is hard to run for this intervention. It can't be blinded, because health workers know when monitors are on the ward. It has to randomise whole hospitals, because the central patient overview and the positive impact on workload affects care for every child on the ward, not only those attached to a monitor. Death is a relatively rare outcome, and baseline neonatal mortality across our sites ranges from 4.3% to 27.8%, which makes a trial harder to power. Ministries are also reluctant to keep hospitals in a control arm for years while neighbouring hospitals have monitoring. So is an RCT needed before scaling? If the effect is real, waiting three to four years for trial results means children in hospitals that could have had monitoring go without it. Last but not least it is important to emphasize that continuous monitoring is already considered standard of care in most parts of the world, so perhaps it is less about if it works but rather to what extent it should be prioritized amidst many other priorities and gaps.

From RP
"Note that a traditional randomized controlled trial may be genuinely hard to run in this context: a monitoring system cannot be disguised as a placebo, and it may change how the whole ward operates. For example, staff know when the monitors are present, and the intervention is bundled with training and supervision. The observed effect size therefore reflects both the device plus the attention surrounding it (i.e., the Hawthorne effect^[5]), 


1. Your current evidence is fairly weak, and low on the evidence ladder. Before and after studies are notorious for being unreliable because there are so many other factors involved.  Hospitals are always improving their systems and there are many reasons why mortality could drop. Seeing consistency between studies is encouraging but evidence needs to be better.

2. Blinding is not very important - the staff seeing the monitors is part of the intervention no? I'm not sure why RP and you think this is a problem? If it changes how the whole ward operates that is part of the benefit of the intervention.  In fact that might be where you main benefit is! I'm confused about this comment by RP here and not sure what point they are making.... The training and supervision front could be a big deal though, especially when you believe your main impact to be ward spillovers. That's why I think training and supervision can't really last more than a few months, then you need to largely leave the hospital be with the monitors with minimal supervision and see if mortality reductions are sustained over 2-3 years. This isn't that long an RCT!

4. Randomising whole hospitals isn't an ethical issue. Ministries will not refuse to allow hospitals being in a control arm (they are getting free monitors) and the evidence isn't nearly good enough yet to argue an RCT is unethical. If the difference in mortality between hospitals is big after year 2 say, the study will be triggered to stop for ethical reasons anyway.

5. The range of mortality rates doesn't make powering a study harder I don't think? (might be missing something) Its low mortality rates and low sample sizes that might be the issue here. But there are lots of hospitals where multiple kids die every week (I have one 500 meters from my home) If you are in the busiest hospitals where 150+ kids die every year, then I would think you wouldn't need neadly that amount (I might Claude this later). If you need to do a multi-country RCT so be it, that seems likely needed.

6. The urgency argument seems unrealistic. Because of the low level of evidence intervention isn't "obviously" better than other things we could spend money on. I don't think your case is nearly strong enough here to make a "bypass the RCT" argument. I think actually even framing it like this could make people lose confidence in the intervention. It feels more like a sales pitch than a good argument.

7. What is considered "Standard of care" in rich countries is close irrelevant in healthcare interventions in low-income countries. I don't think it should even be a factor really. There are 100 "standard of care" things you could introduce to a pediatric ward in Malawi (Better antibiotics, Dr. nos, nursing ratios, frequency checks, infection control, blood cultures, blood gases etc.). This doesn't mean you don't study their benefit in a low-income setting.

As an unrelated point, I feel like the presentation of the intervention here might be too branded. The intervention is a monitoring device, yet you are at pain to emphasise how your intervention is different from other devices. If your plan is to scale through government, its the monitoring system that matters not the branding. It might be better even to brand machines as "Ministry of Health" machines and think of yourself as a support org as much as an innovation org? This is a tricky one though I know.

I absoutely love this intervention - I'd be keen to get on a call and chat about it even if you might be. Sadly someone ffrom your org was pegged to meet me at EAG New York, but I can't go because of travel restrictions due to ebola :(

Hmm...

It’s interesting: on the one hand, your hypothetical could be the way it works out, and it doesn't sound that bad. But on the other hand, as I say in the post, I think a sensible way to understand the badness of failing to make the hire is that it’s the difference between a) a 55-person org which is growing and b) a 56-person org which is growing. In your framing, it sounds like the org is only trivially worse off. In my framing, it’s clear that b) is much better off!

I thought about it for a little while. Here are three reasons why I think the costs of failing to make the hire are much higher than your comment makes them sound.

1. Repeating the hiring round fairly soon and making an equivalently strong hire is unusually fortunate

Here’s what might happen instead:

The manager learns they can’t hire for the role they wanted, and scopes out a smaller one. They hand off the procedural parts of the function and keep the rest. What they wanted was for the function to grow into a strong one, with the attention of someone who can oversee it well and make sure it moves to meet the needs of an aggressively growing organisation. It doesn’t get that. And because the smaller role is filled, the pressure to fix it goes away, so the situation tends to continue indefinitely: the function is weaker than it should be, and the manager keeps spending time on it.

Someone else accepts the role. The candidate is, in expectation, weaker, and in these rounds the gap between the top candidate and the next is often surprisingly large (Rob Wiblin worked through this).

The manager just keeps it. No re-run, no re-scope. The function sits at manager-level attention and the manager’s cycle stays blocked. This happens a lot. I’ve had functions that I ended up running with a small fraction of my time, going badly, and that situation continued indefinitely, sometimes in and out of failed hiring rounds.

2. Blocking other functions being scaled

Here’s a cycle which managers in high-growth orgs often spend their attention on: (1) scope out a function (understand what to do and how to do it, see if you can systematise it without hiring, learn what someone overseeing it would look like and what kind of person could do it); (2) hire someone to oversee the function; (3) onboard and train them; (4) use the freed-up time to repeat 1–3 for the next function.

The manager in your hypothetical has failed to get past (2) for this function, so they’ll move on to this cycle for other functions. That's fine, but failing to make the hire means that they'll be slower, in two ways. First, they’ll be spending (say) 10 hours a week overseeing the function for as long as it has no owner. Second, at some point they’ll do a hiring push to find the person, and that push will take something like 70–200 hours. All of that is time they could have spent spinning up the next function. In a fast-growing org, there's always some sense of haste in trying to get these functions going: because as soon as you get someone overseeing it, they're able to make it work well, then grow it at a similar compound growth rate to what the org is (so that it can meet the needs of the org). 

3. For whatever time there is no hire, the function is being run poorly by the manager and falling behind org needs

The function which the hiring manager failed to hire for was among the highest priority functions to get more attention to. That's why it was being hired for!

“The manager covers it for a little while” means it runs on (say) ten hours a week of the manager’s time until someone is hired into it. It will fall further behind what the fast-growing org needs for that whole time, and might become a real mess (and/or maybe take more of the hiring manager's attention). 

It being increasingly poorly run creates increasing problems for the org, and as it gets more messy it'll take whoever is eventually moved onto it longer to get it on the right track again. 

A note for the more AI-focused readers, which I've left out of the post because it's a different conversation. If your views on AI are such that you think ~everyone should be working on it, broad timelines be damned, then you won't find this post persuasive. But we'd still like to help you! There are plenty of ways for AIM to help with short-timelines work, that you may not be aware of:

Lots of people are trying to start incubators right now, in causes we work on and in ones we don't, like AI safety and catastrophic biorisks. Within EA, we have the most incubation experience(?), so many of them come to us for advice. We plan to help them a lot more through:

• Publishing learnings: Taking more time to distill and publicly share our institutional knowledge (see our shiny new substack).
• Advising: Bespoke support for new incubators throughout their journey, with our exec team carving out time for this from now on. For example, we're starting to work closely with Catalyze, an AI safety incubator staffed by AIM alumni and working out of our London office.
• Incubating incubators: The founder of a South American incubator is currently shadowing our program to see how it works from the inside. We're in talks with others about doing the same, and might pilot a version of the program built specifically for people starting incubators.
• Partnering: We're open to partnering with others, as long as it doesn't pull us away from our core work of launching charities.

We've heard it said: "If AIM took AI seriously, it would pivot to incubating AI safety organizations." I don't think that's right. Incubating AI safety organizations isn't our comparative advantage: we have close to zero institutional knowledge about AI safety (how to research it, how new organizations in the space should navigate it, and so on), and pivoting would mean starting again on everything we've learned about incubating the kinds of charities we already launch. But we can and should help the people who do have that advantage. Those causes need different things from ours, and we'll have more impact helping other incubators do it well than doing it ourselves, especially because incubation can very easily go badly.

If you're starting or running an incubator, in AI safety or anywhere else, and think we could be useful, we'd love to hear from you here.

Some very common failure modes, when working independently:

1. Not getting enough done
   1. Create some accountability systems for yourself
2. Not getting enough feedback
   1. Curate a group who you can ask for feedback, post in places where folks are likely to engage, join a project where you can learn from one-another etc.
3. Not teaching/mentoring/helping others out
   1. If you're competent enough at working independently, chances are you are experienced enough to (start to) give others feedback/guidance/mentorship, or can quickly debug things that would take less experienced people a much longer time
      1. This helps you become stronger in via a "tutor effect" kind of mechanism
4. Being isolated
   1. Find a physical (preferred) or online space where you can engage with others working on proximal ideas
5. Not doing multiple kinds of things
   1. Don't spend ~all your time in a single one of {reading, coding, discussing, writing, thinking, ...} ; notice when this is happening, and consciously make time to do the other things as well. Each week should contain some time in work-related input, output, and conversation/feedback.

Nice post :-)

I agree with a lot of this, and generally think people should act "as if":

1. There is at least one of them in some 'basement' universe.
2. They "control all the the copies" of themself

rather than reason/act as if "I might be in a simulation". In practice I think the two don't come apart massively, but I think there are risks in just adoping this latter approach (e.g. psychological). 

On the two claims I think people should follow as if they were "true": 

If we reject (1) then we can't rule out the case of our observations being non-veridical^[1].

This would make it hard for us to influence the 'base' reality (which for utilitarians might be where most of the potential value is). 

In a large enough multiverse, (1) seems likely. (Though if (1) is false and we're in a non-veridical simulation, it's unclear how much evidence we can have about whether we're in a large multiverse^[2]). 

Even if (1) did not hold, we might hope that some of observations are still useful in helping us influence the basement reality e.g. mathematical/logical/ philosophical arguments ^[3]. Of course simulators could be be arbitrarily Cartesian-demon like, but these seem safe to me to bracket-out. 

I think (2) is not too tricky to accept and widelt accepted in these parts (though I think many academic decision theories would reject). Of the range of views the axes described in this post, one can be an evidentialist/EDT-like or use the algorithmic ontology. 

1. ^{^}

   Regarding veridicity, I think there's some risk of circularity here (and am not certain I've thread the needle): 

   With non-veridical simulations, I want to say something like ruling out simulations that some smart intelligence would say are of "unreasonable counterfactuals" (or are just complete fiction - i.e. they don't aim to capture some of the same dynamics as they believe they have in their own universe/multiverse)

   So for example, if the totality of what exists ("the universe") was determined by some arbitrary (to some highly intelligent observer who understood physics as much as possible ) initial conditions, them I would class the simulation that runs "the universe but with different intial conditions" as reasonable.

   With respect to an intelligent observer that believed in (and had good reason to believe) in a sufficiently large multiverse, then it's plausible that many counterfactuals are "reasonable". 

2. ^{^}

   Conditional on being in a non-veridical simulation, beyond our prior we have an anthropic update (if we are non-updateless) but little else to inform us about the world. (Perhaps our prior has expectations over which simulations that basement beings run)

3. ^{^}

   Without (1), I don't think we can necessarily rule out simulators being able to run "counterlogical simulations" (this may only seem impossible to us because of the world they are happening to simulate).