All of Sindy's Comments + Replies

Answer by SindyJul 30, 202010
0
0

Hey, thank you for the work you are doing! Here are my thoughts (I'm an economist at IDinsight and work on this type of research):

  • If you want to understand the impact of your program, I don't recommend doing an RCT at this stage. This seems like a very small pilot and you won't have enough power / sample size to detect an effect (more see below). You should only consider running an RCT if and when you plan to scale this up later to a sufficient scale.
  • Instead what I advise is trying to understand and improve your impact by doing some small sa
... (read more)
1
brb243
4y
Hello Sindy, Thank you so much. This answers my question. Yes, there will be a before and after qualitative survey asking about own and others' behavior - which may need to be truncated to speak with more different groups. Then, the face covering data can be used to complement the survey information.

To follow up on Michael's last point here: Natalia, do you have any interest in collaborating with academics to feed results from your app to a new version of disability weighting? (He mentioned in his other comment that some academics were working on it but stopped.)

I also posted a comment on the other post outlining challenges you need to overcome to generate rigorous measures for disability weights (e.g. low take-up, unrepresentative sample).

Seems like the Gates Foundation which is funding the Global Burden of Disease study should be interested in ... (read more)

A minor correction: GiveWell uses DALY to measure mortality and morbidity. (Well, for malaria they actually don't look at the impact of prevention on morbidity, only mortality, since the former is relatively small -- see row 22 here.) Maybe what you had in mind is their "moral weights" which they use to convert between life years and income.


Like cole_haus points out below, ESM's results would enter disability weights (which are used to construct DALYs) to affect how health interventions are prioritized. Currently disability weights invo... (read more)

Answer by SindyFeb 01, 20203
0
0

Thanks Linch for the post!


A comment is that there are things that one probably doesn't encounter in the first 10-20 hours that can be hugely useful (at least for me) in thinking about EA (both general and domain specific), e.g. this. (Perhaps that means things like that should work their way into key intro materials...)


In general I wish there were a better compilation of EA materials from intro to advanced levels. For intro materials, perhaps this is good. Beyond that, there are good content from

  • 80,000 Hours career guides, problem profiles, and blog p
... (read more)

Thanks for the post John! Very informative. I know some people thinking of doing another RCT on this and will definitely point them to it.

Also agree that heterogeneities in the actual intervention as well as population under study are major challenges here in generalizing the effects (and they are common in studies on social science interventions which probably lead to lower generalizability than medical trials).


One minor and meta comment on section 2: "How over-optimistic should we expect the evidence to be?" I'm not sure how I feel about h... (read more)

Hi Parth, thank you so much for this post, and for the great work you and your fellow EA organizers are doing at Microsoft!

I live in SF, and have been brainstorming with a few EAs re mobilizing EAs in tech companies (in addition to general EA movement building in the city). Will definitely try to learn from your experience and reach out for more questions if that's ok.

I also wonder if you guys have a broader strategy for EA community building at Microsoft, and/or other EA meetups there (or directing people to EA Seattle)? Also, do you have a way to tr... (read more)

4
ParthThaya
4y
Hi Sindy, thanks for the kind words! Really cool to hear you’ve been looking into doing that, and I’d be interested in hearing more. And of course you’re more than welcome to reach out if you have any questions. I can’t speak for everyone involved, but off the top of my head, my rough strategy is something like: 1. Get more people to hear about EA. Last year, we only managed to get invites out to ~10% of the company, so there’s lots more to do here; 2. As there is more interest and awareness among employees, work with the company to incorporate EA principles/charities into the official Give campaign. Our main metrics today are simply site visits, people tuning into our talks, and feedback we receive. Donations to/through GiveWell from Microsoft is something we could maybe track if they are willing to share that information, but that’s not a conversation we’ve had yet. Bill Gates has stayed away from mixing his Foundation work with Microsoft, as far as I can tell. Our team’s never talked about reaching out to him for support, but maybe we should ...

Rob, thank you so much for the work you and AMF are doing!

GiveWell has written here saying they think your monitoring practice could be improved, though they "continue to believe that AMF stands out among bed net organizations, and among charities generally, for its transparency and the quality of its program monitoring."

I'd first like to applaud that you do have much better transparency and monitoring practices than the typical development NGO. It seems that one reason GiveWell selected AMF rather than other bed net charities as a top chari... (read more)

5
RobM
4y
We have certainly improved monitoring practices since 2016 and it’s important that we continue to look to improve them. The observations and criticisms made in 2014 were valid and it is one of the benefits of independent organisations reviewing our work in detail that we receive feedback and suggestions that can help us do a better job. An example of a recent improvement is the change in the frequency and scale of our post-distribution monitoring. For many years, PDMs were 6 monthly and involved visiting 5% of the households that received nets. As a result of an 18 month trial in Uganda, where we carried out PDMs in 124 health sub-districts split into five randomised groups (Arm 1: 6-monthly, 5% of households; Arm 2: 9-monthly, 5% of HHs; Arm 3; 6-monthly, 1.5% of HHs; Arm 4: 9-monthly, 1.5% of HHs; Arm 5: A PDM at 18 months as a control), we generated the data to support a move to 9-monthly PDMs visiting 1.5% of HHs. This has reduced cost without any loss in the benefit of carrying out the PDMs or value of the data generated. Another example that took place earlier, as a result of feedback from GiveWell, was for AMF itself to make the randomised selections of households to visit rather than leaving this to in-country partners. It is not clear if this changed the outcome and reliability of the PDMs, but the separation of who does the selecting and who does the visiting increased confidence in the results of the PDMs. The increased use of electronic device data collection is another way in which monitoring is being improved with benefits including: lower cost, improved accuracy, earlier detection of problems and faster access to results. Improving monitoring practices is a priority and we continually reflect on how we can do better.

(Context: I've been engaging in "RD" research since my econ PhD focusing on development, and in my past 2.5 years working at IDinsight. All views are my own.)

Thanks a lot for the post. I agree that a more hits-based approach to development within EA is needed. GiveWell says they eventually want to look at economic growth, but they're starting with health policy which is easier to evaluate and it's unclear how long it will take them to look at policies aiming at increasing growth, so it seems valuable for other EAs to look at it in ... (read more)

8[anonymous]4y
I agree that we should keep our focus on human welfare rather than on gdp per capita as such, and that proposed research agenda should consider a broad question such as "how can we ensure democratic, sustainable and equitably shared growth?" As we say, I do think this is best approached outside of RCTs.
[anonymous]4y13
0
0

Hello, thanks for these comments! On the antagonistic point, I personally don't think the post is antagonistic. I think calling something "the case against view x" is what you would expect of a post criticising a particular view. I also don't think there are any parts of the substantive post itself that involve any snark, sneering or things like that. Where we do put forward critical opinions, they seem to me to be stated neutrally and directly, without flourish, rather than in an antagonistic way.

This being said, it has been mentioned... (read more)

This is speculative, but I suspect many of the things you mentioned fall in the category of things that seem pretty impactful, potentially on par with EA's main cause areas (poverty, animals, x-risk), but it doesn't seem like it makes sense to devote that much EA manpower or resources to it right now -- so a small number of EAs who identify one such area can work on it, and it's great, (and the EA movement should encourage that, with sufficient justification of the impact), but I can see why the EA movement doesn't put them as a main ca... (read more)

3
kbog
4y
The answer is simply to grow the EA movement so that more causes have adequate numbers of people working on them. Rather worrying about giving people equal slices of the pie.