- What kind of research is this?
- We conducted a cause prioritisation research at a local level, specifically in Malaysia.
- Who is involved in this research?
- Yi-Yang Chua, Lynn Tan, Chan Tzu Kit, and Kaartik Nagarajan.
- Julian Theseira was also involved in the idea generation stage of the research.
- What did we hope to identify from this research?
- The top cause areas in Malaysia.
- Strategic insights on how we should build the EA community in Malaysia.
- Career pathways that are the most impactful in Malaysia (excluding remote jobs advertised in 80,000 Hours).
- Who we think this research will benefit
- Malaysians, and EA aligned Malaysians seeking to maximise impact.
- Potentially other EAs who are hoping to conduct similar research
- What are the top cause areas in Malaysia, from most impactful to least?
Note: that the following cause area rankings are not definitive and conclusive.
- EA community building
- Farmed animal welfare (land)
- Mental health
- AI risks
- Financial literacy
- Family planning
- Farmed animal welfare (aquatic)
- Improving diversity and inclusion
We think it’s important to list the limitations of our research, so that readers can make better judgements on this research.
- We don’t have a strong conclusion. We don’t think our findings should be taken very seriously, nor do we think our research process is a template that potential researchers should follow strictly. However, we do think our findings can be considered for future research or strategic decisions related to EA. If we were to conduct another 40 hours of research, we think that it’s very likely that the rankings will change somewhat - we estimate the following credences:
- 80% credence that at least four cause areas will remain in the top eight.
- 90% credence that at least one cause area will drop out of the top eight.
- We lacked experienced or trained researchers. Most of our on-the-go learning stemmed from reading Charity Entrepreneurship’s research methodologies. Lynn Tan, who was part of CE’s incubation programme in 2020, also helped out with the research.
- We lacked full-time researchers. All five of us were involved at a voluntary capacity for around 2 hours a week. As a result, we unintentionally spent longer than we intended conducting the research. The entire research process took 8 months - we started around July 2020 and concluded most of our findings by March 2021.
- Our findings may be outdated. Since there’s an approximate four month gap between us publishing this report and completing our literature review, we might have overlooked new evidence that appeared in between this duration.
- We conducted our research at a cause area level rather than at an intervention level. Since findings at such a level are very broad, it’s difficult to generalise conclusions on a specific cause area; moreover, conclusions may change dramatically at the intervention level of a specific cause area.
- We might not have sufficiently considered other metrics or cause areas that could be objectively important (i.e. unknown unknowns). For example, we didn’t include “advocating economic growth friendly policies” as a cause area, which is likely to have qualified in the top eight cause areas.
- In areas which lacked objective estimates, we had different “rigid” subjective beliefs that led to significant differences in conclusions which were difficult to reconcile (i.e. we retained our initial scores) despite having thorough discussion during the recalibration stage.
- Some of the cause areas had final scores that were highly similar, which means some cause areas didn’t rank over the other significantly. For example, the difference between the cause area that rated first and the cause area that rated second was a mere 0.46%.
- Given that we did not detail the exact steps taken and the specific literature we reviewed to inform our decision, our analysis may be difficult to replicate or be checked by independent researchers.
We emulated a large part of our methodology based on Charity Entrepreneurship’s research process. However, we also reduced a large chunk of CE’s research process into a more manageable methodology due to a lack of capacity and research experience. Some key differences include:
- Conducting the research at a cause area level rather than at a intervention level
- Choosing only one method to sort the ideas, namely the weighted factor model
- No individual reports for the top cause areas
Here is our research process:
- Idea generation stage
- We brainstormed for ideas, which produced 43 cause areas.
- Initial ranking stage
- We brainstormed for factors that we wanted to use for weighing the cause areas. Most of the factors can be found here.
- Each researcher weighed the factors using a 100-point distribution privately and independently; then aggregated the weightages using the median. The top three weighted factors, which were used for the initial ranking of cause areas, were neglectedness, cost-effectiveness, and logistical difficulty.
- We evaluated the 43 cause areas using the top three weighted factors and a 5-point Likert scale privately and independently; then aggregated the scores using the median. This evaluation was done without any additional research on any cause areas.
- We picked the top 12 cause areas from the initial 43 cause areas. We chose 12 mostly due to capacity constraints.
- We then combined the several cause areas into eight due to overlapping definitions.
- Weighted factor model stage
- We chose 12 factors that we wanted to use for weighing the top eight cause areas, and grouped them into two categories.
- Overall impact: evidence base, cost-effectiveness, neglectedness, logistical difficulty, metric focus, flexibility, flow through effects, timing, and risk of negative or no impact
- Limiting factor: scale limit, and funding availability, and talent availability
- We weighed the 12 individual factors using a 100-point allocation privately and independently; then aggregated the weightages using the median. We conducted a brief discussion to recalibrate our weightages.
- We weighed the two categories (overall impact and limiting factor) 60% and 40% respectively.
- We conducted a 1-hour literature review for each of the eight cause areas, and evaluated them the 12 factors on a 5-point Likert scale. This research and evaluation was done privately and independently.
- We then aggregated our individual scores using the median.
- We chose 12 factors that we wanted to use for weighing the top eight cause areas, and grouped them into two categories.
- Recalibration stage
- We conducted four 1.5-hour long discussions for each cause area and factor that had more than a two point difference in scores (i.e. we didn’t have similar conclusions). These discussions were used for understanding new evidence and perspectives from one another, which were subsequently used to update our beliefs and recalibrate our scores.
- We then aggregated our individual scores using the median.
|EA community building|
|Interventions that form a community of individuals interested in EA, as well as interventions that increase the chances of people taking significant actions on maximising personal impact (e.g. volunteering for EA groups, effective donations, local priorities research, high impact career planning).|
|Farmed animal welfare (land)|
|Interventions that reduce the suffering of land farmed animals|
|Interventions that improve the mental well-being of individuals.|
|Interventions that reduce harm caused by the use of AI-related technology, from current AI capabilities to superintelligence capabilities. This also includes existential risks caused by AI-related technology. (further reading)|
|Interventions that improve the set of skills and knowledge that allows an individual to make informed and effective decisions with their financial resources.|
|Interventions that improve the ability of individuals and couples to anticipate and attain their desired number of children, as well as the spacing and timing of their births|
|Farmed animal welfare (aquatic)|
|Interventions that reduce the suffering of aquatic farmed animals.|
|Improving diversity and inclusion|
|Interventions that reduce harm causing from unjust and unfair discrimination.|
For more detailed scores, you can find them here.
- Based on the findings of this research, we are more confident that having some voluntary part-time capacity to continue with EA community building in Malaysia is likely the right choice. However, without more evidence, we are uncertain whether a paid full-time capacity will be a good choice.
- We are more confident in the impact and expected value of organising events or discussions around certain EA-related cause areas listed above .
- With regards to further research, we think it might be helpful to:
- Conduct deeper research into EA-related cause areas listed above (e.g. EA community building, farmed animal welfare, mental health, and AI risks), in particular what specific interventions will be impactful and how impactful will they be.
- Conduct shallow research into other cause areas that are potentially impactful but did not make it into the initial ranking stage.
This research did not generate highly decision-relevant insights at this time. However, it did confirm our initial intuition (or increase our confidence) about which cause area is most important.
We should have been more specific with our research goals in the initial exploration stage by forecasting the value of information that we expected this research to generate. We should have then compared and prioritised which projects to take on with our limited capacity. We suspect doing this initial exercise would have helped us make a more informed decision and set better expectations.
EDIT from Yi-Yang: please do check out Jamie Harris's comment and my response below. I think it incapsulates learnings that I wasn't able to write down in our forum post submission. Basically, I wouldn't recommend using this methodology as the first iteration of local priorities research for groups.
If you’re interested in discussing more about our findings, please comment below or reach out to us at our Facebook page. You can also reach out to us by filling up this form.
Thanks so much for publishing this report Yi-Yang! I am really glad there is now one report written up properly on the EA Forum about an EA group's local priorities research.
I haven't been able to write a proper report yet for EA Philippines's local priorities research, but I hope to do that within the next few weeks or months. (If anyone is interested in reading about our work on this though, I wrote a bit about this in our 2020 annual report).
Your research process is more structured than ours, and I like the idea of independent research and aggregating people's ratings, which we aren't doing (or haven't done yet).
I think it would be a lot better though if you had "problem profiles" like 80,000 Hours's for those causes you listed, especially the top 2-4 causes. Maybe that's something to work on next, as something you can use to convince more Malaysians why those causes should be worked on. This is what we're doing at EA Philippines, and it also helps you easily give people resources they can read if they're interested to learn more about that cause area.
Or if not making full problem profiles, putting a few sentences or bullets about the scale and neglectedness of each of the causes would help. For example, you could put the number of farmed land animals and farmed aquatic animals in Malaysia slaughtered/produced every year, or the estimated % of Malaysians with a mental health disorder. You could also try putting a few sentences or bullets about tractability, but that's a lot harder to talk about or research about.
About your actual cause area rankings, my comments are:
thank you for your thoughtful feedback Brian! i'm unconfident to speak on behalf of my team so will bring up your feedback during our next meeting, notably on having problem profiles and the potential need to extend the report to clarify each cause area further (the present ones and some big absents).
Hi Brian! Thank for your response. I'll be using "we" (as a team) to address most of your comments, and "I" at the end to address one point.
Yes if there is a case for conducting further research, we are definitely considering deeper research in the top causes, and producing “problem profiles”.
We realised that our last point at the disclaimer didn’t make clear an additional related issue, which addresses this concern of yours. We didn’t detail which piece of evidence or arguments that made us give a certain score. Technically we did - it’s probably somewhere in our meeting minutes and it’s very messy - hence we’ve decided not to address this issue at this time. However, if we were to conduct another research like this, we definitely want to be better at making explicit our assumptions, evidences, and arguments.
We actually found a huge variance of scores for the above two causes areas in both the initial ranking stage and weighted factor model stage. So some of us in our team do agree with you that these cause areas shouldn’t be in the top 8. It also might be the case that we didn’t brainstorm enough cause areas that may reach the top 8.
As a side note, most of us in our team have a lot of strong feelings with diversity and inclusion issues in Malaysia (although some of us did put a lower score for this cause area, we weren’t that surprised it made it in the top 8). In a nutshell, issues of race and religion are often used as a dividing force within Malaysia at the legislative, political and social level in much of Malaysia’s modern history.
On a personal note, I wouldn't be surprised if these two cause areas actually do drop out in the next iteration of research (unless there's really convincing evidence of a cost-effective intervention).
Would love to check out EA PH's cause prioritisation report soon! :)
I imagine that this was a useful process for you guys to think through cause prioritisation research and relevant considerations. Looking at your methodology though, it seems as if you were attempting to essentially redo EA cause prioritisation research to date from scratch in a short timeframe?
My guess of the most useful process would have been to just take some of the most commonly / widely recommended EA cause areas (and maybe a couple of other contenders) and try to clarify how they seem more or less promising in the Malaysian context specifically.
If you agree with my characterisation of your process, with the benefit of hindsight, would you recommend that other national groups follow your methodology or my suggested alternative?
And, a somewhat separate question: what sorts of considerations do you think differ between Malaysia and other contexts in which these questions have been considered, if any?
You've made some good points that I didn't get to write in our forum post, and I've made an edit to direct readers to your comment.
Yes I agree. I think national groups should highly consider that their first iteration of local priorities research be that - taking recommended EA cause areas and conducting shallow research on them.
That's what we did EA Singapore, although not in a very deliberate way. Once that's out of the way, it saved a lot of time for deeper, more useful research. Here are the reasons why I think EA Malaysia chose to do this instead: (a) we wanted to test out a methodology, (b) we want to have a stronger consensus in our team when many felt some non-EA recommended areas should be included, and (c) we wanted to be sure we didn't miss any potential promising cause areas out.
I don't think they are good reasons per say, but I just wanted to put them out there.
I'm not sure if I understood your question correctly, so please do let me know if I didn't.
I don't think there's any significant considerations that are different, since most of these considerations (or the specific methodology) are from Charity Entrepreneurship. If I were to compare CE's methodologies to other methodologies used by other organisations, I imagine it would be significantly different.
Thanks a lot for the work and the report. Interesting findings you got there.
I have no experiences in EA so i'm just thinking out loud here - apologies in advance for my ignorance. I'm wondering if we have good reliable statistics on causes of deaths in the country (death being a proxy for suffering), and we could look into the categories of avoidable deaths (e.g. curable illnesses) and whether those areas are receiving enough support / funding. Also, from a poverty perspective, I'm curious if we have an idea how many Malaysians live in hardcore poverty and what can be done to get them out of it.
Also, what exactly is EA Malaysia's role as compared to EA global? I can imagine that global issues such as climate change and AI existential risks are also being heavily looked at by EA global and others, and depending on the issue, EA Malaysia's involvement could be either independent, complementary, or redundant.
I'm sure you have already thought about these points. Thanks again for the study.
Hi Zeshen! I'll be answering you from my own personal capacity, so my views are not EA Malaysia's.
For health specific statistics, I've used information from IHME. For animal consumption, I've used data from FAO.
It's a bit tough finding exact information about this. I did find one example from this report on the Lancet.
I have only done a bit of research on poverty, but my intuition tells me that Khazanah Research Institute probably has some information about this. One of the top Google search results is this report, which I find helpful in dealing with the "where is Malaysia's poverty line issue" that you may have seen in the news sometimes.
I love how you framed the outcomes of our involvement. I might even add, "destructive", which is different from "redundant" - our involvement could caused more harm than good.
Ideally, we want to be complementary, if working on a certain thing is not our comparative advantage. For example, I would imagine top AI governance research institutions elsewhere have a better comparative advantage than Malaysia's; this would mean that Malaysian wanting to work in this space using EA's perspective but still want to be in Malaysia, would probably have the most impact in localising AI governance research from elsewhere into policy recommendations.
I don't feel confident giving specific recommendations on reduce risk of doing redundant or destructive work, and increase the chance of doing complementary work. My only intuition to this is to over-coordinate (or coordinate more than you're used to).