How can we trust the findings of EA organisations? It is a genuine possibility that I will change the entire course of my life as a result of the information on 80k hours. I guess many of you already have. Have you checked all of their reasoning? What percentage ought one to check? Or you trust someone else to have done that "due diligence"?
It's not enough to say they are transparent and seem honest - I know plenty of misguided transparent honest people. The issue is that EA organisations might be wrong, not in what they don't know (we cannot avoid being wrong in what we don't know) but in what they do - like a mistaken mathematical proof, their logic might be flawed or their sums might be off. This, by our own logic, would likely have disastrous results.
Frankly, someone needs to be checking their notes and I am not skilled enough, nor do I want to. I have yet to see this in regard to say 80k hours.
In this sense I can imagine three solutions:
Firstly, some sort of independent auditing body. They could read the works of EA organisations and attempt to see if the logic holds, flag up areas where decisions seem arbitrary etc. We would be paying someone to just be really on top of this stuff as their main job and to tell us if they found anything worrying. Arguably this forum kind of does this job, though A) we are all tremendously biased B) are people *really* checking the minutiae? I am not.
Secondly, multiple organisations independently asking the same questions. What if there were another 80k hours (called say, "Nine Years") which didn't try to interact with them, but sought answers to the same problems. "Nine Years" could publish their research and then we could read both summaries and then investigate areas of difference.
Thirdly, publish papers on our explanations as if they were mathematical (perhaps in philosophy journals). Perhaps this already happens (I guess if this post takes off I might research it more), but you could publish rigid testable explanations of the theories which undergird EA as an ideology. It seems well being (for instance) is very poorly defined. I'll explain more if people are interested, (read Deutsch's The Beginning of Infinity) but suffice it to say that to avoid being wrong you want to be definite so you can change. Is our ideology falsifiable? Sometimes EA seems very vague to me in its explanational underpinnings. If you can vary easily, it's hard to be wrong, and if you're never wrong, you never get better. I don't know if journals are the way to go but it seemed the easiest way to clearly suggest becoming more rigid.
Caveats
I do not know enough about EA - I've read about 20 hours of it in my life. Perhaps mechanisms like this already exist or you have reason to not require them.
I recently left religion and for that reason would like to know that I am not fooling myself here also. "Trust EA organisations because they are good" doesn't hold much water since the logic applies elsewhere - "Trust the Church because it is good"?
Summary
I think it would be good to have a mechanism for ensuring that we are not fooling ourselves here. EA redirects a huge number of person-hours and flaws in it could be catastrophic. I don't know what those are, but have got a few suggestions and am interested in your suggestions or criticisms of the ideas suggested here.
While I don't disagree that some kind of "independent auditor" might be useful, my advice to people considering a major change (which I make as a young person with relatively little life experience!) is as follows:
As best you can, try to only change your life by an amount proportionate with your trust in the change you plan to make.
If you don't feel like you really understand 80K's reasoning about a certain issue, read as much of their relevant work as you can find. If that doesn't give you enough confidence, reach out to them directly. If they don't have enough time or coaching slots to respond personally, ask people on the Forum for their thoughts on the likely effects of a given life-change. If you still feel fairly uncertain after that, keep asking questions and looking for better evidence.
And if you still can't find enough evidence after trying everything you can find... consider not changing your life, or trying out some smaller version of the change (freelancing for a few weeks instead of taking a new job, signing up for Try Giving instead of the full Giving What We Can pledge, etc.)
It may be the case that no career path has truly ironclad evidence for effectiveness, at least not in a way that applies to every job-seeker (since no two people have the same skills/personality/alternative options). If you come to believe that, you may be forced to make a change based on whatever evidence you can find (or based on more concrete information like salary, location, and other things that impact your personal well-being). But overall, I hope that people in EA mostly make life-changing decisions if they have a lot of confidence in those decisions, whether because of personal research or because they think highly of the quality of research conducted by EA organizations.
(I work for CEA, but these views are my own -- and, as I mentioned above, they're being made by someone who hasn't undergone too many major life changes.)
Yes, I agree that infrastructure would be better than no infrastructure. I'm not sure who I'd trust to do this job well among people who aren't already working on something impactful, but perhaps there are people in EA inclined towards a watchdog/auditor mentality who would be interested, assuming that the orgs being "audited" could work out an arrangement that everyone felt good about.