Trying to assess the effectiveness of COVID-19 charities in India

This is a repost of something I wrote here: https://gotech.ghost.io/covid19india - this forum asked the same question as I did and I would welcome help from the rest of the community in building out what is a quick first draft.

Oxygen is a key mainstay of COVID treatment and supplies are particularly short in some places.  Jeff Coleman's back of the envelope calculations suggest that donating money for oxygen could be more beneficial in terms of life-years gained per $ than any of GiveWell's top rated charities, and that oxygen concentrators could be particularly impactful at the moment.  

Beyond oxygen, it was difficult to get good data on what key supply chain shortages actually are.  Is shipping oxygen cylinders directly cost effective?  How about donating money locally to buy/produce vaccines?    

In the meantime, I looked through some charities and tried to assess their efforts.  I ended up looking at what activities they were carrying out, how much detail they provided about their activities, the quality of any financial reporting/quality reporting processes they seemed to have and how easy it seemed to be to access financial breakdowns.  The exercise became more of a "gut feel" analysis than I would have liked but was still tremendously useful - I'm open to suggested edits/additions, and will touch on some of the further research I would have liked to have done at the end.

• PATH: a Seattle based charity - has an India Oxygen Drive.  https://www.path.org/p/india-medical-oxygen/.  Their site has good, clear financial reporting and decent rating on charity navigator.  However if you donate to them, it's not entirely clear where the money will go - their website states that "Donations to PATH are allocated to a variety of programs and projects, depending on current needs and emerging health issues." Overall rating: 7/10
• GiveIndia COVID response fund:  https://covid.giveindia.org/  I especially like that they let you choose the cause you want to donate to and have a thorough list (e.g. oxygen, food, cash).  Their website provides detail on what they plan to do (e.g. partner with local NGOs to set up oxygen generation plants and provide oxygen concentrators/cylinders) and how much each item is expected to cost.   Charity Navigator didn't rate them highly when last assessed because of a low "independent audit" score and not having enough independent board members.  They advertise a strict "due diligence framework of verifying all demand and supply channels are completed", and the financials section of their website seems to me to be transparent enough, with auditor reports and detailed breakdowns of costs.  Overall rating: 8/10.
• International Association for Human Values "Help India Breathe Again": https://www.iahv.org/in-en/donate/  Strange charity name but they seem to be doing good work, having partnered with famous spiritual leader Ravi Shankar's Art of Living Foundation. The website outlines a detailed plan for oxygen concentrators and ration kits, and they post photos of their distributions on Facebook. They make it pretty clear that: "all proceeds will be used in procuring and distributing ration kits" and elsewhere that "100% of your contributions will be utilized for the COVID relief work".  Charity Navigator gave them a decent rating (under their old rating system) and their website seems to back this up.  Their financials page also seems pretty good with independent auditor reports that have detailed breakdowns (e.g. salary vs travel expenses vs advertisement).  However the financials only seem to show up to 2018. Overall rating: 7.5/10.

IAHV's Facebook page

• SEWA International USA:  https://www.sewausa.org/Covid-19-India-FAQs They are shipping oxygen concentrators, ventilators and other equipment.  They have specific reports on how many oxygen concentrators have been sent (400), with many more ordered.  They advertise a 4-star rating from Charity Navigator.  They publish the independent auditor reports on the financials section of their website and appear to only spend a small proportion of funds on fundraising and adminstrative expenses. Overall rating: 7.5/10
• Project HOPE: https://www.projecthope.org/project-hope-responding-to-covid-19-surge-in-india/04/2021/
  They are responding by helping to procure PPE, oxygen supplies, ICU equipment, ventilators etc.  They proudly advertise that they meet various charity accountability standards, and are ranked by Charity Navigator, BBB and others.  Their financials section publishes accounts, independent audits and annual reports and it looks like they spend less than 15% on administration/marketing.  I especially like that they run a health policy journal (but found surprisingly little estimation of the impact of their projects last year, even in their annual report).  Overall rating: 8/10

Figuring out how effective these charity efforts are from their website alone is difficult.  I don't of course expect anyone to be carrying out a detailed impact analysis given how acute the COVID situation in India is, but thought that evidence of detailed policy evaluation, thorough reporting and easy-to-access data on previous projects would all point to rigorous processes in place.  The Against Malaria Foundation's website is a great example of the sort of quality that's possible, with detailed financial tables easily accessible from their homepage, a published decision making framework and individual distribution level reporting (though perhaps focusing on the single task of donating bed nets makes it easier).  

What I found was that there doesn't seem to be a great deal to separate the charities with all of them seeming to do decent work.  Most make their tax reporting easy to find and seem to keep the proportion of money allocated to "administration" and "marketing" (versus actual programs) low.  However, there was definitely some variability in the quality and detail of financial reporting, the use of independent audit, as well as the ease with which they made this available.  Proxy markers I know, but still useful for decision making.

Some important things that I haven't been able to look at yet are:

• Again, more specifics about current needs (what would be most useful right now?)
• Some further confirmation of Jeff Coleman's calculations: what saves more lives per $, an oxygen concentrator, a ventilator or an oxygen cylinder? A donation in cash or shipping equipment from abroad?  Which charities have the best handle on this?  Does equipment need to be sent from the US, or are there charities that can buy and ship quickly from other countries?
• Location specifics (e.g. which states/cities is it most important to donate to?)
• The financials of each charity in any detail (e.g. are they overspending on rent/salaries)

For now however, I have enough information to know where I'm going to donate this week...