Wiki Contributions


Economic policy in poor countries

Yes, that seems like the main thing we disagree about. It also seems like we disagree about the likely impact of deworming. 

  1. I would also like to do that but I don't have time to do that properly unfortunately - my time is now focused on longterm relevant stuff. 
  2. I think the point here is that the growth decelerations don't seem like they would be big enough to make the back of the envelope calcs on the world bank, IMF and all economists in mine and Hauke's post lower than RCT stuff. 
  3. Thanks for clarifying. First, (this is not a criticism of you) this is different to the view taken by Open Phil which does try to compare its rich world policy to AMF and is what prompted the main post. So, at least Open Phil can't think it is an apples to orange comparison. Second, I don't really get how it is an apples to oranges comparison. You seem to be saying that the judgement of the probability is too high. but that is different to the metric used to compare the options being incomparable. I don't understand why you think the probability estimate is too high. The back of the envelope on the world bank and IMF seems like a pretty good guide. You really have to believe some implausible things to think that the probability of success is low enough. On the 'all economists' one for example, the estimate is extremely conservative because it only counts the economic gains in China and assumes that is the only thing that all economists achieved. 
  4. The reason I cited the Cochrane review was that GiveWell also cited it in their review of deworming. My main aim is to dispute GiveWell's reasoning as that is what is driving a lot of EA money, which I think could do much more good elsewhere. On external validity, I'm going off the Vivalt paper.
  5. It's a natural way to interpret GiveWell's views because the only charities they recommend, or (I think) have ever recommended, have been tested by RCTs. All of the charities on their standout list have all been tested by RCTs. This suggests at least that they put extremely high weight on RCT evidence. Even if they are not at the extreme of complete agnosticism sans RCT, they are very close to it. To use an example from Lant, Chris Blattman said that the best investment to fight world poverty would be to run an RCT comparing giving people cash or giving people chickens.  I find this stance extremely hard to understand without the implicit premise of agnosticism in the absence of RCTs. I don't think it would be difficult to find similar claims by duflo and banerjee with a bit of extra time. I do think that this goes beyond RCTs and extends to an over-reliance on empirical studies, as I argued here.
  6. I'm not sure which direction RCTs will go in in the future. The publication incentives  are to do that type of work.  
Economic policy in poor countries
  1. You seem here to be eliding the claim that we didn't find any leads with the claim that we didn't recommend anything. We found many  leads. FP found 7 different organisations that we thought it might be worth investigating if we had control of money. In Hauke's appendices to our post,  he lists several organisations and interventions that would be worth investigating. So, there are many plausible examples of things that are worthy of further study. If I put 3 months into a climate change report, I wouldn't argue that the failure to recommend a climate charity proves that aren't any good climate charities.
  2. It is true that economists have caused some bad things, but on balance, as Hauke and I discuss in section 3.2, the net gains of growth episodes have been extremely large since 1950. If economists affected the growth accelerations and decelerations in equal measure, then the effect would be strongly positive. For what you say to be true, the economists would have to have massively disproportionately affected the growth decelerations, which I don't think happened. Further to your criticism of intuition, your argument seems to be either that (i) all subjective probability assessments must be rejected because they are based on intuition or (ii) our subjective probability assessments are too high. Firstly, these two arguments are inconsistent. You cannot both say that you reject our argument because it is based on intuition and then say that according to your intuition the true probability is a lot lower. Secondly, argument (i) applies to all policy work, including the policy work that GiveWell are now going to investigate and that Open Phil already funds; I don't see why economic policy would be more subjective than air quality work. Thirdly, the case for deworming is also based on subjective probability judgements. For example, the various staff replicability adjustments that GiveWell staff members used to provide varied by a factor of 10. If you read the documentation of the way the replicability discount is now done, a lot of it is based on informal and qualitative reasoning. James Snowden describes GiveWell's mission as 'estimating the unknowable'. I don't think there is a sharp distinction in terms of non-subjectiveness between the things that GW recommends and policy stuff.
  3. To avoid misunderstanding here, am I right in interpreting you as saying the comparison as apples to oranges because philanthropists can never cause beneficial policy change? Do you also think comparing San Francisco planning reform and AMF is an apples to oranges comparison? I do in fact think there were ways to influence Chinese economic policy, namely by funding growth research, which is a global public good. As Hauke and I discuss in section 4 of our post, China under Deng sought extensive advice from foreign economists before making its reforms.
  4. I laid out several specific criticisms of the deworming estimate earlier in the thread. There are many many obviously important things that could affect the external validity of the study, and not just the worm burden which is what GW adjusts for. eg the intervention will now take place 20 years later, in a different country, with a different economy, different culture etc etc. The external validity problems are quite clear - there's a study showing that deworming some schoolchildren in Kenya increases income (one of many outcomes they were measuring) and we then need to make the assumption that this result will apply in all other contexts even though we have evidence from lots of other domains that the external validity of all RCTs is pretty minimal.  The 'only considers RCT' criticism of the Cochrane review is one that it is difficult for GW to make given its epistemic approach, which is the one I am criticising. 
  5. It's a bit tricky to carve out a difference between 'not practically possible' and 'not doable in such a way that you could get value of information'. Like you could sort of do an RCT where you allowed foreign investment in some companies but not in others, but it just seems like it would be a total mess to do that. The view that we should be agnostic before RCTs is I think an important guiding implicit assumption that a lot of people make. For example, it seems like by far the most obvious way to explain the approach GiveWell has taken since its inception. Lant discusses an example of a paper in the AEJ which found that building schools closer to children increases enrollment and says "In this paper, we evaluation a simple intervention entirely focused on access to primary schools. The empirical challenge is the potential endogenous relationship between the school availability and household characteristics. [Footnote 1].   Governments, for example, may place schooling either in areas of high demand for education or in areas with low demand for education, in the hopes of encouraging higher participation levels. Either will bias simple cross-sectional estimates of the relationship between access and enrollment." But the finding that proximity to school increases enrollment is something economists have known for ages on the basis of theory and non-RCT evidence .  Not all possible confounders are actual confounders. I didn't say or imply that "theory is going to tell us whether in the real world funding an NGO to lobby for change to Tanzania's macro policy is in expectation a better bet than funding deworming or bednets", it is so implausible that I have never given the impression of having believed it, and explicitly said in my last comment that I wasn't making such a claim- "(i'm not saying this is true of all topics!)"
  6. Yes you're right I'm happy to retract that. I had based my estimate on the Duflo numbers not on the wider field number. In answer to your question, I would have less than 5% of the field work on RCTs, maybe less than 1%. I do also think it is important that top 5 journals disproportionately publish RCTs as this does suggest that the top talent is disproportionately working on RCTs. 
Economic policy in poor countries


  1. My view is that I  think it is very likely (>95% chance) that one would find something better than  GiveWell's top charities if one were to put in 4 person-years of time into the project, which was the claim in my original post. My team put in about 6 months of person time on the project. Hauke and I put in a few months writing up our post, which wasn't about trying to find donation opportunities but just to make the case that there is reason to think we would find something if we tried. I don't know how much time Lant has put in to actually trying to find a donation opportunities, but that doesn't seem to be what he spends his time doing just looking at his research output. A substantial fraction of the time that Stephen and Aidan spent on the project was on establishing what happened if you analysed things in terms of life satisfaction rather than $s, as this is what Alex Berger has asked for in the response to mine and Hauke's post. I wouldn't expect to find a good donation opportunity after about 3-4 person months of trying from two (very smart) people without much prior experience in the area. I think that would take ~4 years, which is what I suggested in the original post. No-one has yet put that in, so I don't think it is surprising that no-one has found anything. 
  2. My claim that we would find something better than deworming is not just based on intuition. There are numerous independent lines of argument that suggest that working on economic policy must be better than directly funding deworming: (a) economic growth explains a substantial fraction of the variance in human welfare in the world today, variations in RCT-evaluable things explain little; (b) the era of focusing on national development rather than programmatic RCT-evaluable things produced greater improvements in human welfare than all prior human history combined, so let's keep doing that; (c) if you conservatively add up the impact of all development economists and make conservative assumptions about the effect they had on China, it looks better than directly funding RCT-type stuff; (d) similar thoughts apply to the World Bank and IMF.  As I say here,  I appreciate that the ICIER numbers may not be right (I haven't checked), but the value here is illustrative and does not affect the basic arguments. Even if you think there is only a slim chance of success of any feasible campaign succeeding, the size of the rewards are so large that the expected benefits are very large, and must be larger than deworming. 
  3. I don't think it is an apples to oranges comparison. I am asking what I would do as a philanthropist with money in eg East Asia in the 1950s, 60s and 70s, or in any developing country at any point. I have the option of something that could in principle be tested by an RCT and something that might increase growth, like privatising agriculture, legalising supermarkets or reducing tariffs. I don't think it would ever be the correct response to fund the RCT thing. I don't think the appeal to what is uncontroversial among economists works in this debate. It is also highly controversial whether deworming produces the posited income benefits. I don't understand why it is not fair to compare a single intervention to the suite of things that might increase growth rates. That seems like a fair comparison to me and the benefits are measured in $s, which is not an apples to oranges comparison. I use the deworming example because the benefits are provided in monetary terms, which makes the comparison easier. I also think the same applies for GW's other top charities. Indeed, I think the health benefits of their other charities is overrated because GiveWell overrates the life satisfaction benefits of saving lives. 
  4. I expect you and I to differ on this, but I do think there should be more criticism of the recommendation of deworming. EA is generally quite concerned about replicability and the various issues with science. But the deworming recommendation is only based on 1 RCT. Two meta-analyses have found that "deworming did not have an impact on outcomes such as cognition, height, and school performance", so it is very unclear how deworming is meant to have produced these economic benefits. There is evidence of data mining and multiple comparisons in the paper. There is strong evidence that the external validity of all RCTs is very low - this is not accounted for systematically and formally in GW's research. This is  a classic case where we would expect the study finding to not be real. 
  5. I don't think those would be useful RCTs because they have low value of information. With the road pricing one for example, i find it hard to see how you would set a control group in any useful sense. I know it is (very) unfashionable in economics to say this, but I think it is  true that most of the case for road pricing can be known from the armchair (i'm not saying this is true of all topics!)  Economists now tend to form their views based on the median published empirical study on a topic, which we have every reason to believe will be wrong. The approach of complete agnosticism on a topic before an RCT is produced on it just seems to me to be similarly wrong 
  6. I'm not sure your 'less than <10%' is right. Analysing the same top 5 journals for the same year (2015) Duflo suggests that a third of development economics papers are RCTs, which is what I based my claim on. 
This Can't Go On

Another point is - how much economic value can you extract from an atom? I have no idea how you would go about answering that question without making some other substantive arguments about the limits to growth. This suggests that the atoms argument is a weak steer on limits to growth. 

Economic policy in poor countries

Hello, yes this was in part a response to the arguments there where he suggested that policy is in the same ballpark as GiveWell top charities, which I don't think can be true given other things he says. 

"Yeah, I think that’s totally right. And I think, again, if you had more of a dominance argument, where it’s like, look, the returns to the policy are just always going to outweigh the returns to evidence-based aid, then I think you would end up with more back and forth and debate between them. But when you see the arguments for cost effectiveness actually ending up in the same ballpark, the same universe, it’s just like, cool, we can all get along. People are going to sort into buckets that appeal to them, or styles that work for them, or interventions that they’re more personally excited about. I think that’s totally healthy. And then when some of them think, actually, the expected value-type argument seems to lead you to think one of these is going to just totally destroy the other, that’s where I think you get a little bit more friction and tension and debate sometimes."

Because there is little public discussion about what happens in the near-termist area,  it is difficult to know why certain decisions are taken. I think it would be better for decisions that affect millions of dollars to be made with more public discussion and scrutiny. 

Economic policy in poor countries

Hello, thanks for this and thanks for assisting with our report on that!

To clarify, it's not the case that the FP report didn't have any recommendations because we didn't find anything good. We stopped because we couldn't guarantee that we could move enough money to the area to make it worth our own time and that of the organisations we evaluated. We didn't discover anything that made me think we wouldn't find something better than deworming if we were to put a couple of person-years into it. 

On your tldr - the point of my post is that I don't see how it could be true that the returns are not higher than direct RCT-tested projects. Either  US policy doesn't actually get close to the GiveWell bar, or poor world policy beats it. 

  1. Yes, what I meant there was RCT-evaluated things rather than run RCTs. Though I think the randomistas would indeed have wanted the Rwandan government to run RCTs?
  2. I have to say I disagree on deworming. If I had philanthropic resources to spend, I think it would have been a clear grave error to go to East Asia after 1950 and try to deworm people rather than getting them to do whatever it is that caused them to grow. This is basically for the reasons outlined by Lant's sniff test. It is true that "the fact that almost no rich or quickly growing  countries didn't do mass school deworming does not logically guarantee that mass deworming is not the most cost-effective economic intervention in the world". The point is that it makes it very unlikely. It is especially unlikely given that the case for deworming only rests on 1 RCT in the early 2000s, which I think has a lot of problems. eg they didn't find any improvements in health that might explain the economic benefits deworming is meant to have produced; the external validity of RCTs in general has been shown to be poor but, while some external validity factors are considered by GiveWell, this broader evidence (as far as I can tell) is not considered. 
  3. I agree that some things in the realms you mention  (agriculture, urban migration and so on) can in principle be tested by RCTs. But almost all of what almost everyone seems to think caused huge economic growth in China  is not testable by RCTs. You can't run an RCT into allowing foreign investment. Realistically, you can't run an RCT into allowing state-owned enterprises to charge a market price for their goods. I don't see how you could run an RCT on an immigration policy that you might  enforce in the real world - how would that work? Could you actually do a useful RCT on road pricing? I don't see how you could do a practicable and useful RCT on tax reform. All of these examples also illustrate that you don't need to run RCTs to know whether these things are good. You could run an RCT on decollectivising agriculture but given the economic history of the 20th century, I don't understand why you would want to do that. 
  4. I haven't looked into the ICIER case, so thanks for doing that. This is exactly the kind of analysis I would like to see happen, especially with organisations doing work now. Note that there are many other cases of economists influencing policy in the world. 
  5. I actually don't think some of the GiveWell numbers are based on more evidence. I don't  think there is more evidence that deworming produces the claimed income benefits than that agricultural decollectivisation is good, for example. The evidence appealed to is one RCT from Kenya, whereas there are countless examples of agricultural decollectivisation producing beneficial effects. The point of my post is that if ad hoc back of the envelope calculations are ok for US policy, then they are ok for poor world policy. Relatedly, I don't understand why EAs do not support a randomista approach to the rich world. 
AMA: Jason Brennan, author of "Against Democracy" and creator of a Georgetown course on EA

Is it sufficient for it to be good to vote for EAs to be better than the median voter? (which I think is probably true.)

AMA: Jason Brennan, author of "Against Democracy" and creator of a Georgetown course on EA

I think there is a typo in the bit about Weyl?

The interesting thing about the Weyl proposal is that it is an alternative to private property that could potentially produce better social outcomes from a consequentialist/utilitarian/social welfare point of view. The reason for this is that it overcomes the tragedy of the anti-commons, such that holdouts can extract rents, sometimes at huge expense to society. If Weyl's proposal would produce better outcomes, would you be in favour of it

AMA: Jason Brennan, author of "Against Democracy" and creator of a Georgetown course on EA

Given the non-identity problem, doesn't the requirement that future people benefit more than they lose allow us to leave future generations with quite a bad situation? eg emitting fossil fuels changes the identities of people in the future, and we could feasibly make the world >10C hotter which would leave lots of tropical countries in a bad state but it would not harm them since they would not have existed had we not emitted

AMA: Jason Brennan, author of "Against Democracy" and creator of a Georgetown course on EA

The point he is making is about worker cooperatives, rather than firms in general. A widely recognised problem with worker cooperatives is that there are disincentives to scale because adding more workers is a cost to the existing coop owners. So, the point doesn't apply to privately owned companies because adding workers do not get a share of the business

Load More