This is the clearest thinking I've ever encountered on this topic. From an international development perspective, but could apply to RCTs for any social intervention, really. Knowing stuff is hard!
This is the clearest thinking I've ever encountered on this topic. From an international development perspective, but could apply to RCTs for any social intervention, really. Knowing stuff is hard!
The appropriate response- already adopted by the best social sciences where possible- is triangulation of research methods. (https://en.wikipedia.org/wiki/Triangulation_(social_science)) Other than that, blanket fetishisation or blanket dismissal of RCTs are both vices.
I found some of what Deaton said here very odd. Especially what sounds like a false equivalence of comparing the inference of causation from correlational data with experiments, which are "just like any other method of estimation."
Obviously, there can be external validity problems with RCTs, but this point seems to be overstated especially when it comes to global health interventions.
It is very difficult, expensive, and time consuming to unbundle the basket of causes that the average RCT is actually checking and test each one separately to see if it is truly critical. Two things that can help: 1. more money for RCT's so the most important ones can be investigated with follow ups, 2. Better RCT design via sharing of expertise among the developmental econ field and drawing in methodology expertise from outside the field.
The appropriate response- already adopted by the best social sciences where possible- is triangulation of research methods. (https://en.wikipedia.org/wiki/Triangulation_(social_science)) Other than that, blanket fetishisation or blanket dismissal of RCTs are both vices.