Hi, I really like this (shared this in my social media channels). I am working on another new research methods project and great to see others working in this area. ( we ( regivolution.org ) presented our work in EA Global London last month, and we hope to make forum post in here soon as well about us as well as our rigorous ultra-rapid review methods ) We might even use this as an inspiration for some materials we will generate in future (We will of course cite/link to this resource)!
Thanks so much for the detailed comments and feedback!
1) I'm aware of Oxford Prio's research into this and had read their research. They looked more into Open Science which is related but slightly different from meta-research. There are many more funders funding transparancy, openness and data sharing than meta-research. I do not recall that Oxford prio looked Registered Reports specifically. I think it's an interesting analysis but I think the funders they listed in their analysis are neglecting Registered Reports.
2) re: the literature review methods: we do have a section titled 'Our Research process' in one of our reports. I did a literature review and based on expertise in science created a shortlist of potentially very impactful projects within meta-research (some other projects I considered are listed in the report linked above). Then I interviewed about 5 people in the meta-research field and asked them about their opinions on these projects. Registered Reports seemed most promising of all of these projects. How assess the quality of the papers I cite: as a general rule I look into the studies corresponding to how much weight I place on the point I'm trying to make. So if the study is crucial to the point I'm trying to I look into it more. I also try to find evidence from many different sources so that no argument rests on a single source (also see Sequence vs. Cluster Thinking by Givewell), and generally see whether there are many counter arguments and cite those. Thanks also for flagging the conflict of interest in the cancer study I cite, I absolutely see where you're coming from and have added a footnote to the analysis. In this particular case, I disagree that the study should be completely dismissed and has little value, only because this study was partially funded by a pharma cooperation. I believe that the results can broadly be informative and give a general idea of the impact of new drugs, though it might be an a biased overestimate. This because
Hi, I really like this (shared this in my social media channels). I am working on another new research methods project and great to see others working in this area. ( we ( regivolution.org ) presented our work in EA Global London last month, and we hope to make forum post in here soon as well about us as well as our rigorous ultra-rapid review methods ) We might even use this as an inspiration for some materials we will generate in future (We will of course cite/link to this resource)!
General feedback about the project and content:
Some improvement suggest... (read more)