All of James Smith's Comments + Replies

The link above no longer works but is available here

Thanks a lot for the response - can I just ask what WAW stands for? Google is only showing me  writing about writing, which doesn't seem likely to be it...

And how often does RP decide to go ahead with publishing academia? 

6
David_Moss
2y
  "WAW" = Wild Animal Welfare (previously often referred to as "WAS" for Wild Animal Suffering). I'd say a small minority of our projects (<10%).

I'm not sure this meets the 'spends the money effectively' criterion - it might, but we don't really know that yet. 

2
Nathan Young
2y
I think I'd like some reasons why you think this. Because it's a very easy thing to think and it could well be wrong.
1
Question Mark
2y
This is definitely true if "it's processes are simple" is a requirement for the longtermist organization in question. Influencing the far future is extremely difficult, and can't even remotely be called a simple process. There definitely are longtermist organizations that have a lot of room for funding and spend their money effectively, though.

What kinds of research questions do you think are better answered in an organisation like RP vs. in academia, and vice versa? 

One major factor that makes some research questions more suited to academia is requiring technical or logistical resources that would be hard to access or deploy in a generalist EA org like RP (some specialist expertise also sometimes falls into this category). Much WAW research is like this, in that I don't think it makes sense for RP to be trying to run large-scale ecological field studies.

Another major factor is if you want to promote wider field-building or you want the research to be persuasive as advocacy to certain audiences in the way that sometime... (read more)

Is there any particular reason why biosecurity isn't a major focus? As far as I can see from the list, no staff work on it, which surprises me a little. 

The short answer is that a) none of our past hires in longtermism (including management) had substantive  biosecurity experience or biosecurity interest and b) no major stakeholder has asked us to look into biosecurity issues.

The extended answer is pretty complicated. I will first go into why generalist EA orgs or generalist independent researchers may find it hard to go into biosecurity, explain why I think those reasons aren't as applicable to RP, and then why we haven't gone into biosecurity anyway.

Why generalist EA orgs or generalist independent r... (read more)

Thanks for posting this.

Just a quick note that it confused me a little to see the statement "And differences of 1030 are almost impossible" until I realised it is meant to be 10^30. It might be worth editing the post to make that clear. 

2
EdoArad
3y
Edited to fix this, thanks!
2
Habryka
3y
Yeah, I was about to say the same thing. Looks like some exponents got lost in the crossposting.

Glad that you enjoyed it. 

You are right on both counts. This is quite an easy overview of meta-research if you want a starting point: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.2005468 

I'd also be happy to have a look at anyone's ideas etc and have a decent idea (I think) of what TED-Ed might like. Just send me a message if I might be helpful. 

Do you have any example costs per time for campuspa.com? I couldn't see any on their website. 

Creating a new academic institute - the EA university - that houses a lot of EA research and (somehow) avoids the many issues seen in traditional academia. 

1
Sami Kassirer
2y
I like this! However, in a perfect world, rather than there being one university (or one institute at one university) that studies global priories, wouldn't all top research universities across the world  have global priorities schools (like business or policy schools are prevalent at most research universities)? With philosophers and scientists working together in one school on having the most impact on humanity, and coordinating with one another on how to do so—where students can get PhDs in Global Priorities Research (with specialization in one of the sub-fields, like business schools offer), and undergraduates at all universities around the world can major in global priorities, with paths towards academia and industry. Students majoring in GPR all take classes in the topics (e.g., longtermism, global health and development,  animal rights) and can create joint-majors with philosophy or one of the (social) sciences.  Business schools were only popularized about 100 years ago, and look at how much their proliferation has incentivized study and work in this space. Also, once the top universities create these GPR schools, many other universities not funded by EA would likely follow (esp. if it’s a profitable, self-sustaining business model). This might cost more than 100 million thought...there's probably data out there on how much it cost initially to start b-schools and policy schools.
4
EdoArad
3y
I'd be interested in thinking more about this, even as just a thought experiment :) 
6
SiebeRozendal
3y
let's add a high school/prep school to it ;-) Seriously though, I think having an institute more separate than GPI would not be great for disseminating research and gaining reputation. It would be nice though for training up EA students.
4
Jonas V
3y
(Also agree with Max. Long lead times in academia definitely qualify as a "convincing reason" in my view)

Thanks a lot, this is useful context. I work in academia so the large lead times are relevant, particularly because other 'traditional' funders would require applications well in advance. It would  be useful to know whether it was necessary to pursue those other funding routes as a 'career hedge' or not, for example, via a commitment to funding. 

I am interested to hear if anyone from LTFF agrees/disagrees with Max's assessment in these circumstances. 

4
evhub
3y
I'm one of the grant evaluators for the LTFF and I don't think I would have any qualms with funding a project 6-12 months in advance.

Is it possible to apply for a grant when the date you would want the funds is quite far in advance (say, for example, one year)? 

2
Jonas V
3y
I wouldn't rule it out, but typically we might say something like: We are interested in principle, but would like to wait for another 6-12 months to see how your project/career/organization develops in the meantime before committing the funding (unless there's a convincing reason for why you need the funding immediately).

A couple of other new publication models that might be worth looking at are  discussed here (Octopus and hypergraph, both of which are modular).  Also this recent article about 'publomics' might have interesting ideas.  Happy to talk about any of this if you are thinking about doing something in the space. 

2
Harrison Durland
3y
Those both seem interesting! I'll definitely try to remember to reach out if I start doing more work in this field/on this project. Right now it's just a couple of ideas that keep nagging at me but I'm not exactly sure what to do with them and they aren't currently the focus of my research, but if I could see options for progress (or even just some kind of literature/discussion on the epistemap/repository concept, which I have not really found yet) I'd probably be interested.

A system somewhat similar to what you are talking about exists. Pubpeer, for example, is a place where post-publication peer reviews of papers are posted publicly (https://pubpeer.com/static/about). I'm not sure at this stage how much it is used, but in principle it allows you to see criticism on any article. 

Scite.ai is also relevant - it uses AI to try and say whether citations of an article are positive or negative. I don't know about its accuracy. 

Neither of these address the problem of what happens if a study fails to replicate - often what ... (read more)

2
Harrison Durland
3y
Thanks for sharing those sources! I think a system like Pubpeer could partially address some of the issues/functions I mentioned, although I don't think it quite went as far as I was hoping (in part because it doesn't seem to have the "relies upon" aspect, but I also couldn't find that many criticisms/analyses in the fields I'm more familiar with so it is hard to tell what kinds of analysis takes place there). The Scite.ai system seems more interesting--in part because I have specifically thought that it would be interesting to see whether machine learning could assist with this kind of semantic-richer bibliometrics. Also, I wouldn't judge based solely off of this, but the Nature article you linked has this quote regarding Scite's accuracy: "According to Nicholson, eight out of every ten papers flagged by the tool as supporting or contradicting a study are correctly categorized."

That view seems reasonable to me and I agree that a clearer analysis would be useful. 

An additional and very minor point I missed out from my comment is that I'm sceptical that the relationship between impact factor and retraction (original paper here) is causal. It seems very likely to me that something like "number of views of articles" would be a confounder, and it is not adjusted for as far as I can tell. I'm not totally sure that is the part of the article that you were referring to when citing this, so apologies if not!

Thanks a lot for writing this post. I'm interested in these topics and was just thinking the other day that a write up of this sort would be valuable. 

A relevant and fairly detailed write-up (not mine) of this problem area and how meta-research might help  is available here: https://lets-fund.org/better-science/ (I didn't see it cited but may have missed it).

In terms of the content of the post, a couple of things that I might push back on a little: 

  1. Peer review: I’m not sure that poor peer review (of papers) is a major cause of ineffective va
... (read more)
4
C Tilli
3y
So glad to hear that, and thanks for the added reference to letsfund! On peer review I agree with Edo's comment, I think it's more about setting a standard than about improving specific papers. On IP, I think this is very complex and I think "IP issues" can be a barrier both when something is protected and when it's not. I have personally worked in the periphery of projects where failing to protect/maintain IP has been the end of road for potentially great discoveries, but also seen the other phenomena where researchers avoid a specific area because someone else holds the IP. It would be interesting to get a better understanding both of the scale of these problems and if any of the initiatives that currently exists seem promising for improving it. 
4
EdoArad
3y
Great points! Re peer-review, I think that your argument makes sense but I feel like most of the impact on quality from better peer review would actually be in raising standards for the field as a whole, rather than the direct impact on the papers who didn't pass peer review. I'd love to have a much clearer analysis of the whole situation :) 

Thanks a lot for sharing this. I need to update the post to add this and other research that has been pointed out to me. 

For future searching, where/how did you come across that paper? 

Good find - thanks for sharing that paper which I hadn't included.  If I update the post I'll add that. 

I haven't thought much about this so can't add anything useful at the moment. If I think of / come across anything I'll reply again. 

Good point. This is similar to what I was trying to get at when talking about lack of willingness to engage in probabilistic reasoning. 

Thanks a lot for the  comment. I was a bit nervous to put my first post up so some positive feedback is very much appreciated.

Thanks a lot for the comment. I do think that what your gesturing at makes sense: if I understand correctly you are saying that certain physical interventions can have more predictable effects that ‘biological’ ones because we have a decent idea of exactly how they work. In some cases this is definitely true: as an extreme example, we don’t need RCTs of aeroplane safety as we have a very good understanding of the physical processes and are able to model them well. If we have an airborne pathogen, it’s hardly necessary to run an RCT to see whether or not th... (read more)

3
Tsunayoshi
3y
There's an additional factor: Marketing and public persuasion. It is one thing to say: Based on a theoretical model, air filters work, and a totally different thing to say: We saw that air filters cut transmission by X% . My hope would be that the certainty and the effect estimate could serve to overcome the collective inaction we saw in the pandemic (in that many people agree that e.g. air filters would probably help, but barely nobody installed them in schools). 

Some quick thoughts (there is certainly already research on these but they seem important, and I don't know about reliability of existing research): 

  • Scope insensitivity: e.g. why do people find it hard to care proportionally more about proportionally bigger things?
  • Probabilistic reasoning: e.g. how can decision makers be ‘taught’ to take seriously low probability, high impact events?
  • Decision-making under uncertainty: e.g. how can this be improved? Can be people be efficiently taught to become more bayesian?
  • Group decision making: e.g. do more diverse groups really make better decisions?
  • Meta: e.g. how can social science become more reliable?
2
Abby Hoskin
3y
I like these a lot! Thanks for sharing :)

I like this perspective. I've never really understood why people find the repugnant conclusion repugnant! 

Not really answering your question, but there is some recent work attempting to forecast clinical trial results that may be relevant: Can Oncologists Predict the Efficacy of Treatments in Randomized Trials? Kimmelman (the senior author) is doing other work on the topic too (e.g. here). I'm not aware of much published work in this space in a biomedical context. 

My guess is that key decision makers in medicine (e.g. funders of trials), would not be very open to paying attention to forecasts (even if shown to be accurate to some degree), as there is a ve... (read more)

This may be less meta than you are hoping for, but may contain some useful advice/references: The dos and don’ts of influencing policy: a systematic review of advice to academics. Influencing policy is at least one way that academic ideas can travel to the wider world. 

I expect another is producing accessible content on the topic in question (e.g. writing popular blog posts, books, documentaries). It seems like these can sometimes be a catalyst for ideas becoming more widely known in the public. Examples of books that might have had or could have a br... (read more)

1
dominicroser
3y
Thanks for this!

As someone who did an undergraduate degree in biology, I think that as a computer scientist you probably already have many of the skills that you'd need to contribute to biology research directly. Welfare bio is a very new field so getting on top of the literature would likely not be too tricky, and most biologists would not have an in depth understanding of that particular sub-field anyway. There may be systematic reviews or modelling studies that you could contribute to, or you could look for existing datasets that could be reanalysed through a welfare b... (read more)

Thanks a lot for this response. I would probably donate through EA funds, so yes that should work. It seems like doing that with GiftAid will be a better bet than GAYE in my case then. The tip about HMRC is really useful to know - I have a friend who is giving regularly through GAYE and paying the 4% fee who is in a higher tax bracket, so I've recommended that he try this instead.

The Precipice by Toby Ord would be high up on my list. It is accessible and covers a lot of ground, illustrating a diversity of possible career paths and study areas that are relevant to existential risk.