All of Eva's Comments + Replies

Eva
3mo10
0
0

There's another point I don't quite know how to put but I'll give it a go.

Despite the comments above about having many ideas and getting feedback early about one's projects - which both point to having and abandoning ideas quickly - there's another sense in which actually what one needs is an ability to stick to things. And the good taste to be able to evaluate when to try something else and when to keep going. (This is less about specific projects and more about larger shifts like whether to stay in academia/a certain line of work at all.)

I feel like some... (read more)

Eva
3mo23
1
0

Thanks. A quick, non-exhaustive list:

  • Get feedback early on. Talking to people can save a lot of time
  • You should have a very clear idea of why it is needed. Good ideas sound obvious after the fact
  • That's not to say people won't disagree with you. If your idea takes off you will need to have a thick skin
  • A super-easy way to have more impact is to collaborate with others. This doesn't help for job market papers, where people tend to want to have solo-authored work. But you can get a lot more done collaborating with others and the outputs will be higher-quality,
... (read more)
Eva
3mo10
0
0

There's another point I don't quite know how to put but I'll give it a go.

Despite the comments above about having many ideas and getting feedback early about one's projects - which both point to having and abandoning ideas quickly - there's another sense in which actually what one needs is an ability to stick to things. And the good taste to be able to evaluate when to try something else and when to keep going. (This is less about specific projects and more about larger shifts like whether to stay in academia/a certain line of work at all.)

I feel like some... (read more)

Eva
1y61
20
1

I've stayed at a (non-EA) professional contact's house before when they'd invited me to give a talk and later very apologetically realized they didn't have the budget for a hotel. They likely felt obliged to offer; I felt like it would be awkward to decline. We were both at pains to be extremely, exceedingly, painstakingly polite given the circumstances and turn the formality up a notch.

I agree the org should have paid for a hotel, I'm only mentioning this because if baseline formality is a 5, I would think it would be more normal to kick it up to a 10 under the circumstances. It makes this situation all the more bizarre.

Eva
1y22
9
1

I would really like to read a summary of this book. The reviews posted here (edit: in the original post) do not actually give much insight as to the contents. I'm hoping someone will post a detailed summary on the forum (and, as EAs love self-criticism, fully expect someone will!).

Eva
1y42
17
1

I'm not going to deal with the topic of the post, but there's another reason to not post under a burner account if it can be avoided that I haven't seen mentioned, which this post indirectly highlights.

When people post under burner accounts, it makes it harder to be confident in the information that the posts contain, because there is ambiguity and it could be the same person repeatedly posting. To give one example (not the only one), if you see X number of burner accounts posting "I observe Y", then that could mean anywhere from 1 to X observations of Y, ... (read more)

2
BurnerExplainer
1y
I agree with the second paragraph of this comment. Regarding the third paragraph, In my specific case, 1. I acknowledged in the post that I previously used a burner two years ago for whose password I did not save (due it being a burner) and therefore found myself logged out of. I would have used the same burner otherwise. 2. I could flip this around and say that I don't have other active burners because fact that my next bullet-point in the post was "I'll do some of that now" with me proceeding to comment on recent grants in the same post under the same burner instead of making a different post with a different burner. 3. The use of the plural term "burner accounts" is me talking about burner accounts in the abstract rather than me saying I have multiple burner accounts.

Here's an example of a past case where a troll (who also trolled other online communities) made up multiple sock-puppet accounts, and assorted lies about sources for various arguments trashing AI safety, e.g. claiming to have been at events they were not and heard bad things, inventing nonexistent experts who supposedly rejected various claims, creating fake testimonials of badness, smearing people who discovered the deception, etc. 

Lizka
1yModerator Comment27
8
0

One thing I'd like to quickly flag on the topic of this comment: using multiple accounts to express the same opinion (e.g. to create the illusion of multiple independent accounts on this topic) is a (pretty serious) norm violation. You can find the full official norms for using multiple accounts here

This doesn't mean that e.g. if you posted something critical of current work on forecasting at some point in your life, you can't now use an anonymous account to write a detailed criticism of a forecasting-focused organization. But e.g. commenting on the... (read more)

Thanks for mentioning the Social Science Prediction Platform! We had some interest from other sciences as well.

With collaborators, we outlined some other reasons to forecast research results here: https://www.science.org/doi/10.1126/science.aaz1704. In short, forecasts can help to evaluate the novelty of a result (a double-edged sword: very unexpected results are more likely to be suspect), mitigate publication bias against null results / provide an alternative null, and over time help to improve the accuracy of forecasting. There are other reasons, as wel... (read more)

As a small note, we might get more precise estimates of the effects of a program by predicting magnitudes rather than whether something will replicate (which is what we're doing with the Social Science Prediction Platform). That said, I think a lot of work needs to be done before we can have trust in predictions, and there will always be a gap between how comfortable we are extrapolating to other things we could study vs. "unquantifiable" interventions.

(There's an analogy to external validity here, where you can do more if you can assu... (read more)

1
Davidmanheim
4y
Agreed on all points! I'd note that the problem with predicting magnitudes is simply that it's harder to do than predicting a binary "will it replicate," though both are obviously valuable.

Great comment. I don't think anyone, myself included, would say the means are not the same and therefore everything is terrible. In the podcast, you can see my reluctance to that when Rob is trying to get me to give one number that will easily summarize how much results in one context will extrapolate to another, and I just don't want to play ball (which is not at all to criticize!). The number I tend to focus on these days (tau squared) is not one that is easily interpretable in that way - instead, it's a measure of the unexplained variation in results - ... (read more)

This video might also add to the discussion - the closing panel at CSAE this year was largely on methodology, moderated by Hilary Greaves (head of the new Global Priorities Institute at Oxford), with Michael Kremer, Justin Sandefur, Joseph Ssentongo, and myself. Some of the comments from the other panellists still stick with me today.

https://ox.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=ec3f076c-9c71-4462-9b84-a8a100f5a44c

And when groups do work on these issues there is a tendency towards infighting.

Some things that could help:

  • Workshops that bring people together. It's harder to misinterpret someone's work when they are describing it in front of you, and it's easier to make fast progress towards a common goal (and to increase the salience of the goal).
  • Explicitly recognizing that the community is small and needs nurturing. It's natural for people to at first be scared that someone else is in their coveted area (career concerns), but overall I think it might be a good thin
... (read more)
0
RomeoStevens
6y
Do you think this is a side effect of the-one-true-ontology issues? Do you happen to know which conferences research results in this area tend to get presented at or which journals they tend to get published in? Could be useful to bootstrap from those networks. I've been tracing some citation chains from highly cited stat papers, but it's very low signal to noise for meta-research vs esoteric statistical methods.

I agree that it would be important to weigh the costs and benefits - I don't think it's exclusively an issue with RCTs, though.

One thing that could help in doing this calculus is a better understanding of when our non-study-informed beliefs are likely to be accurate.

I know at least some researchers are working in this area - Stefano DellaVigna and Devin Pope are looking to follow up their excellent papers on predictions with another one looking at how well people predict results based on differences in context, and Aidan Coville and I also have some work in this area using impact evaluations in development and predictions gathered from policymakers, practitioners, and researchers.