All of ElizabethBarnes's Comments + Replies

Are there any AI Safety labs that will hire self-taught ML engineers?

I think DM clearly restricts REs more than OpenAI (and I assume Anthropic). I know of REs at DM who have found it annoying/difficult to lead projects because of being REs, I know of someone without a PhD who left Brain (not DeepMind but still Google so prob more similar) partly because it was restrictive, and lead team at OAI/Anthropic, and I know of people without an undergrad degree who have been hired by OAI/Anthropic. At OpenAI I'm not aware of it being more difficult for people to lead projects etc because of being 'officially an RE'. I had bad experiences at DM that were ostensibly related to not having a PhD (but could also have been explained by lack of research ability). 

2Rohin Shah4mo
I would love to hear actual details of these cases (presumably less publicly); at this level of granularity I can't tell to what extent this should change my mind (I can imagine worlds consistent with both your statements and mine).
The Future Fund’s Project Ideas Competition

High-quality human data

Artificial Intelligence

Most proposals for aligning advanced AI require collecting high-quality human data on complex tasks such as evaluating whether a critique of an argument was good, breaking a difficult question into easier subquestions, or examining the outputs of interpretability tools. Collecting high-quality human data is also necessary for many current alignment research projects. 

We’d like to see a human data startup that prioritizes data quality over financial cost. It would follow complex instructions, ensure high da... (read more)

COVID-19 brief for friends and family

Although I believe all the deaths were at a nursing home, where you'd expect a much higher death rate

COVID-19 brief for friends and family

Big source of uncertainty is how long the fatigue persists - it wasn't entirely clear from the SARS paper whether that was the fraction of people who still had fatigue at 4 years, or people who'd had it at some point. Numbers are very different if it's a few months of fatigue vs rest of your life. Not sure I've split up the persistent CF vs temporary post-viral fatigue properly

COVID-19 brief for friends and family

A friend pointed me to a study showing a high rate of chronic fatigue in SARS survivors (40%). I did a quick analysis of risk of chronic fatigue from getting COVID-19 (my best guess for young healthy people is ~2 weeks lost in expectation, but could be less than a day or more like 100 days on what seem like reasonable assumptions. )

Thank you for doing this. Has been on my list to look at for a while and am really glad we have numbers to work with.
EA Survey 2018 Series: Donation Data

Thanks for doing this! Some nitpicking on this graph: (donations and income)

1) the trendline looks a bit weird. Did you force it to go through (0,0)?

2) Your axis labels initially go up by factors of 100, then the last one only a factor of 10.

Thanks for your comment Elizabeth. The axis was just mislabelled (one missing 0). We updated the graph to fix that. As to the trendline, we just used a line of best fit, which assumes a linear relationship. The low R^2 (~30%) of this linear Donations~Income regression explains why it "looks a bit weird". It was used as an easy to interpret visual that depicted a simplified relationship between income and donations but one which demonstrated the correct direction of effect. This does have the disadvantage of being prone to overfitting, and as we noted "there are some large outliers driving this very strong relationship". We might expect a better fit for a nonlinear relationship, however, the later analysis with differing linear responses for different donor groups, was a reasonable fit.
Three Biases That Made Me Believe in AI Risk

Thanks for the post! I am generally pretty worried that I and many people I know are all deluding ourselves about AI safety - it has a lot of red flags from the outside (although these are lessening as more experts come onboard, more progress is made in AI capabilities, and more concrete work is done on safety). I think it's more likely than not we've got things completely wrong, but that it's still worth working on. If that's not the case, I'd like to know!

I like your points about language. I think there's a closely related p... (read more)

Why I prioritize moral circle expansion over reducing extinction risk through artificial intelligence alignment

You say

I think a given amount of dolorium/dystopia (say, the amount that can be created with 100 joules of energy) is far larger in absolute moral expected value than hedonium/utopia made with the same resources

Could you elaborate more on why this is the case? I would tend to think that a prior would be that they're equal, and then you update on the fact that they seem to be asymmetrical, and try to work out why that is the case, and whether those factors will apply in future. They could be fundamentally asymmetrical, or evolutionary pressures may tend... (read more)

Why I prioritize moral circle expansion over reducing extinction risk through artificial intelligence alignment

Thanks very much for writing this, and thanks to Greg for funding it! I think this is a really important discussion. Some slightly rambling thoughts below.

We can think about 3 ways of improving the EV of the far future:

1: Changing incentive structures experienced by powerful agents in the future (e.g. avoiding arms races, power struggles, selection pressures)

2: a) Changing the moral compass of powerful agents in the future in specific directions (e.g. MCE).

b) Indirect ways to improve the moral compass of powerful agents in the future (e.g. philosophy r... (read more)

The person-affecting value of existential risk reduction

See also the models in (cost-effectiveness of mitigating biorisk) and (asteroid risk), which have estimates for the risk level, cost of reducing it, and cost per qualy for different future discount levels.

"If we ignore distant future generations by discounting, the benefits of reducing existential risk fall by between 3 and 5 orders of magnitude (with a 1% to 5% discount rate), which is still far more cost-effective than m

... (read more)