All of Oliver Sourbut's Comments + Replies

How do EAs deal with having a "weird" appearance?

Just seconding this. For context I work not in academia but as a software engineer and data scientist in London.

I usually have crazy sticky-up hair that sort of does different things each day especially as it grows. That's my main superficial weirdness (unless you count the unusually big nose) though I have plenty of other quirks which are harder to label and harder to spot from a distance.

In hindsight I think the hair has made me memorable and recognisable in my workplaces (e.g. people have expressed looking forward to seeing me and my hair in meetings...... (read more)

Many Undergrads Should Take Light Courseloads

Thank you, I found myself agreeing with most of this post and reflecting on how I might have optimised during my undergrad experience. On the other hand, I note that neither the post nor any comments yet contains what I consider an important caveat:

Taking extra classes is a great way to explore in the sense of dissolving known- and unknown-unknowns (what fits me? what problem-framings am I missing? what tools do other disciplines have? what concerns to people interested in X have? what even is there if I look further?)

Extra-curricular activities also enabl... (read more)

2Mauricio1moThanks! Good caveat. I'd add a caveat to the caveat: I'd still caution people from taking very heavy courseloads, because exploring (e.g. asking and reflecting on the questions in your second paragraph) seems hard when all of your time and attention goes to meeting deadlines.
2Zach Stein-Perlman1moYes! I'm glad the OP was written and I agree with many of its points. But if I hadn't taken extra classes, I wouldn't have taken CS, which I now (because I took extra classes) know is something that I am interested in — and might develop enough knowledge in to be useful (I'm still an undergraduate), from the point of view of the universe.
Prioritization Research for Advancing Wisdom and Intelligence

Yes yes, more strength to this where it's tractable and possible backfires are well understood and mitigated/avoided!

One adjacent category which I think is helpful to consider explicitly (I think you have it implicit here) is 'well-informedness', which I motion is distinct from 'intelligence' or 'wisdom'. One could be quite wise and intelligent but crippled or even misdirected if the information available/salient is limited or biased. Perhaps this is countered by an understanding of one's own intellectual and cognitive biases, leading to appropriate ('wise... (read more)

3Ozzie Gooen1moThat’s an interesting take. When I was thinking about “wisdom”, I was assuming it would include the useful parts of “well-informedness”, or maybe, “knowledge”. I considered using other terms, like “wisdom and intelligence and knowledge”, but that got to be a bit much. I agree it’s still useful to flag that such narrow notions as “well informedness” are useful.
Ben_Snodin's Shortform

It depends what media type you're talking about (audio, video, display, ...) - $6m/100m is $60CPM ('cost per mille'), which is certainly above the odds for similar 'premium video' advertising, but only by maybe 2-5x. For other media like audio and display the CPMs can be quite a bit lower, and if you're just looking to reach 'someone, somewhere' you can get a bargain via programmatic advertising.

I happen to work for a major demand-side platform in real-time ad buying and I've been wondering if there might be a way to efficiently do good this way. The pricing can be quite nuanced. Haven't done any analysis at this point.

How impactful is free and open source software development?

Hey, let me know if you'd like another reviewer. I'm a medium-experienced senior software engineer whose professional work and side-projects use various proportions of open-source and proprietary software. And I enjoy reviewing/proof-reading :)

Beyond fire alarms: freeing the groupstruck

I appreciated your detailed analysis of the fire alarm situation along with evidence and introspection notes.

I'm not sure if it opens up any action-relevant new hypothesis space, but one feature of the fire alarm situation which I think you did not analyse is that commonly people are concerned also for the welfare of their fellows, especially those who are close by. This makes sense: if you find yourself in a group, even of strangers (and you've reached consensus that you're not fighting each other) it will usually pay off to look out for each other! So pe... (read more)

Beyond fire alarms: freeing the groupstruck

This was a great read, thank you - I especially valued the multiple series of illustrating/motivating examples, and the several sections laying out various hypotheses along with evidence/opinion on them.

I sometimes wonder how evolution ended up creating humans who are sometimes nonconformist, when it seems socially costly, but I think a story related to what you've written here makes sense: at least one kind of nonconformity can sometimes shift a group consensus from a fatal misinterpretation to an appropriate and survivable group response (and furthermore... (read more)

EA Survey 2020: How People Get Involved in EA

Thanks for these very helpful insights! I thought the mosaic charts were particularly creative and visually insightful.

I have one minor statistical nit and one related question.

In cases where 'only one significant difference was found' (at a 95% c.i.), it could be worth noting that you have around 20 categories... so on average one spurious significant difference is to be expected! (If the difference is small.)

Also a question about how the significance test was carried out. so for calling a difference significant at 95% it mat... (read more)

2David_Moss6moHi Oliver. Thanks for your question! We actually just performed the same analyses as we did last year [] , so any references to significance are after applying the Bonferroni adjustment. We just decided to show the confidence intervals rather than just the binary significant/not significant markers this year, but of course different people have different views about which is better.
What harm could AI safety do?

To the extent that you are concerned about intrinsically-multipolar negative outcomes (that is, failure modes which are limited to multipolar scenarios), AI safety which helps only to narrowly align individual automated services with their owners could help to accelerate such dangers.

Critch recently outlined this sort of concern well.

A classic which I personally consider to be related is Meditations on Moloch

Careers Questions Open Thread

I really appreciate these data points! Actually it's interesting you mention the networking aspect - one of the factors that would push me towards more higher education is the (real or imagined?) networking opportunities. Though I get on very well with most people I work or study with, I'm not an instinctive 'networker' and I think for me, improving that could be a factor with relatively high marginal return.

As for learning practical skills... I'd hope to get some from a higher degree but if that were all I wanted I might indeed stick to Coursera and the l... (read more)

Careers Questions Open Thread

I welcome the reinforcement that a) it is indeed a tough call and b) I'm sane and they're good options! Thank you for the encouragement, and the advice.

I remain fuzzy on what shape 'impactful direct work' could take, and I'm not sure to what degree keeping my mind 'open' in that sense is rational (the better to capture emergent opportunities) vs merely comforting (specifying a path is scary and procastinating is emotionally safer)! I acknowledge that my tentative principal goal besides donations, if I continue engineering growth, is indeed working on safet... (read more)

Careers Questions Open Thread

As an engineer (software) myself for a few years, I can encourage you that is rewarding, challenging, and in the right position you can have quite a bit of autonomy to drive decision-making and execute on your own vision. Depending on the role and organisation, it can be far from merely technical; the outline you give of the college project sounds exactly like engineering to me!

That said, there are few or no places where engineers are completely unconstrained. But there are routes from engineering into more 'overseeing'-type roles, e.g. architect, tech dir... (read more)

Careers Questions Open Thread

I'm trying to choose between doubling down on skills in software engineering or branching out with the goal of working on AI safety longer term. I get the impression that a lot of people are in a similar position.

For me, my undergrad was an unusual mix of things but included Maths, Music (!) and Computer Science. I got good grades and I think there's a reasonable chance of my getting into a university like Oxford, Cambridge or Imperial to study a Masters and perhaps subsequently a PhD in Computer Science/AI.

Currently I'm paid well and developing a fair amo... (read more)

7Michelle_Hutchinson1yThat sounds like a great position to be in! This seems like a tough call because both options look high impact in expectation. On the plus side, that means that either decision is a reasonable one to make. Based on what you’ve said about the online courses and projects you’ve done, it sounds right to me that doing a Masters is the natural next step for testing out whether you’re a good fit for research. Anonymous_123’s suggestion of asking your employer about taking a year out to do a Masters sounds like a great plan. I also agree with them that waiting for the promotion sounds worth it. You don’t seem to mention working on AI safety as a software engineer (for example in a role like this [] ), or transitioning to ML engineering to work on safety (though maybe that’s what you were thinking about with ‘impactful direct work’. You could perhaps reach out to effective altruists who had done AI safety engineering such as Richard Ngo [] to get a better sense of how to compare the value of that with research and policy. I guess I tentatively agree that if you’re a great fit for research that would likely be more impactful, but it seems worth looking into.
2anonymous_1231yHey Oliver, this is a tough call. I would hold out for the promotion and vesting if it's only a couple years. Personally, I did a MS in CS/ML part time while working. It was a little brutal at times, took almost 4 years, and limited my choice of schools. But it was very good for my career, since I didn't have to sacrifice any job progression time. It's not the right choice for everyone (I don't have kids, so more flexibility), but it's one option. I will say that while a year or two to wait for a promotion/vesting might seem like a lot, it will fly by, and you will be in a very strong position from there. If you are donating and enjoying your job in the meantime, all the better! If possible, I wonder if you could negotiate a 1.5-2 year break from your company to go for the MS, pause the vesting (rather than lose it), and come back in at the senior position. This would be the best of both worlds if you could swing it, I've definitely heard stories like that. If you get into a top school, you could have good leverage because it looks good for your company. I've even known people to swing the deal so that the company pays the tuition with a 2 year commitment after. If after a year you love research, back out of the deal and continue for the PhD. Sounds like you are in a good position right now for all the reasons you state, so be careful before throwing that away, you could always get unlucky and find yourself simply starting over at a new company after getting the degree. Good luck!
You Should Write a Forum Bio

Thanks for this, Aaron. Joining a new community can be tricky online so it's helpful to have an explicit welcome like this!