Recently, an advisee told me that they've been procrastinating on replying to my email. It sits at the top of their stack each week. When they try to reply, instead they act on the prompts within, and so no longer need to correspond with me for the time-being.
They run this in a loop, and keep moving forward.
My email:
(1) Can you write out, say, 5 questions that you have uncertainty about? What would answers to these questions mean for your decision? (It’s important to pick questions/uncertainties that are actually decision-relevant, such that a different answer to those questions would change the decision you make.)
(2) Can you come up with 2-4 actions that can reduce that uncertainty/help get you more clarity? Are any of those actions particularly cheap? (Maybe reading a particular thing, talking to someone in a particular role or organisation?).
Consider setting up such prompts for your own weekly check-ins. Let me know some of your most effective prompts in the comments!
As an 80k advisor, my ToC is “Try and help someone to do something more impactful than if they had not spoken to me.”
Mainly, this is helping get people more familiar with/excited about/doing things related to AI safety. It’s also about helping them with resources and sometimes warm introductions to people who can help them even more.
Are there any particular pipelines / recommended programs for control research?
Just the things you probably already know about – MATS, Astra are likely your best bets, but look through these papers to see if there are any low hanging fruit as future work
What are the most neglected areas of work in the AIS space?
Hard question, with many opinions! I’m particularly concerned that “making illegible problems legible” is neglected. See Wei Dai’s writing about this
More groundedly, I’m concerned we’re not doing enough work on Gradual Disempowerment and more broadly questions of {how to have a flourishing future/what is a flourishing future} even if we avoid catastrophic risks
In general, AI safety work needs to contend with a collection of subproblems. See davidad’s opinion – A list of core AI safety problems
There are many other such opinions, and it’s good to scan through them to work out how they’re all connected, so that you can see the forest for the trees; and also to work out which problems you’re drawn to/compelled by, and seek out what’s neglected within those 🙂
Some questions about ops-roles:
What metrics should I use to evaluate my performance in ops/fieldbuilding roles? I find ops to be really scattered and messy, and so it's hard to point to consistent metrics.
Hard to talk about this in concrete terms, because ops is so varied; every task can have its own set of metrics. Instead, think through this strategically:
Be clear on the theory(ies) of change, and your roles/activities/tasks in it(them). Once you can articulate those things, the metrics worth measuring become a lot clearer
Sometimes we’re not tracking impact because impact evaluation is notoriously difficult. Look for proxies. Red-team them with people you admire
Fieldbuilding metrics can be easier to generate, but I don’t claim to be an expert here – ask folks at BlueDot, or the fellowships for better input.
How many people completed the readings?
How many people did I get to sign up for the bluedot course?
How many of those finished the bluedot course?
How many people did I get into an Apart Hackathon?
Did any of my people win?
And so on…
Likewise, I have a hard time discerning what “ops” really means. What are the best tangible “ops” skills I should go out of my way to skill up on if I want to work in the field building/programmes space? Are there “hard” ops skills I should become really good at (like, familiarity with certain software programmes, etc)
Ops is usually a “get stuff done” bucket of work. Yes, it can help to have functional experience in an ops domain like “Finance” or “IT/office tech infra/website” (and especially “Legal”), but a LOT of ops can be learned on the job/on your own; AI safety is stacked full of folks who didn’t let “I don’t know anything about ops” stop them from figuring it out and getting it done
Under what circumstances should a “technical person” consider switching their career to fieldbuilding?
First things first:
Fieldbuilding is not a consolation prize. Do fieldbuilding if you’re really passionate about helping AI go well, and fieldbuilding is your comparative advantage.
And doubling down on that:
It really really really helps if fieldbuilders are very competent. A fieldbuilder who doesn’t know their shit about AI risk and AI safety can propagate bad ideas among the people they’re inducting into the field.
This can have incredibly high costs
Pollutes the commons
Wastes time downstream where all this would need to be corrected
Bounces people who might be able to quickly get up to speed, because their initial contact with these fieldbuilders is of poor quality, poor argumentation, poor epistemics
Conversely a great fieldbuilder is one who knows how to tend their flock, what they need to prosper and grow to become competent at thinking about AI safety properly, and being able to do AI safety things
How would you recommend going about doing independent project work for upskilling in-place of doing something like SPAR or MATS?
Why not both? In general, I want people to ask themselves this question when making decisions. You can do a lot more than you give yourself credit for.
At the current margins SPAR, MATS etc. are probably better than independent work
Some of these fellowships have pretty high signal to employers (based on evidence that has been generated over time)
There is a lot that these fellowships offer that are sometimes hard to get without them
Research support, mentorship, community engagement, well-scoped projects with deliverables and accountability
Also softer things like physical space , some money
But if you’re great at doing stuff independently, go for it! Neel Nanda didn’t need a fellowship.
A key idea is to keep your eye on the ball – be productive!
The point is generate outputs
That make you learn
That show that you have learned
That are related to AI safety
That get feedback
That show that you update based on (relevant/good/high-quality) feedback
A few more things I often say that obliquely relate to networking:
Build in public and seek feedback. People like engaging with stuff online. Do stuff in public -- writing, github projects, YouTube shorts, anything -- and ask people for feedback. Repeatedly doing interesting things increases the quality of feedback and engagement in a positive cycle, and opens doors for deeper collaboration. Consider being pseudonymous if it is stressful to do it with own name.
Great post! Was just thinking about an intuition pump of my own re: EV earlier today, and it has a similar backdrop, of vaccine development. Also, you gave me a line with which to lead into it:
The work I do doesn't end up helping other researchers get closer to coming up with a cure.
Oh but it could have helped! It probably does (but there are exceptions like if your work is heavily misguided to the degree that nobody would have worked on it, or is gated).
By doing the work and showing it doesn't lead to a cure, you're freeing someone else who would have done that work to do some other work instead. Assuming they would still be searching for a cure, you've increased the probability that the remaining researchers do in fact find a cure.
I encounter "in 99.9% of worlds, I end up making no progress" a lot in my work, and I offer in its place that it is important and valuable to chase down many different bets to their conclusions, that the vaccine is not developed by a single party alone in isolation from all the knowledge being generated around them, but through the collected efforts of thousands of failed attempts from as many groups. The victor can claim only the lion's share of the credit, not all of it; every (plausible) failed attempted gets some part of the value generated from the endeavour as a whole, even ex post.
"anyone" is a high bar! Maybe worth looking at what notable orgs mightwanttofund, as a way of spotting "useful safety work not covered by enough people"?
Signal boost: Check out the "Stars" and "Follows" on my github account for ideas of where to get stuck into AI safety.
A lot of people want to understand AI safety by playing around with code and closing some issues, but don't know where to find such projects. So I've recently starting scanning github for AI safety relevant projects and repositories. I've starred some, and followed some orgs/coders there as well, to make it easy for you to find these and get involved.
Excited to get more suggestions too! Feel to comment here, or send them to me at [email protected]
Thanks. I sort of don't buy that that's what the Mechanize piece says, and in any case "no matter what you do" sounds a bit fatalistic, similar to death. Sure, we all die, but does that really mean we shouldn't try and live healthier for longer?
Not directly relating to your claim, but:
The Mechanize piece claims "Full automation is desirable", which I don't think I agree with both a priori and after reading their substantiation. It does not contend with the possibilities of catastrophic risks from fully automating, say, bioweapon research and development; it might be inevitable, but on desirability I think it's clear that it's only desirable once -- at the bare minimum -- substantial risks have been planned for and/or suitably mitigated. It's totally reasonable to delay the inevitable!
A stronger technological determinism tempers this optimism by saying that the kinds of minds you get will be whichever are easiest to build or maintain, and that those quite-specific minds will dominate no matter what you do.
Is there a thing you would point to that substantiates or richly argues for this claim? It seems non-obvious to me.
How would you transcribe 1-1s at EAG?
Assuming all parties have consented.
Some ideas from others:
Weekly Prompts
Recently, an advisee told me that they've been procrastinating on replying to my email. It sits at the top of their stack each week. When they try to reply, instead they act on the prompts within, and so no longer need to correspond with me for the time-being.
They run this in a loop, and keep moving forward.
My email:
Consider setting up such prompts for your own weekly check-ins. Let me know some of your most effective prompts in the comments!
Got sent a set of questions from ARBOx to handle async; thought I'd post my answers publicly:
lfg!
Not to karma-farm, but added a few more thoughts here
Following on from this post:
A few more things I often say that obliquely relate to networking:
Great post! Was just thinking about an intuition pump of my own re: EV earlier today, and it has a similar backdrop, of vaccine development. Also, you gave me a line with which to lead into it:
Oh but it could have helped! It probably does (but there are exceptions like if your work is heavily misguided to the degree that nobody would have worked on it, or is gated).
By doing the work and showing it doesn't lead to a cure, you're freeing someone else who would have done that work to do some other work instead. Assuming they would still be searching for a cure, you've increased the probability that the remaining researchers do in fact find a cure.
I encounter "in 99.9% of worlds, I end up making no progress" a lot in my work, and I offer in its place that it is important and valuable to chase down many different bets to their conclusions, that the vaccine is not developed by a single party alone in isolation from all the knowledge being generated around them, but through the collected efforts of thousands of failed attempts from as many groups. The victor can claim only the lion's share of the credit, not all of it; every (plausible) failed attempted gets some part of the value generated from the endeavour as a whole, even ex post.
"anyone" is a high bar! Maybe worth looking at what notable orgs might want to fund, as a way of spotting "useful safety work not covered by enough people"?
I notice you're already thinking about this in some useful ways, nice. I'd love to see a clean picture of threat models overlaid with plans/orgs that aim to address them.
I think the field is changing too fast for any specific claim here to stay true in 6-12m.
Signal boost: Check out the "Stars" and "Follows" on my github account for ideas of where to get stuck into AI safety.
A lot of people want to understand AI safety by playing around with code and closing some issues, but don't know where to find such projects. So I've recently starting scanning github for AI safety relevant projects and repositories. I've starred some, and followed some orgs/coders there as well, to make it easy for you to find these and get involved.
Excited to get more suggestions too! Feel to comment here, or send them to me at [email protected]
Thanks. I sort of don't buy that that's what the Mechanize piece says, and in any case "no matter what you do" sounds a bit fatalistic, similar to death. Sure, we all die, but does that really mean we shouldn't try and live healthier for longer?
Not directly relating to your claim, but:
The Mechanize piece claims "Full automation is desirable", which I don't think I agree with both a priori and after reading their substantiation. It does not contend with the possibilities of catastrophic risks from fully automating, say, bioweapon research and development; it might be inevitable, but on desirability I think it's clear that it's only desirable once -- at the bare minimum -- substantial risks have been planned for and/or suitably mitigated. It's totally reasonable to delay the inevitable!
Thanks Matt. Good read.
Is there a thing you would point to that substantiates or richly argues for this claim? It seems non-obvious to me.