Summary

Operations Research (OR) is the field of applying advanced analytics to make better decisions. That is, making the best possible decision under constraints and uncertainty – exactly the problem that EAs try to solve every day. Given the relevance and apparent lack of engagement with OR, I think this is a neglected skill set within EA. In particular, OR offers 

  1. tools and research for making large-scale projects more effective [more],
  2. a talent pool already skilled with these tools able to contribute to and propose relevant projects [more],
  3. rigorous conceptual frames for thinking about large-scale decision-making problems [more].

The key takeaways of this post are:

  1. EA Orgs should consider whether there are opportunities to apply off-the-shelf OR tools within your programs [more] or if focused OR research would be useful [more].
  2. Community Builders should allocate resources towards recruiting OR PhD students as they are both already skilled and generally receptive to EA ideas [more].
  3. Students  should consider whether any existing or potential OR projects excite them, and whether it is worth it to learn more about OR on their own or by taking classes.

Author Note: I am a current OR PhD student at MIT, so I have a good understanding of OR academia but have less visibility into the EA landscape and the existing operational sophistication of EA orgs. I’d like to thank Daniel Wang, Juan Gil, James Aung, Emma Abele, and Sudhanshu Kasewa for feedback on this post and more generally for their support in bringing me into the EA community.

What is OR?

Operations research (OR; sometimes also called management science, decision science, operations management, systems engineering, industrial engineering) is a combination of applied math, computer science, and economics focused on developing the mathematical theory and techniques necessary to solve large scale decision-making problems and applying such tools in practice.

Solving a decision-making problem usually implies finding the “best” decision out of a large decision space defined by a series of constraints (eg, maximizing the impact of bednet distribution subject to cost and logistical constraints). Mathematically, this is a constrained optimization problem. As such, the main technical pillar of OR is modeling and solving these types of optimization problems.

OR is used widely across industry for optimizing decisions about resource allocation, logistics, supply chain management, scheduling, manufacturing, and much more (see Appendix 1: OR Tools and Applications, and Appendix 2: Why haven’t I heard of OR?).

Why EA needs OR

There are several ways I can see OR being useful within EA: tools and research for making large-scale projects more effective, a talent pool already skilled with these tools able to take on relevant projects, and more rigorous conceptual frames for thinking about large-scale decision-making problems.

Relevant Research

The following is a nonexhaustive list of existing OR research[1] that could be of interest to the EA community. I believe this list demonstrates that

  1. There exists research which is already useful
  2. There exists significant altruistic energy within the OR community
  3. There exists substantial opportunity for more focused OR research on core cause areas that could be made more effective with the right partnerships (see next section)

Biosecurity

Global Health and Development

Philanthropy/Morality

Social Choice

AI Safety

Misc

You can search for your own keywords across OR journals here. There are also some existing altruistic OR orgs and awards, such as Analytics for a Better World (cofounded by my advisor) and the Doing Good with Good OR award.

Relevant Talent and Potential Projects

While the above research is relevant and interesting, it seems clear that with a more EA-focused lens, such research could be made more impactful. For example EA+OR projects, I have recently started two:

  1. Developing a discrete optimization toolkit for mechanistic interpretability of transformers trained on algorithmic tasks
  2. Better algorithms for sparse sensor placement, with the intended application of optimally placing wastewater genomic sequencers to minimize expected detection times under uncertainty of spread dynamics

I am especially excited about coordinating lists of shovel-ready OR projects and speculative research questions that would benefit from an OR perspective. Having such lists would facilitate matchmaking between orgs and students/professors[2] with the appropriate background and skill sets. Improving the previously cited work is a good starting place.

Another large class of projects is just solving canonical OR problems (resource management, logistics, scheduling, manufacturing, etc. – see Appendix 1) for EA orgs. My impression is that this will have the highest impact in mature orgs already operating at steady state, like Givewell top charities. For instance, the AMF ”How we make decisions” page looks like classic material for automated decision support tools that are well within the wheelhouse of OR.

Of course, you can optimize anything, and there are many unique problems/contexts/objectives in EA for which there is less existing work within OR but nonetheless would be appropriate to view from an optimization perspective. A flavor of potential projects in this genre[3]:

  1. Strategic food resilience (i.e. ALLFED): optimize portfolio of production facilities to maximize peak output production subject to cost, nutritional, resiliency, and ramp-up time constraints.
  2. Optimal network seeding: solve important network seeding problems to maximize social influence. Examples:
    1. Use coauthorship graphs of (capabilities) AI researchers to determine most important nodes in shifting field to safety
    2. Use cosponsorship graphs of legislators to determine most important nodes in passing X-risk policy
  3. Lead remediation: use hyperspectral remote sensing to map the concentration of lead (and other heavy metals) in soil/waterways. Optimize use of remediation resources to minimize public health damage/risk.
  4. Multiperiod network design for global development: develop decision support tools to help determine how to build out critical infrastructure (roads, power grids, internet, sewers, etc.) over long time horizons in developing countries to maximize welfare at minimum cost.
  5. Coal regulation: optimize shutdown of coal plants (or the design of incentives to do so) to jointly minimize further emissions, reduction of QALYs, and replacement energy costs.
  6. So much more: process design for scaling up industrial kelp farming, develop better tools for analyzing RCTs, a never ending supply of climate projects, cost-benefit analyses for policy proposals, optimal social choice mechanisms, optimizing shape of PPE stockpiles to maximize biorisk resilience, etc, etc, etc.

In the future, I hope to greatly expand this list (in length and specificity) and circulate it around OR departments. Please reach out if you have ideas or want to collaborate on developing such a list!

Conceptual Lessons from OR

I think the EA community already has many of the right intuitions regarding optimization and decision-making problems. Regardless, I think EA stands to benefit from having more rigorous conceptual foundations, especially around formulating and analyzing optimization problems.

Related to knowing what you're optimizing for, the first step in a decision making problem is to actually try to formulate a stylized optimization model. The main components of an optimization model are your decision variables, constraints, uncertainties, and objectives. As an example, when thinking about the design of the Nucleic Acid Observatory, a potential model would be

  1. Decision variables: binary variables indicating whether or not to place a genomic sequencer at all candidate locations under consideration
  2. Constraints: total costs, regulatory considerations
  3. Uncertainties: the dynamics of spread given an initial infection
  4. Objectives: worst-case detection, average-case detection time

Of course, there is much supporting data and other modeling choices you would need to make to actually construct and solve a real model, but even writing down something simple can clarify your thinking.

After you solve your model and get an optimal solution, you always have a corresponding set of tight constraints and slack constraints. Tight constraints indicate what your current bottlenecks are, and hence where you can expect positive marginal returns with additional investment. This can be instructive in thinking about the meta-optimization problem of how to change the constraints of your first-order problem so as to maximally increase your objective further (this sequence contains many examples of this flavor of reasoning; this section expands on constraint tightness).

When solving a multiobjective optimization problem (and most real world problems are multiobjective), rather than finding a single optimal solution, one typically wants to understand the whole pareto front. That is, the set of solutions for which there is no change that could improve one objective without reducing any of the other objectives. Understanding the pareto front is the foundation for understanding what tradeoffs you are actually faced with, and helps ground discussions of prioritization.

Why OR is a good target for EA outreach

I believe the OR community (especially PhD students) will be particularly receptive to EA ideas and working on EA style problems. As a discipline with a large focus on practice, OR is in constant search of new and interesting problems that could benefit from OR tools. In parallel, “social good” projects are becoming increasingly fashionable in academia, with many students and younger faculty being genuinely more altruistic. While the business-class old guard may be less naturally altruistic, there is still an incentive to keep the field relevant and attract young people by taking on high impact projects. However, without the right partnerships in place, I worry that much of this altruistic energy will be “frittered away” on projects that aren’t effective to begin with or by solving stylized problems which don’t help in practice.

There seems to exist a clear opportunity for many synergistic partnerships between the EA and OR communities. OR gets a (much needed) injection of fresh problems that are more exciting to students and expands its relevance in and out of academia (which is important given OR’s branding difficulties). In return, EA orgs get ~free and high-quality technical consulting, while growing the community and establishing further academic credibility.

Possible critiques and rebuttals

Possible critiques to the claim that OR has much to offer the EA community and, therefore, that 1) some EAs should study OR and 2) community builders should engage more with the OR community.

Critique: Most EA problems are not amenable to OR tools due to large uncertainties, nonlinearities, and less quantifiable objectives.

Rebuttal: I think this might be true regarding the highest-level resource allocation and project prioritization questions (eg, probably a bad idea to use vanilla portfolio theory to guide FTX philanthropy), but as EA projects continue to mature, bottlenecks are going to increasingly look more like standard operations problems that are the bread and butter of OR.

Critique: Most people able to do good OR are likely technical enough to work on AI safety and should just pursue CS instead!

Rebuttal: Even if you believe AI safety is many OOM more important than everything else, I think the OR perspective has much to offer to AIS, especially if you think an AGI architecture will include discrete/symbolic components. Even if it doesn’t, it still seems good to have people thinking about other avenues of attack given alignment’s pre-paradigmatic status.

Critique: At established orgs, this is just chasing after diminishing marginal returns, and at small orgs, this is premature optimization!

Rebuttal: I think both premature optimization of a small org and over-optimization in general are easy failure modes to be cautious of. It is true it isn’t worth chasing 5-10% efficiency gains in orgs with operational bottlenecks that will change in character with growth. In contrast, a 5-10% operational efficiency gain at a Givewell top charity operating in steady state seems like a pretty obvious win worthy of significant effort. The exact gains depend on the existing operational sophistication of these charities, but I suspect there is low-hanging fruit to be had.

Next steps

I see at least 3 distinct groups of people that might take an action item away from this post: students/professionals interested in learning more about OR, EA orgs that may be able to benefit from OR expertise, and community builders that could seek out altruistically minded people in the academic or professional OR communities.

Students/Professionals

If you want to get acquainted with the powerful OR tools I described, I would recommend looking at the Gurobi tutorials or the Mosek modeling cookbook. In general, I think learning how to formulate optimization models is the highest leverage skill, and that the theory is only necessary if you are attempting to do something fancy or really large scale.

If you are a current student, you could consider taking a class in OR/optimization that seems relevant to any of the problems/projects I have previously outlined. If you were really excited by what you read, you might even consider doing a minor/major[4]/graduate degree in OR. As a current PhD student in OR, I can attest that I probably have more topic freedom than any other non-OR graduate student I have ever met (congressional redistricting -> dynamical systems modeling -> deep learning interpretability + sensor placement) and could especially recommend this track to aspiring technical generalists.

EA Orgs

If you work at an EA(-adjacent) org operating at scale, you should consider whether there are opportunities to apply OR decision-making tools within your operations. If there are such opportunities and you don’t have the staff resources to invest in such a project, or just have questions about how to think about modeling your problem, please get in touch. If there is enough demand, this would further motivate the creation of EA consultancies with an OR unit.

Community Builders

I suspect it would be especially high leverage to divert a small fraction of existing OR talent into more altruistic (and effective) projects. This is probably best done in university graduate departments (those which I listed in Appendix 2) where the concentration of young talent is the greatest. This segment is also generally the most altruistic, has freedom to change research direction, and is easiest to access via existing university groups.

Get in touch 

Don’t be a stranger! I would love to hear from you about your ideas and how I can help!

 I will be in Berkeley until the end of July, at EAG-SF, and then in Cambridge, MA for the foreseeable future after that.

Appendix 1: OR Tools and Applications

To give further intuition on the flavor and breadth of problems OR can help solve, here are some major applications of OR in a non-EA context:

  1. Resource allocation/management
    1. Portfolio optimization: optimize asset allocation to maximize profit subject to risk constraints.
    2. Project portfolio optimization: Optimize the selection and scheduling of projects to maximize business value subject to budget, talent, and scheduling constraints.
    3. Marketing campaign optimization: Optimize the timing, frequency, and personalization of emails to maximize revenue subject to personal contact preferences.
  2. Transportation/routing
    1. General vehicle routing: many variants to this problem but generally optimize routes of a vehicle (fleet) to minimize costs subject to service constraints.
    2. Ambulance routing: Optimize placement and routes of ambulances to minimize expected response times.
    3. School bus routing: Optimize routes and stops to minimize costs subject to quality of service constraints (no student rides longer than an hour).
  3. Scheduling
    1. Sports Scheduling: Schedule the best match-ups in the TV time slots that have the widest possible audience subject to schedule rules (each team plays equal home and away games, 2 divisional games against each team, etc.)
    2. Staff scheduling: Assign workers to shifts to maximize individual preference satisfaction subject to operational constraints and labor regulation.
    3. Train scheduling: Optimize energy efficiency of train schedules subject to quality of service constraints.
  4. Logistics/Supply chain/inventory management
    1. Like all of Amazon
    2. All shipping companies
  5. Manufacturing (production and facilities planning): generally optimizing material flows, batch sizes, machine usage, etc. Example case studies:
    1. Bus manufacturing
    2. Chemical manufacturing
    3. Steel manufacturing
  6. Grid operations
    1. Unit commitment problem: Optimize electricity production decisions to supply to the expected energy demand at minimum cost subject to physical and regulatory constraints
    2. Power grid expansion
    3. Power storage optimization: optimize battery charge and discharge decisions to maximize revenue subject to physical and regulatory constraints.

See all the case studies from Gurobi for more examples.

Many optimization techniques have been developed to solve a wide variety of problems. Three important classes include

Discrete Optimization (often called mixed-integer programming) - Most decision-making problems feature binary yes/no decisions (should a facility be built at this location or not?) or otherwise consider indivisible quantities (we cannot assign 3.42 humans to a particular task) This means that continuous optimization methods (like gradient descent) do not work, and worse, these problems are combinatorial in nature (eg, there are ~10^62 possible routes a person could take on a 50 city tour), so brute force search is intractable. [learn more]

Robust/Stochastic Optimization - In the real world, we always have uncertainties. The two main paradigms for modeling with uncertainty are

  1. Robust Optimization - Optimize the worst case over a bounded set of uncertain parameters [learn more]
  2. Stochastic Optimization - Optimize the expected outcome over a finite set of deterministic scenarios (generally less scalable, but also less conservative than robust) [learn more]

Bayesian Optimization - Global optimization technique used for optimizing arbitrary expensive-to-evaluate black-box functions (eg, hyperparameter tuning of neural networks or sending out assets in a search-and-rescue mission) [learn more]

Given the breadth of “good decision making” and the general applicability of optimization, OR students and practitioners are typically educated in and regularly contribute to many adjacent fields:

  1. Computer Science - making good decisions requires making good use of data!
    1. ML/stats
    2. Algorithm design
    3. Classic AI (eg, search and satisfiability solvers)
  2. Economics - much of microeconomic theory rests on theory of linear+convex optimization
    1. Game theory
    2. Mechanism design
    3. Social choice
  3. Applied Math - lots of theoretical tools needed for mathematical modeling and optimization
    1. Stochastic processes (especially queuing theory)
    2. Graph theory/combinatorics
    3. Numerical analysis/simulation

Appendix 2: Why haven’t I heard of OR?

If OR is so great, why haven’t I heard of it?!

Historical Development  OR was initially developed for wartime operations and logistics during WWII, the first major peacetime applications were for oil production planning, and most modern day practitioners spend their days optimizing the bottom line of large multinationals. In other words, its historical overlap with altruistically minded people has been rather tiny.

Academic Fragmentation  OR goes by many names across academia, with each program having their own unique focus. For instance, some of the top departments in the US:

  1. MIT - Operations Research (OR)
  2. Stanford - Management Science and Engineering (MSE)
  3. Georgia Tech - Industrial and Systems Engineering (ISYE)
  4. Cornell - Operations Research and Information Engineering (ORIE)
  5. Berkeley - Industrial Engineering and Operations Research (IEOR)
  6. Michigan - Industrial and Operations Engineering (IOE)
  7. Princeton - Operations Research and Financial Engineering (ORFE)
  8. Columbia - Industrial Engineering and Operations Research (IEOR)
  9. CMU - Operations Research (OR)
  10. UIUC - Industrial & Enterprise Systems Engineering (IESE)
  11. UPenn - Operations, Information, and Decisions (OID)
  12. Rice - Computational Applied Mathematics and Operations Research (CMOR)

To make matters worse, many schools actually have two OR departments: one in the college of engineering and one in the business school (eg, Operations, Information & Technology at Stanford or The Decision, Risk, and Operations Division at Columbia)

Lacking Ecosystem The machine learning ecosystem is massive and high quality, with great open source libraries, a large community happy to answer questions, and tons of tutorials to get people up to speed. The optimization ecosystem lacks all of these: the best libraries (Gurobi, MOSEK, CPLEX) are all proprietary (free for academics but very expensive for everyone else); the community of people able and inclined to answer stackoverflow questions is quite small; most good tutorials are made by these companies (and are limited in number).

Bad Branding Operations research is not an intuitive name. Many of the core OR tools, like mixed-integer programming are also poorly named. The OR community has traditionally made ties with boring industries, which is good for business and funding but bad for attracting students and for brand image. 

  1. ^

    My criteria for what counts as OR (as opposed to CS, econ, etc.) is that the research appeared in an INFORMS journal/conference, was published by OR student/faculty, or used mixed integer optimization techniques.

  2. ^

     I am focusing on academic OR both because it is what I understand best, and because it is quite easy for students to change research directions and to do research on more applied problems. However, my hope is that eventually such projects are done within a professional EA consultancy, rather than being one-off PhD projects.

  3. ^

     Again with the caveat that I have high uncertainty of the quality of these ideas and of the existing operational sophistication of existing orgs in these spaces; this is mostly intended for illustration purposes.

  4. ^

     I think OR is best thought of as a graduate subject and therefore would not recommend it as a primary major. Instead, I would recommend something more foundational like CS, math, or econ (in that order).

68

New Comment
20 comments, sorted by Click to highlight new comments since: Today at 7:48 PM

Meta: would change the title to tell people what OR is, I would have been much more likely to click if I knew what it was and didn't just read every forum post.

But otherwise, I really enjoyed this post and cool to learn about a new field! Would love to see more like this.

Agreed. A title of "Why EA needs Operations Research" would likely result in more people reading these ideas.

Agreed! Commenting to make my strong upvote even stronger

Thanks for the feedback! I hope the revised title is an improvement.

Is there a reason you chose not to expand the 'OR' acronym in your revised title (or, for that matter, in your profile bio)? Its meaning may be obvious to you, but you may be overestimating how obvious it is to others.

I mostly expected people not to know what OR stood for, and then hover over the link to find out more. I also don't think "Operations Research" is actually much more elucidating,  especially given how overloaded the word "operations" is within EA, but it seems this may have been mistaken given this feedback so I have updated.

I think this post is on the money! I've proactively created a tag for operations research; feel free to add any information you think would be useful to have on the wiki entry.

I went to Cornell, and (as you state) operations research is one of the majors offered in the engineering school, but I never took any OR classes because I wrongly assumed they would be too dry and difficult. But in retrospect, I think that taking even a whirlwind tour of OR would have been incredibly insightful for me. We have an ML engineer on my team at work who was using mixed-integer programming (or something like that) for model selection.

I have some suggestions for next steps:

  • Compile a list of open-source OR tools (such as spark-or, which integrates with Spark, and these libraries for Python).
  • Do a workshop at a future EA Global or EAGx where you walk through an example OR problem related to EA, preferably with live coding.
  • Volunteer to improve open-source OR libraries (possibly for pay) and fund their development,  aiming for parity with the commercial ones.

Thank you for creating the tag! I was also a CS major at Cornell who actually only took the intro OR class (which was great but my impression is that undergrad ORIE at Cornell is not taught efficiently otherwise).

As for your suggestions

  • Emily Tucker has nice list of broadly useful resources for academic OR but this could probably be improved for an EA audience
  • This is a great idea!
  • The open-source solvers are way behind the commercial ones in terms of performance so I am skeptical this has high EV. AFAICT, much of Gurobi's licensing is customer specific, so I suspect non-profit orgs could get a steep discount if not a free license.

Jumping in on this comment thread: I would LOVE to see a collection of resources for self-teaching, either in the form of a well-organized syllabus if possible, or simply thrown together as a list with a short description of each resource. I've always felt bad that so much knowledge appears "locked" behind academia, and I am not able to access that knowledge (unless I have a GRE and a few recommendation letters and a 40,000 USD and two years to commit to the learning).

Very cool! I'm an epidemiologist, but I've long had the sense that most of the questions I find most interesting and relevant are actually OR! Your biosecurity list really resonated, many real-world problems reduce to where to position assets.

I have some concrete problems that I'd love to discuss, will reach out!

Yes, yes, yes! Although I have no/minimal background with the more technical aspects of Operations Research, my understanding of the field is that it has very large potential contributions to EA. I think that tools and methods from Operations Research would help a large variety of EA organizations achieve their goals more effectively and efficiently.

Occasionally senior people in EA write about how more professional management is needed, and I suspect that the tools and methods from Operations Research would fulfill a non-negligible part of that need/desire.

No real questions or concerns that I want to express at this point. Just hearty support. I'm glad you wrote this, and I hope it gets traction.

I was excited to read this post! At EA at Georgia Tech, some of our members are studying industrial engineering or operations research. Should we encourage them to reach out to you if they're interested in getting involved with operations research for top causes?

Yes, definitely!

Re: "Why haven't I heard of OR?", I think your comments on the fragmentation and branding challenges are extremely on point. Last year Effective Institutions Project did a scoping exercise looking at different fields and academic disciplines that intersect with institutional decision-making, and it was amazing to see the variety of names and frames for what is ultimately a collection of pretty similar ideas. With that said, I think the directions that have been explored under the OR banner are particularly interesting and impressive, and am really glad to have someone in the community who knows that field well!

My Operations Research degree definitely primed me for EA. Agree with this post.

I'd also add that economic forecasting involves OR techniques, and I think there are opportunities to integrate economic forecasts more into prediction platforms. This is my area of research, and for example my colleagues and I have recently used optimization to identify universal basic income policies that minimize disruption against the status quo, in the UK and in the US.

Sweet! At CMU, we also have a Decision Science major and a Center for Risk Perception & Communication LOL - it is definitely fragmented. Loved this post as an intro/info hub/general guide c:

As someone who did an industrial engineering undegrad and MS in management science I could not agree more with how conducive OR is to EA in general! I do want to pitch Stanford's MS in Management Science & Engineering program, which I recently completed, because I think it would be a great opportunity for some EAs who want to get a more technical graduate degree, dive deeper into a focus area, and plug in to the bay area EA and tech communities. They have concentrations that coincide with a lot of EA cause areas and paths, such as Financial Analytics (for earning-to-give), Technology and Engineering Management, Computational Social Science, Decision Analysis and Risk Analysis, and Health Systems Modeling. It's also a relatively short program (45 quarter units, 1-1.5 years) and allows you to take courses across many departments, like CS and economics.

Rethink Priorities has recently hired a full-time operations researcher. I'm excited to see what she comes up with.

[This comment is no longer endorsed by its author]Reply

I think what RP means by that term is "a researcher focused on figuring out & improving various operations-related things", which is closer to industrial/organizational psychology + business administration + some other stuff than to the field of "operations research" ("a discipline that deals with the development and application of advanced analytical methods to improve decision-making [...] considered to be a subfield of mathematical sciences" (Wikipedia)). 

So I think this is just an unfortunate overlap of terminology, rather than us actually doing something fairly similar to what this post is about.

(I guess I blame whoever named the field of operations research, since the name really doesn't feel intuitive, but that ship has probably long since sailed.)

(Unnecessary disclaimer: I work at RP and Peter is my manager, but that's not relevant to me reading this post or leaving this comment)

Thanks for the clarification!