Summary

Operations Research (OR) is the field of applying advanced analytics to make better decisions. That is, making the best possible decision under constraints and uncertainty – exactly the problem that EAs try to solve every day. Given the relevance and apparent lack of engagement with OR, I think this is a neglected skill set within EA. In particular, OR offers 

1. tools and research for making large-scale projects more effective [more],
2. a talent pool already skilled with these tools able to contribute to and propose relevant projects [more],
3. rigorous conceptual frames for thinking about large-scale decision-making problems [more].

The key takeaways of this post are:

1. EA Orgs should consider whether there are opportunities to apply off-the-shelf OR tools within your programs [more] or if focused OR research would be useful [more].
2. Community Builders should allocate resources towards recruiting OR PhD students as they are both already skilled and generally receptive to EA ideas [more].
3. Students  should consider whether any existing or potential OR projects excite them, and whether it is worth it to learn more about OR on their own or by taking classes.

Author Note: I am a current OR PhD student at MIT, so I have a good understanding of OR academia but have less visibility into the EA landscape and the existing operational sophistication of EA orgs. I’d like to thank Daniel Wang, Juan Gil, James Aung, Emma Abele, and Sudhanshu Kasewa for feedback on this post and more generally for their support in bringing me into the EA community.

What is OR?

Operations research (OR; sometimes also called management science, decision science, operations management, systems engineering, industrial engineering) is a combination of applied math, computer science, and economics focused on developing the mathematical theory and techniques necessary to solve large scale decision-making problems and applying such tools in practice.

Solving a decision-making problem usually implies finding the “best” decision out of a large decision space defined by a series of constraints (eg, maximizing the impact of bednet distribution subject to cost and logistical constraints). Mathematically, this is a constrained optimization problem. As such, the main technical pillar of OR is modeling and solving these types of optimization problems.

OR is used widely across industry for optimizing decisions about resource allocation, logistics, supply chain management, scheduling, manufacturing, and much more (see Appendix 1: OR Tools and Applications, and Appendix 2: Why haven’t I heard of OR?).

Why EA needs OR

There are several ways I can see OR being useful within EA: tools and research for making large-scale projects more effective, a talent pool already skilled with these tools able to take on relevant projects, and more rigorous conceptual frames for thinking about large-scale decision-making problems.

Relevant Research

The following is a nonexhaustive list of existing OR research^[1] that could be of interest to the EA community. I believe this list demonstrates that

1. There exists research which is already useful
2. There exists significant altruistic energy within the OR community
3. There exists substantial opportunity for more focused OR research on core cause areas that could be made more effective with the right partnerships (see next section)

Biosecurity

• “Where to Locate COVID-19 Mass Vaccination Facilities?” Bertsimas, Dimitris, et al. Naval Research Logistics (NRL). https://doi.org/10.1002/nav.22007.
• “Designing Response Supply Chain Against Bioattacks.” Simchi-Levi, David, et al. Operations Research. https://doi.org/10.1287/opre.2019.1862.
• “Sequential Allocation of Vaccine to Control an Infectious Disease.” Rao, Isabelle J. Mathematical Biosciences.  https://doi.org/10.1016/j.mbs.2022.108879.
• “Forecasting COVID-19 and Analyzing the Effect of Government Interventions.” Li, Michael Lingzhi, et al. Operations Research. https://doi.org/10.1287/opre.2022.2306.
• "Accelerating Vaccine Innovation for Emerging Infectious Diseases via Parallel Discovery", Barberio, Joseph, et al.  National Bureau of Economic Research p. w30126 https://doi.org/10.3386/w30126.
• “OR’s Next Top Model: Decision Models for Infectious Disease Control.” Long, Elisa F., and Margaret L. Brandeau. Decision Technologies and Applications. https://doi.org/10.1287/educ.1090.0063.
• “Spatial Resource Allocation for Emerging Epidemics: A Comparison of Greedy, Myopic, and Dynamic Policies.” Long, Elisa F., et al. Manufacturing & Service Operations Management. https://doi.org/10.1287/msom.2017.0681.
• “Shield-Net: Matching Supply with Demand for Face Shields During the COVID-19 Pandemic.” Alcock, Rebecca, et al. INFORMS Journal on Applied Analytics. https://doi.org/10.1287/inte.2021.1112.
• “Placing Sensors in Sewer Networks: A System to Pinpoint New Cases of Coronavirus.” Nourinejad, Mehdi, et al. PLOS ONE. https://doi.org/10.1371/journal.pone.0248893.

Global Health and Development

• “Advancing Public Health and Medical Preparedness with Operations Research.” Lee, Eva K., et al. Interfaces. https://doi.org/10.1287/inte.2013.0676.
• “Ambulance Emergency Response Optimization in Developing Countries.” Boutilier, Justin J., and Timothy C. Y. Chan. Operations Research. https://doi.org/10.1287/opre.2019.1969.
• “The Nutritious Supply Chain: Optimizing Humanitarian Food Assistance.” Peters, Koen, et al.  INFORMS Journal on Optimization https://doi.org/10.1287/ijoo.2019.0047.
• “Analysis and Improvement of Blood Collection Operations: Winner—2017 M&SOM Practice-Based Research Competition.” Ayer, Turgay, et al. Manufacturing & Service Operations Management. https://doi.org/10.1287/msom.2017.0693.
• “Humanitarian and Disaster Relief Supply Chains: A Matter of Life and Death.” Day, Jamison M., et al. Journal of Supply Chain Management. https://doi.org/10.1111/j.1745-493X.2012.03267.x.
• “Polio Eradicators Use Integrated Analytical Models to Make Better Decisions.” Thompson, Kimberly M., et al. Interfaces. https://doi.org/10.1287/inte.2014.0769.
• Redesigning Sample Transportation in Malawi Through Improved Data Sharing and Daily Route Optimization. Gibson, Emma, et al. https://doi.org/10.2139/ssrn.3712556.
• “Stochastic Optimisation Model for Integrated Decisions on Relief Supply Chains: Preparedness for Disaster Response.” Manopiniwes, Wapee, and Takashi Irohara. International Journal of Production Research. https://doi.org/10.1080/00207543.2016.1211340.
• “The Impact of Unifying Agricultural Wholesale Markets on Prices and Farmers’ Profitability.” Levi, Retsef, et al. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1906854117.

Philanthropy/Morality

• “Getting the Rich and Powerful to Give.” Kessler, Judd B., et al. Management Science. https://doi.org/10.1287/mnsc.2018.3142.
• “Self-Interested Giving: The Relationship Between Conditional Gifts, Charitable Donations, and Donor Self-Interestedness.” Chao, Matthew, and Geoffrey Fisher. Management Science. https://doi.org/10.1287/mnsc.2021.4039.
• “The (In)Elasticity of Moral Ignorance.” Serra-Garcia, Marta, and Nora Szech. Management Science. https://doi.org/10.1287/mnsc.2021.4153.
• “Why Do People Volunteer? An Experimental Analysis of Preferences for Time Donations.” Brown, Alexander L., et al. Management Science. https://doi.org/10.1287/mnsc.2017.2951.
• “Moral Universalism: Measurement and Economic Relevance.” Enke, Benjamin, et al. Management Science. https://doi.org/10.1287/mnsc.2021.4086.

Social Choice

• “Fair Algorithms for Selecting Citizens’ Assemblies.” Flanigan, Bailey, et al. Nature. https://doi.org/10.1038/s41586-021-03788-6.
• Collective Discrete Optimisation as Judgment Aggregation. Boes, Linus, et al. http://arxiv.org/abs/2112.00574.
• Liberal Radicalism: A Flexible Design For Philanthropic Matching Funds. Buterin, Vitalik, et al. https://doi.org/10.2139/ssrn.3243656.
• “Preference Elicitation for Participatory Budgeting.” Benadè, Gerdus, et al. Management Science. https://doi.org/10.1287/mnsc.2020.3666.

AI Safety

• Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. Katz, Guy, et al. http://arxiv.org/abs/1702.01135.
• “A Robust Optimization Approach to Deep Learning.” Bertsimas, Dimitris, Xavier Boix, et al. ArXiv:2112.09279 [Cs, Math, Stat]. http://arxiv.org/abs/2112.09279.
• “Identifying Critical Neurons in ANN Architectures Using Mixed Integer Programming.” ElAraby, Mostafa, et al. ArXiv:2002.07259 [Cs, Math, Stat]. http://arxiv.org/abs/2002.07259.
• “Robustness Verification for Attention Networks Using Mixed Integer Programming.” Liao, Hsuan-Cheng, et al. ArXiv:2202.03932 [Cs]. http://arxiv.org/abs/2202.03932.
• “A Primal Dual Formulation For Deep Learning With Constraints.” Nandwani, Yatin, et al. Advances in Neural Information Processing Systems. https://proceedings.neurips.cc/paper/2019/hash/cf708fc1decf0337aded484f8f4519ae-Abstract.html.
• “Evaluating Robustness of Neural Networks with Mixed Integer Programming.” Tjeng, Vincent, et al. ArXiv:1711.07356 [Cs]. http://arxiv.org/abs/1711.07356.

Misc

• “Identifying Exceptional Responders in Randomized Trials: An Optimization Approach.” Bertsimas, Dimitris, Nikita Korolko, et al. INFORMS Journal on Optimization. https://doi.org/10.1287/ijoo.2018.0006.
• “Managing Catastrophic Climate Risks Under Model Uncertainty Aversion.” Berger, Loïc, et al. Management Science. https://doi.org/10.1287/mnsc.2015.2365.
• “Managing Supply Chains in Times of Crisis: A Review of Literature and Insights.” Natarajarathinam, Malini, et al. International Journal of Physical Distribution & Logistics Management, edited by R. Glenn Richey. https://doi.org/10.1108/09600030910996251.
• “Operations Research to Improve Disaster Supply Chain Management.” Ergun, Ozlem, et al. Wiley Encyclopedia of Operations Research and Management Science. https://doi.org/10.1002/9780470400531.eorms0604.
• “Value of Global Catastrophic Risk (GCR) Information: Cost-Effectiveness-Based Approach for GCR Reduction.” Barrett, Anthony Michael. Decision Analysis. https://doi.org/10.1287/deca.2017.0350.

You can search for your own keywords across OR journals here. There are also some existing altruistic OR orgs and awards, such as Analytics for a Better World (cofounded by my advisor) and the Doing Good with Good OR award.

Relevant Talent and Potential Projects

While the above research is relevant and interesting, it seems clear that with a more EA-focused lens, such research could be made more impactful. For example EA+OR projects, I have recently started two:

1. Developing a discrete optimization toolkit for mechanistic interpretability of transformers trained on algorithmic tasks
2. Better algorithms for sparse sensor placement, with the intended application of optimally placing wastewater genomic sequencers to minimize expected detection times under uncertainty of spread dynamics

I am especially excited about coordinating lists of shovel-ready OR projects and speculative research questions that would benefit from an OR perspective. Having such lists would facilitate matchmaking between orgs and students/professors^[2] with the appropriate background and skill sets. Improving the previously cited work is a good starting place.

Another large class of projects is just solving canonical OR problems (resource management, logistics, scheduling, manufacturing, etc. – see Appendix 1) for EA orgs. My impression is that this will have the highest impact in mature orgs already operating at steady state, like Givewell top charities. For instance, the AMF ”How we make decisions” page looks like classic material for automated decision support tools that are well within the wheelhouse of OR.

Of course, you can optimize anything, and there are many unique problems/contexts/objectives in EA for which there is less existing work within OR but nonetheless would be appropriate to view from an optimization perspective. A flavor of potential projects in this genre^[3]:

1. Strategic food resilience (i.e. ALLFED): optimize portfolio of production facilities to maximize peak output production subject to cost, nutritional, resiliency, and ramp-up time constraints.
2. Optimal network seeding: solve important network seeding problems to maximize social influence. Examples:
   1. Use coauthorship graphs of (capabilities) AI researchers to determine most important nodes in shifting field to safety
   2. Use cosponsorship graphs of legislators to determine most important nodes in passing X-risk policy
3. Lead remediation: use hyperspectral remote sensing to map the concentration of lead (and other heavy metals) in soil/waterways. Optimize use of remediation resources to minimize public health damage/risk.
4. Multiperiod network design for global development: develop decision support tools to help determine how to build out critical infrastructure (roads, power grids, internet, sewers, etc.) over long time horizons in developing countries to maximize welfare at minimum cost.
5. Coal regulation: optimize shutdown of coal plants (or the design of incentives to do so) to jointly minimize further emissions, reduction of QALYs, and replacement energy costs.
6. So much more: process design for scaling up industrial kelp farming, develop better tools for analyzing RCTs, a never ending supply of climate projects, cost-benefit analyses for policy proposals, optimal social choice mechanisms, optimizing shape of PPE stockpiles to maximize biorisk resilience, etc, etc, etc.

In the future, I hope to greatly expand this list (in length and specificity) and circulate it around OR departments. Please reach out if you have ideas or want to collaborate on developing such a list!

Conceptual Lessons from OR

I think the EA community already has many of the right intuitions regarding optimization and decision-making problems. Regardless, I think EA stands to benefit from having more rigorous conceptual foundations, especially around formulating and analyzing optimization problems.

Related to knowing what you're optimizing for, the first step in a decision making problem is to actually try to formulate a stylized optimization model. The main components of an optimization model are your decision variables, constraints, uncertainties, and objectives. As an example, when thinking about the design of the Nucleic Acid Observatory, a potential model would be

1. Decision variables: binary variables indicating whether or not to place a genomic sequencer at all candidate locations under consideration
2. Constraints: total costs, regulatory considerations
3. Uncertainties: the dynamics of spread given an initial infection
4. Objectives: worst-case detection, average-case detection time

Of course, there is much supporting data and other modeling choices you would need to make to actually construct and solve a real model, but even writing down something simple can clarify your thinking.

After you solve your model and get an optimal solution, you always have a corresponding set of tight constraints and slack constraints. Tight constraints indicate what your current bottlenecks are, and hence where you can expect positive marginal returns with additional investment. This can be instructive in thinking about the meta-optimization problem of how to change the constraints of your first-order problem so as to maximally increase your objective further (this sequence contains many examples of this flavor of reasoning; this section expands on constraint tightness).

When solving a multiobjective optimization problem (and most real world problems are multiobjective), rather than finding a single optimal solution, one typically wants to understand the whole pareto front. That is, the set of solutions for which there is no change that could improve one objective without reducing any of the other objectives. Understanding the pareto front is the foundation for understanding what tradeoffs you are actually faced with, and helps ground discussions of prioritization.

Why OR is a good target for EA outreach

I believe the OR community (especially PhD students) will be particularly receptive to EA ideas and working on EA style problems. As a discipline with a large focus on practice, OR is in constant search of new and interesting problems that could benefit from OR tools. In parallel, “social good” projects are becoming increasingly fashionable in academia, with many students and younger faculty being genuinely more altruistic. While the business-class old guard may be less naturally altruistic, there is still an incentive to keep the field relevant and attract young people by taking on high impact projects. However, without the right partnerships in place, I worry that much of this altruistic energy will be “frittered away” on projects that aren’t effective to begin with or by solving stylized problems which don’t help in practice.

There seems to exist a clear opportunity for many synergistic partnerships between the EA and OR communities. OR gets a (much needed) injection of fresh problems that are more exciting to students and expands its relevance in and out of academia (which is important given OR’s branding difficulties). In return, EA orgs get ~free and high-quality technical consulting, while growing the community and establishing further academic credibility.

Possible critiques and rebuttals

Possible critiques to the claim that OR has much to offer the EA community and, therefore, that 1) some EAs should study OR and 2) community builders should engage more with the OR community.

Critique: Most EA problems are not amenable to OR tools due to large uncertainties, nonlinearities, and less quantifiable objectives.

Rebuttal: I think this might be true regarding the highest-level resource allocation and project prioritization questions (eg, probably a bad idea to use vanilla portfolio theory to guide FTX philanthropy), but as EA projects continue to mature, bottlenecks are going to increasingly look more like standard operations problems that are the bread and butter of OR.

Critique: Most people able to do good OR are likely technical enough to work on AI safety and should just pursue CS instead!

Rebuttal: Even if you believe AI safety is many OOM more important than everything else, I think the OR perspective has much to offer to AIS, especially if you think an AGI architecture will include discrete/symbolic components. Even if it doesn’t, it still seems good to have people thinking about other avenues of attack given alignment’s pre-paradigmatic status.

Critique: At established orgs, this is just chasing after diminishing marginal returns, and at small orgs, this is premature optimization!

Rebuttal: I think both premature optimization of a small org and over-optimization in general are easy failure modes to be cautious of. It is true it isn’t worth chasing 5-10% efficiency gains in orgs with operational bottlenecks that will change in character with growth. In contrast, a 5-10% operational efficiency gain at a Givewell top charity operating in steady state seems like a pretty obvious win worthy of significant effort. The exact gains depend on the existing operational sophistication of these charities, but I suspect there is low-hanging fruit to be had.

Next steps

I see at least 3 distinct groups of people that might take an action item away from this post: students/professionals interested in learning more about OR, EA orgs that may be able to benefit from OR expertise, and community builders that could seek out altruistically minded people in the academic or professional OR communities.

Students/Professionals

If you want to get acquainted with the powerful OR tools I described, I would recommend looking at the Gurobi tutorials or the Mosek modeling cookbook. In general, I think learning how to formulate optimization models is the highest leverage skill, and that the theory is only necessary if you are attempting to do something fancy or really large scale.

If you are a current student, you could consider taking a class in OR/optimization that seems relevant to any of the problems/projects I have previously outlined. If you were really excited by what you read, you might even consider doing a minor/major^[4]/graduate degree in OR. As a current PhD student in OR, I can attest that I probably have more topic freedom than any other non-OR graduate student I have ever met (congressional redistricting -> dynamical systems modeling -> deep learning interpretability + sensor placement) and could especially recommend this track to aspiring technical generalists.

EA Orgs

If you work at an EA(-adjacent) org operating at scale, you should consider whether there are opportunities to apply OR decision-making tools within your operations. If there are such opportunities and you don’t have the staff resources to invest in such a project, or just have questions about how to think about modeling your problem, please get in touch. If there is enough demand, this would further motivate the creation of EA consultancies with an OR unit.

Community Builders

I suspect it would be especially high leverage to divert a small fraction of existing OR talent into more altruistic (and effective) projects. This is probably best done in university graduate departments (those which I listed in Appendix 2) where the concentration of young talent is the greatest. This segment is also generally the most altruistic, has freedom to change research direction, and is easiest to access via existing university groups.

Get in touch 

Don’t be a stranger! I would love to hear from you about your ideas and how I can help!

 I will be in Berkeley until the end of July, at EAG-SF, and then in Cambridge, MA for the foreseeable future after that.

Appendix 1: OR Tools and Applications

To give further intuition on the flavor and breadth of problems OR can help solve, here are some major applications of OR in a non-EA context:

1. Resource allocation/management
   1. Portfolio optimization: optimize asset allocation to maximize profit subject to risk constraints.
   2. Project portfolio optimization: Optimize the selection and scheduling of projects to maximize business value subject to budget, talent, and scheduling constraints.
   3. Marketing campaign optimization: Optimize the timing, frequency, and personalization of emails to maximize revenue subject to personal contact preferences.
2. Transportation/routing
   1. General vehicle routing: many variants to this problem but generally optimize routes of a vehicle (fleet) to minimize costs subject to service constraints.
   2. Ambulance routing: Optimize placement and routes of ambulances to minimize expected response times.
   3. School bus routing: Optimize routes and stops to minimize costs subject to quality of service constraints (no student rides longer than an hour).
3. Scheduling
   1. Sports Scheduling: Schedule the best match-ups in the TV time slots that have the widest possible audience subject to schedule rules (each team plays equal home and away games, 2 divisional games against each team, etc.)
   2. Staff scheduling: Assign workers to shifts to maximize individual preference satisfaction subject to operational constraints and labor regulation.
   3. Train scheduling: Optimize energy efficiency of train schedules subject to quality of service constraints.
4. Logistics/Supply chain/inventory management
   1. Like all of Amazon
   2. All shipping companies
5. Manufacturing (production and facilities planning): generally optimizing material flows, batch sizes, machine usage, etc. Example case studies:
   1. Bus manufacturing
   2. Chemical manufacturing
   3. Steel manufacturing
6. Grid operations
   1. Unit commitment problem: Optimize electricity production decisions to supply to the expected energy demand at minimum cost subject to physical and regulatory constraints
   2. Power grid expansion
   3. Power storage optimization: optimize battery charge and discharge decisions to maximize revenue subject to physical and regulatory constraints.

See all the case studies from Gurobi for more examples.

Many optimization techniques have been developed to solve a wide variety of problems. Three important classes include

Discrete Optimization (often called mixed-integer programming) - Most decision-making problems feature binary yes/no decisions (should a facility be built at this location or not?) or otherwise consider indivisible quantities (we cannot assign 3.42 humans to a particular task) This means that continuous optimization methods (like gradient descent) do not work, and worse, these problems are combinatorial in nature (eg, there are ~10^62 possible routes a person could take on a 50 city tour), so brute force search is intractable. [learn more]

Robust/Stochastic Optimization - In the real world, we always have uncertainties. The two main paradigms for modeling with uncertainty are

1. Robust Optimization - Optimize the worst case over a bounded set of uncertain parameters [learn more]
2. Stochastic Optimization - Optimize the expected outcome over a finite set of deterministic scenarios (generally less scalable, but also less conservative than robust) [learn more]

Bayesian Optimization - Global optimization technique used for optimizing arbitrary expensive-to-evaluate black-box functions (eg, hyperparameter tuning of neural networks or sending out assets in a search-and-rescue mission) [learn more]

Given the breadth of “good decision making” and the general applicability of optimization, OR students and practitioners are typically educated in and regularly contribute to many adjacent fields:

1. Computer Science - making good decisions requires making good use of data!
   1. ML/stats
   2. Algorithm design
   3. Classic AI (eg, search and satisfiability solvers)
2. Economics - much of microeconomic theory rests on theory of linear+convex optimization
   1. Game theory
   2. Mechanism design
   3. Social choice
3. Applied Math - lots of theoretical tools needed for mathematical modeling and optimization
   1. Stochastic processes (especially queuing theory)
   2. Graph theory/combinatorics
   3. Numerical analysis/simulation

Appendix 2: Why haven’t I heard of OR?

If OR is so great, why haven’t I heard of it?!

Historical Development  OR was initially developed for wartime operations and logistics during WWII, the first major peacetime applications were for oil production planning, and most modern day practitioners spend their days optimizing the bottom line of large multinationals. In other words, its historical overlap with altruistically minded people has been rather tiny.

Academic Fragmentation  OR goes by many names across academia, with each program having their own unique focus. For instance, some of the top departments in the US:

1. MIT - Operations Research (OR)
2. Stanford - Management Science and Engineering (MSE)
3. Georgia Tech - Industrial and Systems Engineering (ISYE)
4. Cornell - Operations Research and Information Engineering (ORIE)
5. Berkeley - Industrial Engineering and Operations Research (IEOR)
6. Michigan - Industrial and Operations Engineering (IOE)
7. Princeton - Operations Research and Financial Engineering (ORFE)
8. Columbia - Industrial Engineering and Operations Research (IEOR)
9. CMU - Operations Research (OR)
10. UIUC - Industrial & Enterprise Systems Engineering (IESE)
11. UPenn - Operations, Information, and Decisions (OID)
12. Rice - Computational Applied Mathematics and Operations Research (CMOR)

To make matters worse, many schools actually have two OR departments: one in the college of engineering and one in the business school (eg, Operations, Information & Technology at Stanford or The Decision, Risk, and Operations Division at Columbia)

Lacking Ecosystem The machine learning ecosystem is massive and high quality, with great open source libraries, a large community happy to answer questions, and tons of tutorials to get people up to speed. The optimization ecosystem lacks all of these: the best libraries (Gurobi, MOSEK, CPLEX) are all proprietary (free for academics but very expensive for everyone else); the community of people able and inclined to answer stackoverflow questions is quite small; most good tutorials are made by these companies (and are limited in number).

Bad Branding Operations research is not an intuitive name. Many of the core OR tools, like mixed-integer programming are also poorly named. The OR community has traditionally made ties with boring industries, which is good for business and funding but bad for attracting students and for brand image. 

1. ^{^}

   My criteria for what counts as OR (as opposed to CS, econ, etc.) is that the research appeared in an INFORMS journal/conference, was published by OR student/faculty, or used mixed integer optimization techniques.

2. ^{^}

    I am focusing on academic OR both because it is what I understand best, and because it is quite easy for students to change research directions and to do research on more applied problems. However, my hope is that eventually such projects are done within a professional EA consultancy, rather than being one-off PhD projects.

3. ^{^}

    Again with the caveat that I have high uncertainty of the quality of these ideas and of the existing operational sophistication of existing orgs in these spaces; this is mostly intended for illustration purposes.

4. ^{^}

    I think OR is best thought of as a graduate subject and therefore would not recommend it as a primary major. Instead, I would recommend something more foundational like CS, math, or econ (in that order).