I am a research scientist working at Improbable.

My research is focused on models and simulations of complex systems, which has a large overlap with Machine Learning.

I am also very interested in AI safety, and AI research in general.


What complexity science and simulation have to offer effective altruism

These links are excellent! I hadn't come across these before, but I am really excited about the idea of using roleplay and table top games as a way of generating insight and getting people to think through problems. It's great to see this being applied to AI scenarios.

What complexity science and simulation have to offer effective altruism

@djbinder Thanks for taking the time to write these comments. No need to worry about being negative, this is exactly the sort of healthy debate that I want to see around this subject.

I think you make a lot of fair points, and it’s great to have these insights from someone with a background in theoretical physics, however I would still disagree slightly on some of them, I will try to explain myself below.

I don’t think the only meaningful definition of complex systems is that they aren’t amenable to mathematical analysis, that is perhaps a feature of them, but not always true. I would say the main hallmark is that there is a surprising level of sophisticated behaviour arising from only apparently simple rules at the level of the individual components that make up that system, and that it can be a challenge to manage and predict such systems. 

It is true that the terms “complexity” and “emergence” are not formally defined, and this maybe means that they end up getting used in an overly broad way. The area of complexity science has also been a bit prone to hype. I myself have felt uncomfortable with the term “emergence” at times, it is maybe still a bit vague for my tastes, however I have landed on the opinion that it is a good way to recognise certain properties of a system and categorise different systems. I agree with Eliezer Yudkowky’s point that it isn’t a sufficient explanation of behaviour, but it is still a relevant aspect of a system to look for, and can shape expectations. The aspiration of complexity science is to provide more formal definitions of these terms. So I do agree that there is more work to do to further refine these terms. However just because these terms can’t be formally or mathematically defined yet, doesn’t mean they have no place in science. This is also true of words like “meaning” and “consciousness”, however these are still important concepts.

I think the main point of disagreement is whether “complexity science” is a useful umbrella term. I agree that plenty of valuable interdisciplinary work applying ideas from physics to social sciences is done without reference to “complexity” or “complex systems”, however by highlighting common themes between these different areas I think complexity science has promoted a lot more interdisciplinary work than would have been done otherwise. With the review paper you linked, I would be surprised if many of the authors of those papers didn’t have some connection to complexity science or SFI at some point. In fact one of the authors is director of a lab called “Center for complex networks and systems research”. Even Steven Strogatz, whose textbook you mentioned, was an external SFI professor for a while! Although it’s true that just because he’s affiliated with them doesn’t mean that complexity science can take credit for all his prior work. Most complexity scientists do not typically mention complexity or emergence much in their published papers, they will just look like rigorous papers in a specific domain. Although the flip side of this is maybe that casts into doubt the utility of these terms, as you argued. But I would say that this framing of the problem (as “complex systems” in different domains having underlying features in common) has helped to motivate and initiate a lot of this work. The area of complexity economics is a great example of this, economics has always borrowed ideas from physics (all the way back to Walrasian equilibrium), however this process had stalled somewhat in the latter half of the 20th century. Complexity science has injected a lot of new and valuable ideas into economics, and I would say this comes from the idea of framing the economy as a complex system, not just because SFI got the right people in the same room together (although that is a necessary part).

Perhaps I am just less optimistic than you about how easy it is to do good interdisciplinary work, and how much of this would happen organically in this area without a dedicated movement towards this. I maintain that complexity science is a good way to encourage researchers to push into problem areas that are less amenable to reductionism or mathematical analysis, since this is often very difficult and risky. 

Anyway the main reason I wanted to write this blog post is not so that EA people go around waxing lyrical with words like “complexity” and “emergence” all the time, but to point to complexity science as an example of a successful interdisciplinary movement, which maybe EA can learn from (even just from a public relations point of view), and also to look at some of the tools from complexity science (eg. ABMs) and suggest that these might be useful. @Venkatesh makes a good point that my main recommendation here is that ABMs may be useful to apply to EA cause areas, so perhaps I should have separated that bit out into a separate forum post.

Which non-EA-funded organisations did well on Covid?

I would add the New England Complex Systems Institute, particularly Yaneer Bar Yam: 

In this article from January 2020, which has aged very well, they were advocating for restrictions on international movement and warning of the effect of superspreader events on estimates of R0. 

Yaneer Bar Yam also started this multidisciplinary effort to tackle covid: 

A bunch of reasons why you might have low energy (or other vague health problems) and what to do about it

Hey Alex, thanks for writing this, loads of useful advice in here that I want to try!

I have had similar (but seemingly milder than yours) problems with low energy, where I just felt very lethargic and drained periodically (about once a month). I would compare it to how you feel in the first day of getting a flu or cold, with low energy and mild muscle aches. I went to the doctor and had a similar story to you, they ran some blood tests and found nothing wrong, and that was it.

The answer: it was almost definitely stress. I was in a management position at work, I think I was kidding myself about the stress because I wasn't working super long hours or anything like that, but the thing that really made it so bad was the constant uncertainty and chaos. I was working at a startup that was going through constant re-organisations and strategy pivots, which really took its toll after a while. It was made much worse by the fact that I was a manager and felt responsible for shielding my team from this. This is all to say that people will have varying levels of resilience to different types of stress, for me uncertainty and being responsible for others is difficult, but I am quite resilient to other situations that a lot of people find stressful (for example tight deadlines or public speaking etc.). 

The solution was to change my role away from being a manager and into an individual contributor role in a research team. It took quite a long time for recovery but it's been about a year and a half now and the situation is much better. It felt like a very difficult decision at the time (because I was really stressed and this was affecting my decision making)  but in retrospect it was a really obvious and great decision! I have also subsequently turned down several opportunities to go back to being a manager.

I also think avoiding my commute because of working remotely during covid has helped. 

I took way longer than it should have for me to realise that stress was the likely culprit. I think I was convinced that there was something "really wrong", like some sort of more medical explanation. I don't think I fully appreciated the affects that stress can have on the body, particularly when it builds up over a very long time. Another lesson is that it can take almost as long to unwind and undo those effects, sometimes a 2 week break will not be enough, it requires a more permanent change in role or lifestyle.

Additionally I found that this book helped change my attitude on some things:  Stress-related Illness: Advice for People Who Give Too Much

Would an EA have directed their career on fixing the subprime mortgage crisis of '07-'08 before it happened?

Hey Venkatesh, I am also really interested in Complexity Science, in fact I am going to publish a blog post on here soon about Complexity Science and how it relates to EA.

I've also read Bookstaber's book, in fact Doyne Farmer has a similar book coming out soon which looks great, you can read the intro here.

I hadn't heard of the Complexity Weekend event but it looks great, will check that out!

Would an EA have directed their career on fixing the subprime mortgage crisis of '07-'08 before it happened?

This is an interesting thought experiment and I like the specific framing of the question.

My initial thoughts are that this clearly would have been a good thing to try to work on, mainly due to the fact that the 2008 financial crisis cost trillions of dollars and arguably also led to lots of bad political developments in the western world (eg. setting the stage for Trumpism). If you buy the Tyler Cowen arguemnts that economic growth is very important for the long term then that bolsters this case. However a caveat would be that due to moral uncertainty it's hard to actually know the long term consequences of such a large event.

Here are some other ways to think about this:


As Ramiro mentions below, very few people were alert to the potential risk before the crisis, so an additional person thinking and advocating for this would have increased the proportion of people thinking about this by a lot.


Even if you had predicted how the crisis would have unfolded and what caused it, could you have actually done anything about it? Would you just ring up the federal reserve and tell them to change their policy? Would anyone have listened to you as a random EA person? This is potentially the biggest problem in my view.

Other benefits - better economic modelling

I am a strong believer in alternative approaches to economic modelling which can take into account things like financial contagion (such as agent-based models), so a potential benefit of working on this type of thing before the crisis is that you might have developed and promoted these techniques further, and these tools could help with other economic problems. In my view this is still a valuable and neglected thing to work on.

Other benefits - reputation

An additional benefit of calling the alarm on the financial crisis before it happened is the reputational benefit. Even if no one listened to you at the time, you would be recognised as one of the few people who "foresaw the crisis", and therefore your opinions might be given more weight, for example the people from The Big Short who are always wheeled out on TV to provide opinions. You could say "hey, I predicted the financial crisis, so maybe you should also listen to me about this other <<insert EA cause area here>> stuff"

Actions to take for a career change towards EA (advice needed)

Hey JP, thanks for your question, here are some questions that may be useful in your search, and may help people other provide you more advice: 

1. Do you have any criteria for what you consider a "job within EA"? There are many types of job which could be considered EA related, from jobs within EA organisations, to jobs which have a large potential impact but are not directly for EA orgs (for example working in certain government departments or private companies). It might be worth reframing how you think about this as "how can I find a job that has the biggest impact", rather than "how can I get an EA job".

2. Do you have a particular cause area that you care a lot about? That may help to focus your search, and help your CV stand out to any potential employers.

3. What would you consider to be your comparative advantage? Since you have worked as an economist and data scientist do you consider these technical skills to be your main strengths? Are you looking for a hands on technical job? Technical skills such as software engineering and data science are always in demand so this is worth bearing in mind.

I would add to this that it's obviously worth checking out 80,000 hours careers advice if you haven't already, they spend a lot more time thinking about this than me!

I wish you the best of luck in your job search!

Opportunity to work in Downing Street and influence UK policy

Regarding AI alignment and existential risk in general, Cummings already has a blog post where he mentions these:

So he is clearly aware and responsive to the these ideas, it would be great to have an EA minded person on his new team to emphasise these.