All of Algo_Law's Comments + Replies

"I think the second view is basically correct for policy in general, although I don't have a strong view yet of how it applies to AI governance specifically. One thing that's become clear to me as I've gotten more involved in institution-focused work and research is that large governments and other similarly impactful organizations are huge, sprawling social organisms, such that I think EAs simultaneously underestimate and overestimate the amount of influence that's possible in those settings."

This is a problem I've spoken often about, and I'm curren... (read more)

Good post, thank you.

"or other such nonsense that advocates never taking on risks even when the benefits clearly dominate"

An important point to note here - the people who suffer the risks and the people who reap the benefits are very rarely the same group. Deciding to use an unsafe AI system (whether presently or in the far future) using a risks/benefits analysis goes wrong so often because one man's risk is another's benefit.

Example: The risk of lung damage from traditional coal mining compared to the industrial value of the coal is a very different risk/reward analysis for the miner and the mine owner. Same with AI.

This would be an interesting approach to generate charitable donations. I would caution though that this seems to me (a non-USA person, so take this with a grain of salt) to skirt a little close to some laws surrounding this so I'd definitely check that out first. One man's charitable fundraising could be another man's false representation!

Still though, interesting thought :)

This is helpful. My entire career revolves around "conceal, but don't mislead" and even I'm still learning where lines are. Thank you for this post.

This was a great event which I followed very closely indeed. It generated so much interesting exploration of different areas and I learned so much about the world I live in.

I just want to add that the idea of having 'good-faith' submission prizes was a fantastic addition, and really helped level the playing field for people who otherwise might not have been able to contribute.  I heard from a couple of people that they may not have been able to submit without them. I'd love to see more of these in similar contests in future.

You raise some fair points, but some others I would disagree with. I would say that just because there isn't a popular argument that AGI risk affects underpriviliged people the most, doesn't make it not true.  I can't think of a transformative technology in human history that didn't impact people more the lower down the social strata you go, and AI thus far has not only followed this trend but greatly exaccerbated it. Current AI harms are overwhelmingly targetted towards these groups. I can't think of any reason why much more powerful AI such as AGI w... (read more)

Good points. An important point to bear in mind though is that once again well-roundedness, volunteer work, hobbies etc are all related to factors apart from motivation/ability. Generally, people from wealthier backgrounds have much more of these on their resume than people from poorer backgrounds because they could afford to take part in the hobbies, could afford to work for free, etc. Lots of supposedly 'academic filtering' is actually just socioeconomic filtering with extra steps.

Great post! I'm gonna throw out two spicy takes.

Firstly, I don't think it's so much that people don't care about AI Safety, I think it's largely who cares about a threat is highly related to who it affects. Natural disasters etc affect everyone relatively (though not exactly) equally, whereas AI harms overwhelmingly affect the underpriviliged and vulnerable. People who are vastly underrepresented in both EA and in wider STEM/academia, who are less able to collate and utilise resources, who are less able to raise alarms. As a result, AI Safety is a field wh... (read more)

This would be very helpful. It's often confusing for the applicant as they have no idea what to change/work on. For me, I've been rejected by every EA fellowship I've ever applied for (woop woop, highscore) but I don't know how to improve. Twice orgs have legit emailed me out the blue saying they like my blog/forum content and asking me to apply and then rejected me. I have no idea what stage I failed at. Was my application poorly written? Were my research suggestions poor? Is it a CV issue? Am I over or under qualified? Who knows. Certainly not me. So I'm... (read more)

The audience section is a wide scope which is useful, but just to confirm - would doing an explainer for AI Safety topics/issues affecting other areas with those as the audience count?

Eg.  Producing content on AI Safety risks in economics for economists? Or producing content on AI Safety topics in medicine for doctors?

I think you're right in your third paragraph. I lead a small group far outside of a 'hub' and I'd find this really useful as being immersed in a fully EA environment for some time. It wouldn't so much being a case of pulling me FT to Prague, but more a chance to spend time in an EA environment that is different from my home one. That's largely what was behind my own application, anyway.

No problem RE timescale of reply! Thank you for such a detailed and thoughtful one :)

I really enjoyed this post, thank you for writing it. I'm commenting from an AI law and policy centric view, so this comment is mainly aimed at that angle.

I agree with much of your post, but I want to highlight that there is a need for social scientists in some areas of AI Safety research. I have worked on a few projects for the UK government around AI Safety, helping to build legal, regulatory, and mitigation strategies in the AI Safety field. This is often part of an interdisciplinary team. A few of us are usually sociologists which, with me having a mix... (read more)

Preface: There is every possibility that I have misinterpreted or misread the post. Please do let me know if this is the case and I will rescind this comment.

This is an interesting post, thank you for making it. It must have taken a lot of effort, and it's always a bold move putting culture and community related thought pieces out there because they're often not 'safe' topics. Though I enjoyed many of your past posts, I don't agree with this one for a couple of reasons.

Firstly, I think the term 'elitism' is used too broadly in some areas. Eg. sometimes you... (read more)

I've purposely built my engagement with EA in line with the principles you wrote:

1. Social - have non-EA friends. Ideally have some be local. Talk about other stuff with them, mostly.
2. Financial - do not rely on EA funding sources for income that you couldn't do without. Don't apply for EA jobs.
3. Emotional - do not have a unidimensional sense of self-worth that boils down to "how am I scoring on a vague, amorphous impact scale as envisioned in EA terms

In this way I think I'm fairly permanently 'out' of EA. But I think I get the best of both worlds like this... (read more)

Finally. An EA fellowship that cannot reject me.  Victory is mine!

Joking side, I really like a lot of the questions here. It's also worth bearing in mind a lot of the categories can overlap with each other which is another big bonus I see. Ideally, would these discussions take place on a single megathread, or across multiple smaller ones? Perhaps stickied topics in each tag?

Yeah, I'm going to agree with JeffreyK here and say that you could definitely pursue philosophy as an interdiscipline. Tbh, PhD is a philosophy degree in every area. My law PhD is technically a philosophy degree.  I'm doing a lot of philosophy in it despite not having a prior interest in that area, because it's required for new knowledge.

If you have an interest area such as law, tech, animal rights, etc, you can always combine that with philosophy. That way you get to be a philosopher whilst also doing something you enjoy doing.

Also, be aware that EA ... (read more)

This is a fantastic and much-needed post with loads of consideration. Really great to see, especially with a small but growing legal and legal-adjacent community in EA.

I did a similar route to this but in the UK, so in the spirit of adding context for those who aren't US-based, I'm going to cover some points in this comment. It's not agreeing or disagreeing with anything you said in your post, but providing a little bit of info from a different area for anyone from there :)

I did my undergrad in AI, formally the BSc (Hons) Computer Science with Artificial I... (read more)

This is a really great, high-quality look at this area. Thank you all so much for writing it, especially in such an easy to read way that doesn't sacrifice any detail.

One bit I really like about this is it addresses a major blind spot in AI safety:

"AGI is not necessary for AI to have long-term impacts. Many long-term impacts we consider could happen with "merely" comprehensive AI services, or plausibly also with non-comprehensive AI services (e.g. Sections 3.2 and 5.2)."

I feel that Section 4 is an area that the current AI Safety research really neglects, w... (read more)

This is really fantastic news, and is a much needed area of work. I did my master's degree in space law and it was such an intruiging area of governance. The current state of space governance and space law is the equivalent of trying to run modern society using only the laws of the Roman Empire. I think if people knew how outdated our international agreements were, and how unsuitable they are for modern space governance (let alone expansion, extraction, and colonisation!), they'd be a lot more panicked about getting space governance right.

Very excited to see the publication of your research agenda.

This is a really interesting piece of research. It is certainly a good omen for access to justice, both directly and indirectly. 

The issue of balancing privacy with transparency is an interesting one, and one I've done a lot of work in within Criminal Justice. It's never an easy decision to make, and I had never considered what good training material it would make for privacy-centred LLMs.

I'm still not completely sold on LFAI, but I agree that this is a promising factor in bringing it from theory to a more experimental basis.

Pre-Warning: Please don't read any of this as a criticism of those people who fit into the super-intelligent,  hard degrees at Oxford etc.  If you're that person, you're awesome, not critical of you, and this comment is directed at exploring the strengths of other paths :) 

Tl;dr at bottom of post for the time-constrained :)

This was a really interesting post to read. I wrote a slightly controversial piece a little while back that highlighted that 'top' Universities like Oxford, Cambridge, Stanford have a lot of class and wealth restrictions s... (read more)

Thank you for such an informative and well-thought-out reply. I appreciate you taking the time :)

I think you raise some good points here, and yes I have personally found getting access to money much easier than with most other orgs. I still do think that there may be an unintentional chilling effect on people from rent-seeker discourse, but I think we can both agree with @Levin that perhaps using a different term related to good and bad faith may be a good avenue to pursue.

All in all I think you do raise really good points both in the original post and in this reply, but do also think it's worth being mindful, as always, of unintended consequences :)

That's a good point, about community organisers being kind of a filter. I like to think I'd know if someone was looking to extract profit. To be honest we usually have the other problem. I've heard a few times before from people they 'dont want to take the p*ss' and I have to convince them it's alright to stay at a 2 star instead of a 1 star! I think the groups function well because it's (in theory for me, never happened yet) possible to tell when someone's shifty. So I agree with that point. 

I do still think though that too much focus on the discours... (read more)

That's an interesting tie-in to the 'burnout' discourse we've been seeing lately that I had not even considered.

It's something I would be willing to write if others wanted to read it, unless the original poster would rather do it.

I'm actually going to reply to my own comment here with the cardinal sin of thinking of another point after hitting 'post', but not wanting to disrupt the flow of the original comment!

I believe there IS a case to be made for teaching organisers how to better spend funds smartly. I have been to larger EA events before where I've thought to myself 'this could have been done at half the price'.  Maybe it's the fact I grew up in an environment where you had to make every penny stretch as far as possible, but it blew me away when another group leader menti... (read more)

I think this is a good guide, and thank you for writing it. I found the bit on how to phrase event advertising particularly helpful.

One thing I would like to elaborate on is the 'rent-seekers' bit. I'm going to say something that disagrees with a lot of the other comments here. I think we need to be careful about how we approach such 'rent-seeking' conversations. This isn't a criticism of what you wrote, as you explained it really well, but more of a trend I've noticed recently in EA discourse and this is a good opportunity to mention it. 

It's importa... (read more)

I face enormous challenges convincing people of this.  Many people don't see, for example, widespread AI-empowered human rights infringements as an 'existential catastrophe' because it doesn't directly kill people, and as a result it falls between the cracks of AI safety definitions - despite being a far more plausable threat than AGI considering it's already happening. Severe curtailments to humanity's potential still firmly count as an existential risk in my opinion.

Awesome, thanks for the link! :)

I've often thought about the idea of paying automated, narrow-AI systems such as warehouse bots or factory robots a wage even though they're not sentient or anything would help with many of the issues ahead of us with increased general automation. As employment goes down (less tax money) and unemployment (voluntary or otherwise) and therefore social welfare goes up, it creates a considerable strain. Paying automated systems a 'wage' which can then be taxed might help alleviate that. It wouldn't be a wage, obviously, more like an ongoing fee for using such ... (read more)

"There's some vague sense that current-day concerns (like algorithmic bias) are not really AI Safety research. Although I've talked to some who think addressing these issues first is key in building towards alignment. 

Now don't go setting me off about this topic! You know what I'm like. Suffice to say I think combatting social issues like algorithmic bias are potentially the only way to realistically begin the alignment process. Build transparency etc.  But that's a conversation for another post :D

Frances, your posts are always so well laid out with just the right amount of ease-of-reading colloquialism and depth of detail. You must teach me this dark art at some point!

As for the content of the post itself, it's funny that recently the two big criticisms of longermism in EA are that EA is too longtermist and that EA isn't longtermist enough!  I've always thought that means it's about right, haha.  You can't keep everyone happy all of the time. 

I'm one of those people you mention who only really interacts with the longtermist side of E... (read more)

No problem!

Absolutely, I can see what you mean. Personal lens counts for a lot and people can run away with ideas of badness. Things are rarely as bad as people criticise them for being, and EA is no different. Yeah it has a few issues here and there but the media and Twitter can often make these issues look far, far worse than they actually are. I can totally see where you're coming from.

I think this idea bears some merit in itself, but it would be a lot more complex in practice I think. Some other replies have covered that and I have nothing useful to add, so I won't. One thing I would say is that we have to acknowledge that some of the criticism you list is pretty genuine. We aren't a perfect community and this does impact our activities. Some examples from what Timnit et al discussed on Twitter:
 

 'EA and longtermism are claimed to be divorced from the lived experience of common people'

The majority  of EA's base is in the ... (read more)

That's a very good point. I still feel there could be more contests, grants, orgs etc in this area but you're right in that there's resources there and there's some serious knowledge at those orgs. Perhaps talent, not funding,  is main bottleneck we need to address. They two may be interrelated to an extent.

It's really frustrating to see so much governance talent at law conferences but very little within EA working on Longtermist issues. I think it's a mixture of a lack of outreach in those industries and the fact EA's reputation has taken a couple of... (read more)

A whole range of things, from elements on the 'fancy spreadsheet' side such as recidivism and predpol to the more complex elements surrounding evidential aspects. I am aware none of these are close to AGI, considering no current AI is given its hyper-specialism, but the point of that paragraph isn't about the AI but how humans and organisations have been shown to use AI (or automation software, if you're more comfortable with that phrase). When the first actual AGI is developed, it is likely to be in a very well-funded lab - a lab likely under control of a... (read more)

Lesson learned, thank you :)

You're right in that those situations aren't impossible, but also governance doesn't have to be an end goal but a process. Even helping to govern current AI efforts will shape the field, much the same kind of attitude regulation has with the nuclear field.

Thanks! I'm not certain of concrete data, as this has been just from my experience interacting with others within EA's field, at EAGs, and checking out resource lists/experiencing the courses.
 

There's one list here for example that shows current AI Safety resources and the spread of major AI works within EA such as books. In addition there is a tool here but the CompSci focus might not be because that field is that way, but because the creator felt CompSci was most relevant, which I acknowledge. 

That said, I was pleasantly surprised that in The A... (read more)

That's a really important point to know, actually. I'm glad you told me that. I was always scared if I went over the limit, it might get revoked or I might get rejected in future if I was 'taking the mick'. It's good to know it's not as strict. I tend to ask for lower amounts because if I get rejected, it's catastrophic, so I'd rather suffer more and raise my odds than risk it. 

I got £500 approved to go to Oxford and London split between them (£300 and £200), but in future I might ask for more.

It's becoming clear in this thread that a lot of the probl... (read more)

Thanks! Yes, I am sure some parts are misinterpreted or just down to my own experience, but tbh EA as an org tries super hard to be inclusive so they're probably working on it. Let me know when you next hit up an EAG and I'll come say hi. My girlfriend is a paramedic student too, so winner winner chicken dinner RE any future medical cost concerns :) She didn't charge me when I broke my ankle that one time, anyway ;)

Sorry to hear about your experience Joe, and thanks Julia for the heads up on procedures for everyone.

Regarding the socioeconomic background community element of EA, I feel the same. I started a blog lately and my first post was a post about my own socioeconomic challenges in EA, as well as some of the socioeconomic bottlenecks we face. It may be interesting to you to see you're not the only one:  https://legal-longtermist.ghost.io/why-eas-talent-bottleneck-is-a-barrier-of-its-own-making/. If it helps, reading this let me know I wasn't the only one ei... (read more)

I had a similar experience. I couldn't afford both nights in a hotel, so I slept on the Megabus on Friday night and chose to stay in a hotel for Saturday night as my most important meetings were Sunday morning and I wanted to be fresh.

Obviously didn't sleep on the bus because it keeps stopping and the chairs are designed not to be slept in - not helped by the fact the bus was apparently being driven by Colin McRae. So I had worked all day Friday, been awake all night (but safe and warm!) on the bus, and then had an entire day of conference on the Saturday.... (read more)

I agree with you, and with John and the OP. I have had exactly the same experience of the Longtermist community pushing away Phase 2 work as you have - particularly in AI Alignment. If it's not purely technical or theoretical lab work then the funding bodies have zero interest in funding it, and the community has barely much more interest than that in discussion. This creates a feedback loop of focus.

For example, there is a potentially very high impact opportunity in the legal sector right now to make a positive impact in  AI Alignment. There are curr... (read more)

This is a good post. At present I'm happy being a public EA, but I recently was recommended for my first public appointment - I wonder if any of my posts will ever be held to me to justify? Perhaps used as evidence of my character, beliefs, or bias. Also, not everything ages well. Look how LGBTQ+ language has changed just in a decade. What is acceptable verbiage today is hateful tomorrow, and vice versa. I just write my comments with that in mind for the moment and try to be the best person I can, but it does indeed mean there are risks attached. It's a co... (read more)

This is a really valuable idea and is certainly an area we should research more heavily. I have some brief thoughts on the 'pros' and some ideas that aren't so much 'cons' as 'areas for further exploration (AFFE)'. The AFFE list will be longer due to the explanation necessary, not because there's more AFFE than Pros :)

Pros:

• Law tends to be quite a precise field, which lends itself to CompSci more than many other areas
• Law (generally) evolves to reflect society's current moral beliefs and values
• Law has a huge focus on 'unintended consequences' which is a big