All of Alexander Saeri's Comments + Replies

Yes, this was a cheeky or sarcastic comment. I wrote it to share with some colleagues unfamiliar with AI safety who were wondering what it looked like to have 'good' outcomes in AI policy & governance.

Good AI governance is pretty easy.

We just have to

  1. solve a bunch of 2000+ year old moral philosophy questions (e.g., 'what is good', 'what is the right action in a given circumstance', 'what are good rules for action'), then
  2. figure out how to technically implement them into a non-deterministic software / algorithmic form, then
  3. get international agreement on complex systems of regulation and governance to ensure that technical implementation is done correctly and monitored for compliance without compromising values of democracy, right to privacy, free expre
... (read more)
1
yanni
4mo
Upvoting because the forum needs more sass / diversity in tone of voice. Strong upvote because you are making important and clear arguments.
5
NickLaing
4mo
Hey Alexander. I'm genuinely not sure whether you are being sarcastic or not here - intuitively many of these steps seem very difficult to me.

Thanks for writing this up, Emily. I think your decision to do this helped me feel more secure about taking a career break of my own - including some time set aside to do no work or career planning!

1
Emily Grundy
5mo
That makes me very happy to hear, Zan!

I'm glad that Australia has signed this statement.

It's worth noting that until quite recently, the idea of catastrophic misuse or misalignment risks from AI have been dismissed or made fun of in Australian policy discourse. The delegate from Australia, Ed Husic who is Minister for Industry, Science, and Resources, actually wrote an opinion piece in a national newspaper in June 2023 that dismissed concerns about catastrophic risk 

In an August 2023 public Town Hall discussion that formed part of the Australian Government's consultation on 'Safe and Resp... (read more)

Thanks for this comment. I couldn't agree more. 

As you say, the overwhelming perspective in Australia up until a few months ago was in favour of acceleration for economic gain and somewhat interested in near-term ethical concerns. It's probably only a marginal overstatement to say that the political environment was actively hostile to AI safety considerations. 

There was a real risk that Australia was going to 'drag the chain' on any international AI safety work. Anyone who has been involved in international negotiations will know how easily one o... (read more)

Glad that fasting works for you! I have tried it a couple of times and have found myself too hungry or uncomfortable to sleep at the times I need to (eg, a nap in the middle of the flight).

Great points on equipment; I think they are necessary and think that the bulk of a good neck pillow in carry on luggage is justified because I can't sleep without it. I also have some comically ugly and oversized sunglasses that fit over my regular glasses and block light from all sides.

Thanks for the post!

I'm familiar with EGMs in the spaces you mentioned. I can see EGMs being quite useful if the basic ideas in an area are settled enough to agree on outcomes (eg the thing that the interventions are trying to create)

Right now I'm unsure what this would be. That said, I support the use of EGMs for synthesising evidence and pointing out research directions. So it could be useful to construct one or some at the level of "has anyone done this yet?"

Thanks for this guide!

One thing that I appreciated when attending a GWWC event was that expectations of responsible conduct were made clear with an explicit announcement at the beginning of the event. I thought this was a good way to create a social agreement among attendees.

I think that some people are reluctant to do this because they think it might bring the mood down, or it feels awkward to call attention to the possibility of harmful behaviour at what is supposed to be a fun or professional event. They might also not be sure exactly what to say. One idea for addressing these barriers would be to provide a basic script that organisers could say, or rewrite in their own words.

1
GraceAdams
8mo
I actually do have this in a template for slides for events organisers are doing! I've also included in a doc on how to run events that it's important to set the tone for the event, and that this can be a good time to speak about what behaviours you do (and don't) want to see!

Thanks for writing up this work, Zoe. I'm pleased to see a list of explicit recommendations for effective charities to consider in framing their requests for donations.

Selfishly, I'm also pleased that our paper (Saeri et al 2022) turned up in your search!

It's be interesting to understand your motivations for the literature review and what you might do next with these findings / recommendations.

One thing that our paper necessarily didn't do was aggregate from individual studies (it only included systematic reviews and meta-anlayses). So it's interesting to see some of the other effects out there that haven't yet been subject to a review.

I was motivated to write this story for two reasons.

First, I think that there is a lack of clear visual metaphors, stories, or other easy accessible analogies for concepts in AI and its impacts on society. I am often speaking with intelligent non-technical people - including potential users or "micro-regulators" (e.g., organisational policymaker) of AI tools - who have read about AI in the news but don't have good handles on how to think about these tools and how they interact with existing organisational processes or social understandings. 

Second, th... (read more)

Thanks Jacques, I'll need to check this out. Appreciate the pointer and keen to hear more about an LLM layer on this (e.g., identifying action items or summarising key decision points in a meeting, etc). 

I had wondered if it was too hyperbolic to claim that this was an example of proto- or early-PASTA. My earlier draft hedged and said that the next version of these tools would be something like an early PASTA. I would characterise Holden Karnovsky's post introducing PASTA as describing an agentic system that could improve by making copies of itself and improving itself. 

However, when he first introduces the idea of 'explosive' scientific and technological advancement, it's through the thought experiment of creating digital people, which mean that many... (read more)

Thanks for this Jakub! One thing I've seen students / participants ask for is more concrete actions that they could take alongside or after AGISF, so I think this will be a useful resource.

Thanks for the update. 

I'd like to recommend that part of the process review for providing travel grant funding includes consideration of the application process timing for CEA-run or supported events. In my experience, key dates in the process (open, consideration/decision, notification of acceptance, notification of travel grant funding) happen much closer to the date  of the event than other academic or trade conferences. 

For example, in 2022, several Australian EAs I know applied ~90 days in advance of EAG London or EAG SF, but were acce... (read more)

Thanks for your detailed comment! I work on the events team so I can add some info. 
Yep, we broadly agree here. We're keen to open and review applications earlier than we have been doing, for many of the reasons you mention. It's something we've been actively working on for a while, but unfortunately we have been dealing with a variety of bottlenecks on this front. We have designed a new application system for EAG 2023, which will open  very soon. It's an application for all of the EAGs in 2023, rather than on a conference by conference basis. So... (read more)

Based on your comment I looked this up:

Right now flights from London to San Francisco cost £400-£500, compared to what they may be shorter notice (approx £1500+ in some cases). The difference is 2-4x , and you could buy flights + accommodation for a week now (around 2 months out) for less than just the flights may be around 2 weeks out (which is when the EA Global website says you would hear by). This is a significant difference when acting under the assumption of not being able to receive travel grant funding. I can see this in many cases being the differ... (read more)

Thanks Peter! I appreciate the work you've put in to synthesising a large and growing set of activities.

Nicholas Moes and Caroline Jeanmaire wrote a piece, A Map to Navigate AI Governance, which set out Strategy as 'upstream' of typical governance activities. Michael Aird in a shortform post about x-risk policy 'pipelines' also set (macro)strategy upstream of other policy research, development, and advocacy activities.

One thing that could be interesting to explore is the current and ideal relationships between the work groups you describe here. 

For ex... (read more)

3
PeterSlattery
1y
Thanks Alex! I agree that it could be interesting to explore the current and ideal relationships between the work groups. I'd like to see that happen in the future. I think that deferring sounds a bit strong, but I suspect that many workers/organisations in AI governance (and in other work groups) would like strategic insights from people working on AI Strategy and Movement Building. For instance, on questions like:  * What is the AI Safety community's agreement with/enthusiasm for specific visions, organisations and research agendas? * What are the key disagreements between competing visions for AI risk mitigation and the practical implications? * Which outcomes are good metrics to optimise for? * Who is doing/planning what in relevant domains, and what are the practical implications for a subset of workers/organisations plans? With that said, I don't really have any well-formed opinions about how things should work just yet!

Thanks for the plausible explanation!

Re: adding images to your post, I literally just copy and paste. But you could also read a longer post on how to enable advanced editing features such as tables and images.

3
DAOMaximalist
1y
Hey thanks a lot for the image tip and the link. It solved my problem. Just in case anyone else is wondering how to include images in their posts, you can just drag and drop the image from your device into the post editor directly. 

Thanks for pointing this out, Peter. As I understand it, you found this by searching for "effective altruism" and then sorting by date, not relevance.

I did not see any results for "less wrong"

But I did see similar results to your observation for "alignment forum"

5
DAOMaximalist
1y
"Less wrong" is a broad term that could be read in various contexts outside of EA so there are likely other webpages which Google sees as more relevant for this term (more relevant than lesswrong.com ) It seems you are trying to get results from lesswrong.com that are indexed in Google scholar. If that is correct, I would suggest that you use the following search >   site:lesswrong.com

Thanks for this detailed write-up, Ninell. I'll be applying several of the principles for organisation and roles to a version of AGISF I'm facilitating in Australia in late 2022.

1
nell
2y
Thank you and all the best with the fellowship!

This was great fun, and I enjoyed contributing to it!

I'm really excited to see this survey idea getting developed. Congratulations to the Rethink team on securing funding for this! 

A few questions on design, content and purpose:

  • Who are the users for this survey, how will they be involved with the design, and how will findings be communicated with them?
    • In previous living / repeated survey work that I've done (SCRUB COVID-19), having research users involved in the design was crucial for it to influence their decision-making. This also got complex when the survey became successful and there were different
... (read more)
2
David_Moss
2y
Thanks Alexander! I appreciate the offer to meet to talk about your experiences, that sounds very useful! We envisage the main users of the survey being EA orgs and decision-makers. We’ve already been in touch with some of the main groups and will reach out to some key ones to co-ordinate again now that we’ve formally announced. That said, we’re also keen to receive suggestions and requests from a broader set of stakeholders in the community (hence this announcement).  The exact composition of the survey, in terms of serving different users, will depend on how many priority requests we get from different groups, so we’ll be working that out over the course of the next month as different groups make requests. Related to the above, we don’t know exactly how much we’ll be making public, because we don’t know how much of the survey will be part of the core public tracker vs bespoke requests from particular decision makers (which may or may not be private/confidential). That said, I’m optimistic we’ll be able to make a large amount public (or shared with relevant researchers) regarding the core tracker (e.g. for things we are reporting publicly). We’re essentially trialing this for 12 months, to see how useful it is and how much demand there seems to be for it, after which, if all goes well, we would be looking to continue and/or expand.  The monthly cadence is influenced by multiple considerations. One is that, ideally, we would be able to detect changes over relatively short time-scales (e.g. in response to media coverage), and part of this trial will be to identify what is feasible and useful. Another consideration is that running more surveys within the time span will allow us to include more ad hoc time sensitive requests by orgs (i.e. things they want to know within a given month, rather than things we are tracking across time). I think it’s definitely quite plausible we might switch to a different cadence later, perhaps due to resource constraints (including

Thanks for this Sean! I think work like this is exceptionally useful as introductory information for busy people who are likely to pattern match "advanced AI" to "terminator" or "beyond time horizon".

One piece of feedback I'll offer is to encourage you to consider whether it's possible to link narrow AI ethics concerns to AGI alignment in a way that your last point, "there is work that can be done" shows how current efforts to address narrow AI issues can be linked to AGI. This is especially relevant for governance. This could help people understand why it's important to address AGI issues now, rather than waiting until narrow AI ethics is "fixed" (a misperception I've seen a few times).

Great to see this initiative, Vael. I can think of several project ideas that could use this dataset of interviews with researchers.

Edit:

  1. Analyse transcripts to identify behaviours that researchers and you describe as safe or unsafe, and identify influences on those behaviours (this would need validation with follow up work). Outcome would be an initial answer to the concrete question "who needs to do what differently to improve AI safety in research, and how"

  2. Use the actors identified in the interviews to create a system/actor map to help understand f

... (read more)
1
Vael Gates
2y
I think my data has insights about 3, and not about 1 and 2! You can take a look at https://www.lesswrong.com/posts/LfHWhcfK92qh2nwku/transcripts-of-interviews-with-ai-researchers to see what 11 interviews look like; I think it'd have to be designed differently to get info on 1 or 2. 

Thanks for cross posting this Peter. I may be biased but I think this is a great initiative.

Do you have any info about the kinds of people who are reading a newsletter like this? Eg, are they mostly EAs who are interested in behaviour science, or mostly behaviour scientists who are interested in EA?

Akhil, thanks for this post. Your post happened to coincide with an email I received about a new article and associated webinar, "Centring Equity in Collective Impact". You and others in this space might find it relevant:

I use Feedly to follow several RSS feeds, including everything from the EA forum, LessWrong, etc. This lets me read more EA-adjacent/aligned content than if I visited each website infrequently because Feedly has an easy to use app on my phone.  

Here is a screenshot on browser of my Feedly sidebar. (I almost never use a browser)
Here is an example of the Feedly 'firehose' from my mobile phone, previewing several posts from EA forum and elsewhere.

 

I liken it to a 'fire hose' in that I get everything, including all the personal blogs and low-effort c... (read more)

4
Nathan Young
3y
This is exactly my point. Imagine you could customise your RSS feed like you do your front page.

Thanks for this list. Your EA group link for Focusmate just goes to the generic dashboard. Do you have an updated link you can share?

2
Michelle_Hutchinson
3y
Sorry about that. Here's the link: https://www.focusmate.com/signup/EffectiveAltruism (I'll fix it in the article)

If you're comfortable sharing these resources on prioritisation and coordination, please also let me know about them.

3
EdoArad
3y
We plan to post it publicly in a couple of months, but I'll send you privately what we have now :)

I'm a researcher based in Australia and have some experience working with open/meta science. Happy to talk this through with you if helpful, precommitting to not take any of your money.

Quick answers, most of which are not one off, donation target ideas but instead would require a fair amount of setup and maintenance.

  • $250,000 would be enough to support a program for disseminating open / meta science practices in Australian graduate students (within a broad discipline), if you had a trusted person to administrate it.

  • you could have a prize for best ope

... (read more)
3
gavintaylor
3y
I joined a few sessions at the AIMOS (Association for Interdisciplinary Metascience and Open Science) conference a few weeks ago. It was great and I wrote up some notes about the talks I caught here. That said, beyond hosting their annual conference, I'm not really sure what other plans AIMOS has. If it's of interest I can put the OP in touch with the incoming 2021 president (Jason Chin from USyd Law School) to talk further. Otherwise, many of the speakers were from Australia and you might find other ideas for local donation recipients on the AIMOS program. Paul Glasziou from Bond Uni mentioned something in his plenary that stood out to me - inefficient ethical reviews can be a huge source of wasted research time and money (to the tune of $160 million per annum in Australia) - if that's of interest he may be able to suggest a way to spend the money to push for ethical review reforms in Australia.

Friendly suggestions: expand CHAI in the first instance of a post, for readers who are not as familiar with the acronym; clarify the month and day (eg Nov 11) for readers outside the United States

Thanks Markus.

I read the US public opinion on AI report with interest, and thought to replicate this in Australia. Do you think having local primary data is relevant for influence?

Do you think the marginal value lies in primary social science research or in aggregation and synthesis (eg rapid and limited systematic review) of existing research on public attitudes and support for general purpose / transformative technologies?

4
MarkusAnderljung
4y
Thanks Alexander. Would be interested to hear how that project proceeds. I think having more data on public opinion on AI will be useful primarily for understanding the "strategic landscape". In scenarios where AI doesn't look radically different from other tech, it seems likely that the public will be a powerful actor in AI governance. The public was a powerful actor in the history of e.g. nuclear power, nuclear weapons, GMOs, and perhaps the industrial revolution not happening sooner (The Technology Trap makes this argument). Understanding the public's views is therefore important to understanding how AI governance will go. It also seems important to understand how one can shape or use public opinion for the better, though I'm pessimistic about that being a high leverage opportunity. Following on from the above, I think the answer is yes. I'd be particularly keen for this work to try to answer some counterfactual history questions: What would need to have been different for GMO/nuclear to have been more accepted? Was it possible to see the public's resistance in advance?

I really appreciate this, Michelle. I'm glad to see this kind of piece on the EA forum.

If you haven't already, please upload a version to the open science framework as a preprint: https://osf.io/preprints

3
Davidmanheim
4y
I have uploaded it to preprints.org, linked above, pending the final layout and publication. (With an open source license in both cases.)

Thanks for posting this, Nick. I'm interested in how you plan to run this course. Are you the course coordinator? Is there an academic advisor? Who are the intended guest lecturers and how would they work? Who are the intended students?

1
nickwhitaker
4y
Hi Alexander, We have a program called GISP that allows students to run their own course, essentially a group independent study. It should be able to count for an elective philosophy credit too. There is an academic advisor, a professor in our medical school who has been involved in EA. We've been having a lecture series this semester of EA people in the Boston area. We have a lot of students on our mailing list that we've met while tabling that are either vaguely familiar with EA or expressed interest in learning more about it.

Michael, thanks for this post. I have been following the discussion about INT and prioritisation frameworks with interest.

Exactly how should I apply the revised framework you suggest? There are a number of equations, discussions of definitions and circularities in this post, but a (hypothetical?) worked example would be very useful.

1
Michael_Wiebe
4y
Yes, the difficult part is applying the ITC framework in practice; I don't have any special insight there. But the goal is to estimate importance and the tractability function for different causes. You can see how 80k tries to rank causes here.

Very valuable piece, and likely worth a separate write up.

Jeff, this is really lovely and I appreciate you thinking out loud through your reasoning. Is be interested to hear what you think will be hard for them as they grow up with "parts with strong unusual views" and whether you think this would be qualitatively different from other unusual views (eg strongly religious, military family, etc)

One way we try to make it easier is by making it clear that the children can make personal choices about things like donation, diet, and eventually career. E.g. we have the full range from vegan to meat-eaters in our house, and when Lily decided she wanted to be vegetarian for a while we said "It's your choice."

I can imagine having conflict later about her wanting to use the money we donate differently (for spending on "extras" or for donating to something we don't think is effective). But I don't expect it to be worse than the conflict parents and children typically have about money.

Thanks for this excellent piece, Karolina. In my work (research enterprise working w/ government and large orgs), we are constantly trying to get clarity on the implicit theory of change that underpins the organisation, or individual projects. In my experience, the association of ToC with large international development projects has meant that some organisations see them as too mainstream/stodgy/not relevant to their exciting new initiative. But for-profit businesses live or die on their ToC (aka business model), regardless of whether they are large or small.