I put together this reading list on AI agents, with a tilt more towards AI governance. I hope it may be of help to others working on this area. If you know of other great resources or have comments on these resources, please say so in the comments!

Top priority papers:

• OpenAI – Practices for Governing Agentic AI Systems
• OpenAI – Research into Agentic AI Systems
• Deepmind – The Ethics of Advanced AI Assistants
• Alan Chan et. al – Visibility into AI Agents (see also)

A bit about the agents that currently exist:

• VisualWebArena: EVALUATING MULTIMODAL AGENTS ON REALISTIC VISUAL WEB TASKS
• WebVoyager : Building an End-to-End Web Agent with Large Multimodal Models
• METR – An update on our general capability evaluations

Good background:

• METR – Autonomous replication threat models
• Evaluating Frontier Models for Dangerous Capabilities
• Introduction to Cooperative AI
• Governing AI Agents
• Request for proposals: benchmarking LLM agents on consequential real-world tasks

Lower priority papers (and a tweet):

• Foundational Challenges in Assuring Alignment and Safety of Large Language Models p31-35
• Regulated Advanced Artificial Agents
• Harms from Increasingly Agentic Algorithmic Systems
• Frontier AI ethics
• https://twitter.com/sebkrier/status/1777081791855186191