This is part 3 in my unofficial series on how to run great event sessions. In part one, I ranted about panels and (certain versions of!) group brainstorming, and in part two I strongly recommended microphones. Also on my substack.
Choosing which sessions (or “content”) to run at an event is a difficult task. If you’re an event organizer, part of your job is gathering people into one space, asking them to sit in a room, and then directing their attention to one or two people who will say things that your attendees will hopefully find valuable. Maybe attendees talk to other people in the room too, and maybe they’re not sitting, but this is roughly what goes on.
In some cases, you can make decisions that are likely to lead to things being said that attendees find valuable. If you manage to book a preeminent expert in your academic field who rarely makes public appearances at a conference serving that field, there’s a good chance that attendees will be interested in what the expert has to say.
But in many other cases, the decision about who and what to book is much more challenging than this. In the absence of obvious top-down gaps like the above example, event organizers need to know a great deal about the relevant topic or field or group of people to know who has valuable things to say. That kind of context takes a lot of time to build up, or requires the organizers to be tightly networked with a large group. Even when one has enough context to choose the “right” speakers, the speakers might say things people have already heard, or deliver remarks without saying anything valuable.
My team and I convene the people best-positioned to ensure transformative AI leads to good futures. We invite people whose time we consider to be very valuable (senior AI safety researchers, organization leaders, donors and funders), and we care a lot about making sure that the time they spend at our event is valuable to them.
(Before I get into our approach to sessions, it’s worth emphasizing that we think 1:1 conversations are usually the best way for people to spend time at events. I covered why in another post. At our events, we dedicate more than half of the event time to 1:1 conversations, and make explicit in our communications that this is what attendees should prioritize.)
Content continues to play an important role, however. Talks and discussions can still provide very valuable information, and serve as a launching point for conversations later in the event.
We’ve organized many disappointing sessions, and will no doubt organize many more. But recently, with the help of many iterations and lots of feedback, we’ve found a system for designing event content that I think makes sessions at our events worth our attendees’ time. Here’s what we do:
- First, we choose a few speakers we’re confident will have something useful to say, much like the academic expert example. We review their recent work, get in touch, and, if they agree to speak, we work with them to figure out what topic they should talk about. For a two-day event, we typically book 3–6 sessions in this “traditional” way. This is not the interesting part, but I want to stress that a lot of our most valuable sessions are the result of this process!
- We also ask our attendees to submit written memos. Memos are typically 1–5 pages long, and are a vehicle for whatever the attendee wants to communicate. Memos can make arguments, ask questions, communicate announcements, outline organizational strategies, or summarize recent research. We don’t impose many constraints, and explicitly encourage attendees to submit drafts or paragraph-long summaries if they don’t have time to finish the piece before the event.
We don’t only solicit memos to generate event content; memos do the work of event sessions even without a session happening. They communicate useful information, and do so in a very time-efficient way; it typically takes someone ~10 minutes to read the amount of words it takes someone 20 minutes to say, and I suspect our attendees are faster-than-average readers and can gather the information they need without reading every word. That’s >2x the pace in terms of information consumption. This might be why some of our attendees report that reading memos before or even at the event is one of the best uses of their time.
- Memos also serve as a very useful starting point for later 1:1s and conversations; attendees can spell out their view (and obvious counterpoints) in the memo in detail, and then talk to other attendees about specific disagreements, next steps, or considerations they hadn’t covered.
We review submitted memos and share them with other attendees via a web page. Reading memos helps our attendees build shared context, sync up with the field, and arrive with a clearer sense of what they want to talk about and who to talk about it with. We effectively generate a mini journal or subreddit for each event we run.
To help direct attendees to the most valuable memos, we ask attendees to upvote the memos they’re interested in talking about at the event. Our current system is quite scrappy—we leave a comment on the Google Doc version of the memo, and ask attendees to emoji react to that memo with a thumbs up if they’re interested in talking about it. We then sort the memos on our shared platform by upvotes.
Now, we essentially have a ranked list of topics that attendees want to talk about at the event. We then map out a theory of change for potential sessions, with an eye to figuring out how we can advance the discussion put forward by memos.
For some, we suggest that the memo author uses a session to gather more feedback. We might suggest starting with 10–15 minutes of commenting on the memo, then asking session attendees to vote again on which topics they want to discuss further, and then opening up a discussion.
Others plan a session to make progress on their core ideas. An author who wants ideas for their new project might lead a session where attendees propose and vote on potential projects (largely via Google Doc comments and emoji reactions, of course).
Some are more open-ended discussions; the author might tee up the conversation by highlighting some key points they’re interested in discussing and then a conversation or debate naturally gets going. This is more suitable for smaller groups.
A straightforward Q&A might make sense if a memo provides information but where the talking points merit further detail or discussion. In some cases, the memo has said enough; if it’s an announcement, for example, we tend not to prioritise a session since the information has been conveyed.
We also sometimes opt to ask memo authors to give a talk, but this isn’t often very valuable because they’ve already said what they want to say.
- As well as a theory of change, another essential ingredient we look for when designing a session is a champion of that session. One person (usually the memo author) who personally wants a session to happen, and who is willing to invest time in preparing and ensuring the session is valuable. There might be several memos about a popular topic, and there might be a salient way to progress the conversation, but if there isn’t someone willing to give the time to make sure that conversation goes well, it usually doesn’t get off the ground.
We’re still experimenting with this format, particularly the final stages of designing the session. In several cases, we’ve followed this process, identified a popular topic, spent time designing a session, and still ended up running sessions that weren’t valuable—it’s definitely not foolproof, and the success of the session still largely depends on how well it’s run on the day.
Why does this work? The core mechanism at play is reactivity. This process surfaces the conversations that are actually live for our attendees and then prompts us to design sessions that facilitate and catalyse those conversations. While we pick some winners in advance, we’re largely deferring to the marketplace of ideas that we create among our attendees. This means that, even without deep and up-to-date context on a given field, we can quite quickly identify some of the thinkers and ideas at the frontier of that field and platform them.
Some caveats:
- We’ve found this process to be effective for gatherings of 30–250 people who are already networked with each other and where attendees are in a position to submit memos that advance the conversation.
- It probably isn’t suitable for larger events, events open to the general public, or events where most people haven’t met each other before. I think this system works well for e.g., team retreats.
- Some event organizers will have deep context, or work with experts who are, and will be in a position to know which topics the event should cover. I don’t think this memo format can reliably beat this kind of expertise, but it can be a nice complement.
- I also expect our audience to be familiar with the memo format; in EA/AI safety land, writing scrappy Google Docs and sharing them around for feedback is very normal. This might not be true for other fields (e.g., academia, where the standard for sharing work is much higher, and the process of getting feedback is much more formal and time-intensive).
- This process requires not just significant time investment from the event organizers, but very spiky time investment. Because attendees submit memos close to the event, we typically end up designing the majority of our sessions in the final two weeks. We often add sessions the day before the event begins. We usually have one person focused more than full-time on this process in the final weeks, and I know a lot of event teams don’t have that capacity.
- A related downside is that attendees can only see a partial schedule until right before the event. We usually flag where we’ll add content to the schedule in advance, but attendees won’t know if they want to commit to going until just before. We’ve asked attendees about how they feel about this approach, and they’ve endorsed it.
Acknowledgements: I really want to stress that this is not an idea I came up with. It’s the result of lots of experimentation and feedback, much of which was done by other people. This includes: Sophie Thomson, Isaac Dunn, Amy Labenz and several others. My thanks also to Mihan Borhani, my colleague who leads on our event content and who I’ve been working closely with to refine this model.
