Summary 

  • EA researchers would benefit from adopting communication norms from the policy world, which emphasize clarity and ease of reading. 
    • As an AI governance researcher, I’ve gotten a lot of value from using these practices – they help me clarify my ideas and get useful feedback. I appreciate it when colleagues use these practices, too.
    • By contrast, it can be difficult to read and parse some important EA documents, which are written as long blog posts without accessible summaries.
  • Three best practices to adopt:
    • 1. Put your key points up front.
    • 2. Use bullet points or numbered lists.
    • 3. Use headings and bolding to make your document easy to skim.
  • I suggest practicing by writing a summary of an important but not-maximally-accessible piece of work. For an example, see my summary of Gwern’s Scaling Hypothesis post at the end of this post.
  • I recommend Holden Karnofsky’s blog posts, particularly this summary of his blog post series on the “most important century”, as good examples of accessible writing on complex topics.

Policy-sphere writing optimizes for clarity and ease of reading –  two valuable attributes that EA docs sometimes lack

EA researchers would benefit from adopting communication norms from the policy world, which emphasize clarity and ease of reading.

Policy briefs are written for people with little time who need to make decisions quickly, and I find them easy and pleasant to read because they respect my time as a reader. 

Longtermists I know who've been exposed to these norms generally find them valuable, and want them to be more widespread within EA.

In my experience, adopting these norms takes some effort, but is extremely worthwhile on a personal level.

  • It forces me to clarify my thinking – which is actually a really big deal!
    • When I started out as a researcher, I tried to write things in academic-style prose, and I think this made it harder for me to think things through properly. [1]
  • Writing docs this way means it’s easier to get feedback. And the feedback I get is probably more useful–for example, more focused on core ideas, since it’s clearer what those are.
  • It makes my docs easier to reference later on. That’s true for me, and I’m guessing it’s true for my readers as well. When I'm coming back to my doc a month later and trying to remember what I think about topic X, having used executive summary style makes it much easier to get back up to speed.
    • It’s actually fun to check back in on my rough ideas and works in progress, instead of stressful.

I think following these norms is also very good for your readers, both in and out of the EA sphere. 

  • In the EA sphere: Following these norms saves readers valuable time. Although our community tends to select for people who enjoy reading long, dry online texts, not all EAs are like this, and even the ones who are have high opportunity costs. Writing a piece with a good summary and skimmable structure makes it much easier for your readers to make informed decisions about how much to read, what sections to prioritize, and in what depth. Whatever they do end up reading, it’ll be much easier for them to quickly extract useful insights.
  • Outside of the EA sphere: Following these norms could also make external-facing EA writeups more accessible to policymakers, executives, and other senior decision-makers who may lack context on our ideas and the time to get up to speed on them. (Frankly, it also helps policy reports look more legitimate, although I’m guessing most EAs interested in influencing policy have some professional experience and so are already familiar with this style.)

Concrete recommendations

Three core tips

A useful starting point is to adopt the three following practices:

1. Put your key points up front. 

  • Have a summary section at the top of your doc.
  • If a particular section is long or confusing, add a clear & simple summary to the beginning of the section.
  • In both cases, include your key takeaways, not just "in this section I will discuss..".

2. Use bullet points or numbered lists.  

3. Make your document easy to skim. 

  • Bold your key points / key sentences.
  • Have section titles reflect the key idea of the section or the flow of ideas, not just a generic subject. E.g., “X implies Y”, not just “X” or “effects of X”.

Other best practices

Use clear and accessible sentences

  • Avoid jargon where possible
    • If it’s necessary to include specialized terms, define them!
  • Avoid long sentences, especially ones with lots of parentheticals or clauses that make them hard to follow.

Assume your audience is smart, but has limited bandwidth.

  • Imagine a busy reader who is reading this summary on the way to a meeting with the piece’s author. What do you want them to know, so they can ask useful questions and have a productive discussion?
  • Imagine yourself reading this in a month, when the piece is no longer fresh in your mind. What are the few key points you’d most like to remember, and how can you write them so they’ll still make sense to you when you lack context?

Examples

What to improve on

Academic books and papers are expected to have a summary-and-conclusion structure, and sometimes these are really helpful! But in my experience they often do a mediocre job of condensing their key takeaways or helping me figure out how to engage with them further. 

Worse yet, a lot of really valuable texts on AI risk & longtermism are basically long blog posts, which can have a particularly heinous overlap of “interesting and valuable content” plus “lack of a clear summary or easy-to-skim structure.” 

Some specific examples:

  • I like Slate Star Codex, but the man demands a lot of your reading time. Consider e.g. this classic post on parapsychology. It’s ~4000 words, and I bet you could summarize the key ideas in 300. (Some of this is because Scott is aiming to entertain, not just inform -- executive summary style is less useful in that case.)
  • This recent Nature article is pretty readable, but it lacks a concise summary:
  • Gwern’s article about the scaling hypothesis, like most of his work, is super interesting–  but also full of long paragraphs and sentences.
    • It has a summary section, but that summary is wordy and kinda hard to skim. And it doesn’t summarize some important, interesting points, like his musing about different AI labs’ interest in scaling in the Prospects section.

Learning from good executive summaries

The 1 page Research synopsis of this RAND AV safety paper (143 pages!). Good features include:

  • Nested structure
  • Bolding
  • Each sentence is easy to understand

Holden’s summary of his blog post series on the “most important century”. Good features include:

  • Clear, concise summary of complex ideas
  • Short paragraphs
  • “Map” where you can jump in to ideas you want to investigate in greater detail
    • Seems really valuable to do this well; big research outputs can be very hard to orient to.
  • Evocative graphics help ideas stick in your head
    • I like that various longtermist writers have experimented with this – e.g. in Owen Cotton-Barratt's various docs or MIRI’s Embedded Agency sequence. I knew someone at FHI who paid people to make nice versions of his back-of-the-napkin sketches. It could be valuable for more of us to experiment in this direction.

Practice task

For readers who plan to do significant research or policy writing, I’d guess it’s worthwhile to spend 30 minutes practicing the executive summary writing style. Consider taking that time now, or schedule a time on your calendar! I ran a practice session with a team of AI governance researchers earlier this year and they found it quite useful. I think it helps build a sense for why and how to use these techniques.

A practice task I’d suggest: summarize a useful piece of writing. 

  • Pick one of the pieces of writing listed below, or something else that’s interesting to you but is decently long and lacks a summary
  • Try to summarize it in around 10 sentences.
  • Imagine your target audience is researchers focused on the cause area(s) you’re interested in, but that at least some of them lack background knowledge relevant to the specific topic of the piece (e.g., lack much technical AI knowledge).
  • If you think the article emphasizes things differently than you would, or that some parts of it are confusing, you may want to summarize your perspective on the article. That could well be more useful for yourself and your colleagues, after all!

If you’re interested in further practice, you could try summarizing another piece, or play around with these variations:

  • Try condensing your summary down to just 4-5 sentences. What are the core points here?
  • If you were originally summarizing what the article itself said, add a summary of your perspective as well. What are your key takeaways? What are the main questions you’re left with?
  • Pick an important term that people might not know, and add a sentence that introduces it to your reader.

Texts to summarize

Example summary: Gwern’s post on the scaling hypothesis

Meta notes: I focused on re-writing the content in the doc’s summary section in a way that would be more useful to an unfamiliar reader. But I also pulled in some content that doesn’t exist in Gwern’s summary, and included my own takes. The “right” amount of external content and viewpoint to pull in depends on the goal of your summary. Often, I aim to leave readers with something like “the key pieces of my current viewpoint on the key questions addressed by the summarized document.” What that means in any particular case is, naturally, a judgment call!

The performance of large language models, most recently GPT-3,[2]  provides evidence for the scaling hypothesis – the idea that we could get to highly intelligent AI simply by building bigger neural nets.  

  • GPT-3 is the latest in a line of language models which have gotten better mostly by being bigger than their predecessors.
  • It does some impressive things, improving on various benchmarks and demonstrating meta-learning capabilities: it’s able to perform well at lots of tasks it wasn’t explicitly trained on.
  • It does this without being programmed especially cleverly: it’s mostly a much larger version of the basic Transformer architecture, without lots of the improvements that researchers have figured out they could include.


 

If the observed scaling of model performance continues, and orgs decide to invest in bigger language models, we could see highly intelligent AI created on relatively short timelines. 

  • The largest neural nets models still cost relatively little to train, compared to the R&D budgets of large organizations: they cost tens of millions of dollars, while top budgets are in the billions.
    • Gwern thinks it’s unlikely that other orgs build much bigger models: he thinks OpenAI is unusually set on the scaling hypothesis and other orgs are unlikely to buy into it.
      • Since the post was written in late 2020, Google and Microsoft have published bigger models than GPT-3. I think it’s plausible that they get serious about competing with OpenAI to build useful large language models (LLMs), and that this competition eventually leads to many orgs competing to build highly capable AI.
      • That said, I think Gwern’s viewpoint was more plausible at the time of writing, when it had mostly been OpenAI at the frontier of LLM scaling.
  • Just pouring more compute into language models will probably hit diminishing returns eventually, but a) the models might become very capable first, and b) developers might figure out some good tricks that stop diminishing returns. 


 

Thanks to Abi Olvera, Amanda El-Dakhakhni, and Michael Aird for feedback on this piece, and the Rethink Priorities Longtermism Department in general. 

 

For interested readers, some earlier discussions of clear communication in research:

  1. ^

    As an aside: For early-stage writing, I would also recommend being open, coarse, and informal – being willing to say things like “maybe X?” or “I don’t really know where this intuition is coming from.” Being honest about these things with myself is super valuable, and being honest with others helps them give you useful feedback. Many researchers in the EA community are good about this, but I’ve noticed that junior researchers I’ve mentored are sometimes constrained by attempting an overly formal, academic style.

  2. ^

     As of writing time in late 2020. Recently, tech companies have created even bigger and better AI models like Turing Megatron NLG, Gopher, and PaLM. See https://www.aitracker.org/ for a list of large models.

158

New Comment
12 comments, sorted by Click to highlight new comments since: Today at 2:54 PM

Thanks for this post! As you know*:

  • I agree that it'd be really valuable for more EA-aligned people to goddamn write summaries at all, to write them well in the ways this post explains, and to use this writing style in general (not just for the summary sections).
  • I think my own writing is much better as a result of adopting these practices.
  • I think this post is a really valuable resource for advocating for and explaining these tips, and I intend to often share it with people and to add it to my (somewhat messy) doc of Tips & readings on good writing/communication.

*Disclaimer: The author and I interact a lot and our views aren't super independent.

Here are a couple other resources that I like and that people might also find useful on approximately this topic (besides just the "related work" you mention):

And here are my own quick tips on summaries, from Tips & readings on good writing/communication, which are less useful than but somewhat complementary to this post:

  • "You should usually include an actual tl;dr/summary/key takeaways section right near the start - even in most cases where you feel it’s unimportant or inappropriate.
    • (At least when writing for e.g. EAs. Sometimes when writing for mass audiences, you’ll better engage people by deliberately not making it clear what you’re writing about or what you’ll ultimately claim.)
    • See Reasoning Transparency and Using the “executive summary” style: writing that respects your reader’s time
    • Usually don’t skip this section
    • Usually don’t just have a section with that sort of name but where you actually just say “This post will cover x, y, and z”
      • I don’t just want to know you say something about x, y, z; I want to know the core of what you actually say!
    • There’s a good chance you - whoever you are - think “The key takeaways are too complex to be explained briefly before someone has actually read my introduction, how I explain the terms, etc.” You’re probably wrong.
      • I kept thinking this for ~8 months, till finally the many many times I was advised to add summaries got to me and I started really trying to do that, at which point I realised it really was typically possible & valuable.
      • Have you actually spent 5 minutes, by the clock, really trying to summarise the key takeaways in a way that will make sense to a reader who hasn’t read the whole thing?
    • (There are some exceptions, e.g. for extremely short posts)"

it'd be really valuable for more EA-aligned people to goddamn write summaries at all

To get more people to write summaries for long forum posts, we could try adding it to the forum new post submission form? e.g. if the post text is over x words, a small message shows up advising you to add a summary.

Or maybe you're thinking more of other formats, like Google docs?

Yeah, I've actually discussed that idea briefly with the EA Forum team and I think it'd probably be good. I'll send a link to this thread to them to give them one more data point in favor of doing this. (Though it's plausible to me that there's some reason they shouldn't do this which I'm overlooking - I'd trust their bottom-line views here more than mine.)

But yeah, I'm also thinking of GDocs, blog posts posted elsewhere, and any other format, so I think we also need nudges like this post. 

I think the main downside would be that some people might read only the summary and miss out on the reasoning parts, which are often the parts that are actually the most valuable. Or, relatedly, some authors like Scott Alexander do better when people have to read his hook and get sucked into his discursive style.

However, overall I've been sold on this idea, but do think that the UI would need to be done well. We haven't prioritized it yet, but I'm definitely tempted, and I think that's true for other team members as well.

Cool, glad to hear that.

fwiw, I think a good summary can & should also summarise the reasoning, not only the conclusions. (Though of course it'll still miss many details of both the reasoning and the conclusions.)

I'd also flag that I think this should be some sort of nudge, suggestion, optional box to fill in, or whatever, rather than a required box. So people could still decide to not have a summary if they wanted, e.g. if that would mess with their engaging discursive style in a given piece.

I like your document, and I've saved it in my Google keep for future reference. So thanks!

Thanks for normalizing this! I'm trying this for my newest blog post, and I've realized that it's much easier to use a bullet-point summary than to try to come up with a creative intro section. So this will make my writing process much less stressful 😅

In consulting this is also referred to as the "Minto pyramid" or "pyramid principle of communication": you put the key message and main insight concisely as the first step.

In a fast-paced environment, this means the one piece with the highest information density is ensured to be delivered before e.g. getting interrupted. It also enables the listener to decide whether this topic is relevant early on in the communication and therefore shows respect for their attention.

After delivering the key message you can elaborate on the underlying insights or what brought you to the conclusion. If further elaboration is necessary or there is time for it the receiver of the message can ask further questions.

Here is an image to visualize the principle. The related book is called: "The Pyramid Principle: Logic in Writing and Thinking" by Barbara Minto.

Another thing you can do to respect the time of your readers is to think a little about who doesn't need to read your post, and how you can get them to stop. I don't have a lot of advice about what works here, but it seems like a good goal to have.

A trick to let your readers decide what to skip over and what to skip to:

Use titles

+1  not only pointing out that posts should be shorter (as some posts do, ironically, in great length), but also giving concrete advice on how!