This is a linkpost for Ten Commandments for Aspiring Superforecasters by Good Judgment[1]. I think the lessons also apply outside of explicit forecasting.

In Superforecasting: The Art and Science of Prediction, Good Judgment co-founder Philip Tetlock and his co-author Dan Gardner summarize the Good Judgment Project research findings in the form of “Ten Commandments for Aspiring Superforecasters.” These commandments describe behaviors that have been “experimentally demonstrated to boost [forecasting] accuracy.” You can learn more about these commandments—and practice applying them under the guidance of professional Superforecasters—at one of Good Judgment’s training workshops.

Here are the 10 commandments:

  1. Triage. "Focus on questions where your hard work is likely to pay off".
  2. Break seemingly intractable problems into tractable sub-problems.
  3. Strike the right balance between inside and outside views.
  4. Strike the right balance between under- and overreacting to evidence.
  5. Look for the clashing causal forces at work in each problem.
  6. Strive to distinguish as many degrees of doubt as the problem permits but no more.
  7. Strike the right balance between under- and overconfidence, between prudence and decisiveness.
  8. Look for the errors behind your mistakes but beware of rearview-mirror hindsight biases.
  9. Bring out the best in others and let others bring out the best in you.
  10. Master the error-balancing bicycle. "Implementing each commandment requires balancing opposing errors".

Last but not least, there is the “11th commandment”:

“It is impossible to lay down binding rules,” Helmuth von Moltke warned, “because two cases will never be exactly the same.”

  1. ^

    I wanted this to be a crosspost, but it was not possible due to copyright restrictions.

13

0
0

Reactions

0
0
Comments2


Sorted by Click to highlight new comments since:

I think this leaves out what is perhaps the most important step in making a quality forecast, which is to consider the baserates!

Nice point, Mathias! I agree reference class forecasting is super important. I think it is supposed to be included in the 3rd commandment about the inside and outside view:

Superforecasters know that there is nothing new under the sun. Nothing is 100% “unique.” Language purists be damned: uniqueness is a matter of degree. So Superforecasters conduct creative searches for comparison classes even for seemingly unique events, such as the outcome of a hunt for a high-profile terrorist (Joseph Kony) or the standoff between a new socialist government in Athens and Greece’s creditors. Superforecasters are in the habit of posing the outside-view question: How often do things of this sort happen in situations of this sort?

Curated and popular this week
JamesÖz
 ·  · 3m read
 · 
Why it’s important to fill out this consultation The UK Government is currently consulting on allowing insects to be fed to chickens and pigs. This is worrying as the government explicitly says changes would “enable investment in the insect protein sector”. Given the likely sentience of insects (see this summary of recent research), and that median predictions estimate that 3.9 trillion insects will be killed annually by 2030, we think it’s crucial to try to limit this huge source of animal suffering.  Overview * Link to complete the consultation: HERE. You can see the context of the consultation here. * How long it takes to fill it out: 5-10 minutes (5 questions total with only 1 of them requiring a written answer) * Deadline to respond: April 1st 2025 * What else you can do: Share the consultation document far and wide!  * You can use the UK Voters for Animals GPT to help draft your responses. * If you want to hear about other high-impact ways to use your political voice to help animals, sign up for the UK Voters for Animals newsletter. There is an option to be contacted only for very time-sensitive opportunities like this one, which we expect will happen less than 6 times a year. See guidance on submitting in a Google Doc Questions and suggested responses: It is helpful to have a lot of variation between responses. As such, please feel free to add your own reasoning for your responses or, in addition to animal welfare reasons for opposing insects as feed, include non-animal welfare reasons e.g., health implications, concerns about farming intensification, or the climate implications of using insects for feed.    Question 7 on the consultation: Do you agree with allowing poultry processed animal protein in porcine feed?  Suggested response: No (up to you if you want to elaborate further).  We think it’s useful to say no to all questions in the consultation, particularly as changing these rules means that meat producers can make more profit from sel
 ·  · 2m read
 · 
In the U.S., egg prices have recently skyrocketed. The text below is copied and pasted from a proposition I produced about an idea to replace eggs with plant-based alternatives in New York City businesses that sell baked goods. I'm posting here for 1) general feedback 2) potential pitfalls/oversights 3) brainstorming 4) contact from people who want to either help me with this or take over the idea entirely 5) to inspire similar projects in other U.S. cities.   VALUE PROPOSITION The current U.S. climate with egg prices and bird flu present a unique opportunity to promote the use of plant-based methods for binding and emulsification in baking. Small businesses hurt by the increased costs may currently be more receptive to alternate recipes for baked goods. Following a similar model to Plantega [an NYC org that has resulted in menus of veganized offerings at local delis], PlantBake could offer curated information and support to help businesses transition to egg-free baking with minimal disruption to their daily routine and product quality. Each success can be modular, low-barrier, and incremental; transitioning even one business to egg-free baking can be of substantial value to plant-based advocates.   FORMAT materials   Volunteer baking consultants (one already identified) work to put together clear, concise, and simple materials on transitioning common baked goods to eggless recipes. A website, pamphlets, and short videos are produced by a promotional media consultant (already identified). candidates   A running spreadsheet of potential candidate businesses in NYC will be populated. PlantBake will aggressively pivot and update the list as we determine which types of businesses are the most receptive. outreach   The candidates will be approached by phone, email, and in person with the value proposition, resources, and contact information. Businesses who request more information will be offered additional assistance (phone or in-person consultation) as resources
 ·  · 38m read
 · 
Summary Kidney stones cause an immense amount of suffering worldwide. Weighted by their intensity, and depending on some assumptions, we posit that kidney stones may account for a large fraction of human suffering worldwide.[1] Finding cost-effective solutions for this hedonic catastrophe should be a very high priority in any suffering-focused research agenda. While researching kidney stone pain, we were very impressed with the online reports of patients using chanca piedra (Phyllanthus niruri) to treat their pain and prevent the formation of stones, so we decided to take a closer look. Given the limited number of high-quality studies on the efficacy of chanca piedra, we decided to look for additional evidence online and collected thousands of WebMD, Amazon, and Reddit reviews about chanca piedra and other kidney stone treatments, and classified them along a few dozen different dimensions (such as reported side effects, reported pain reduction, etc.). While we acknowledge that internet reviews are not usually a reliable source of information concerning medical matters, we also make the case that the reviews of chanca piedra in particular should not be dismissed out of hand. In particular, we find that chanca piedra: 1. Is effective for an exceptionally high percentage of people, judging by both the star ratings and review contents. 2. Has an exceptionally low rate of reported side effects. 3. Has high-quality reviews, judged by criteria such as providing adequate context and believable narratives consistent with kidney stone experiences. 4. Is supported by reviews that are equally very positive across WebMD, Amazon, and Reddit. 5. Compares very favorably to the WebMD reviews of standard prescription medications for kidney stones, including: tamsulosin (aka Flomax), allopurinol, potassium citrate, and hydrochlorothiazide. 6. Has a far lower rate of reported side effects and a higher rate of effectiveness than melatonin and ashwagandha, two widely used over-t