Executive Summary

  • On October 23-25 2020, we hosted the inaugural online EA Debate Championship - a three-day debate championship with EA-themed topics.
  • The championship had 150+ participants, from roughly 25 countries, that span 6 continents.
  • The championship was supported by the World Universities Debating Championship, aka WUDC - one of the largest international student-driven events in the world.
  • There were a total of 7 debate rounds - 5 preliminary rounds and 2 knockout rounds. The knockout rounds were held in 2 different language proficiency categories to promote inclusivity. In total over the course of that weekend over 500 EA-related speeches were delivered.
  • The championship featured a Distinguished Lecture Series as non-mandatory preparation material - 9 lectures, 3 debate exercises and 1 Q&A session containing introductory EA materials (totalling ~10 hours), with top EA speakers including Ishaan Guptasarma, Joey Savoie, Karolina Sarek, Kat Woods, Lewis Bollard, Olivia Larsen, Nick Beckstead, and Will MacAskill. The debate exercises were filmed by world-renowned debate teams.
  • The championship included a research component to examine if debating on EA topics changes the stance of debaters towards EA values.
  • Most of the participants were not familiar with EA prior to the competition, or had limited exposure to core EA ideas. However, when asked after the tournament many were highly positive on the prospect of attending a future EA debating championship, and reported a strong willingness to continue their engagement with the EA community.
  • During the tournament, over $2,000 were donated to effective charities by the participants (with most of the funds going to the Against Malaria Foundation). The funds were doubled via donation matching provided by Open Philanthropy.
  • The competition was initiated and organized by members of EA Israel who are also debaters; with the support of several highly influential international debaters and the World Championship. This collaboration was possible due to the strong ties that exist between the debating community and the EA community in Israel. We think that there is room to building similar ties on a more global scale.
  • In the rest of the post we will explain our motivation to run the event, describe the program and its outcomes in detail, share what we have learned from the process, and discuss our next steps.
  • Organizing the tournament was an effort of a great many. We thank them all, and would like to stress that any mistakes or inaccuracies in the description are our own. In particular we would like to thank Adel Ahmed, Ameera Moore, Barbara Batycka, Bosung Baik, Chaerin Lee, Connor O’Brien, Dana Green, Emily Frizell, Enting Lee, Harish Natarajan, Ishaan Guptasarma, Jaeyoung Choi, Jessica Musulin, Joey Savoie, Kallina Basli, Karolina Sarek, Kat Woods, Lewis Bollard, Milos Marajanovic, Mubarrat Wassey, Nick Beckstead, Olivia Larsen, Omer Nevo, Sally Kwon, Salwaa Khan, Seoyoun, Seungyoun Lee, Sharmila Parmanand, Tricia Park, Will MacAskill and Yeaeun Shin for their contributions in running the tournament, filming lectures or creating exercises; to David Moss, David Reinstein and Stefan Schubert for their advice on running the tournament survey; and to the many incredibly qualified debate adjudicators & speakers that made the event possible

Motivation

We initiated this effort due to the impression that themed debating tournaments (along with matching preparation materials) can be a relatively broad yet high-fidelity outreach opportunity. We believe this is the case for several reasons:

  • The international debating community mostly consists of undergraduate students from around 50 countries (elite universities are represented across all continents).
  • Debaters tend to be willing to sit and reflect upon themes for hours, both when preparing for a competition, and when analyzing their performance after each  debate round. Long online discussions are the norm in the community, similar to EAs.
  • As such, a theme-specific championship is a rare opportunity to engage a highly diverse audience in a meaningful way that can spark interest for a long time.
  • To further seize the opportunity we also launched a complimentary Distinguished Lecture Series of videos on core EA topics, and conducted research surveys that questioned the positions of the crowd with regards to the championship’s topics.

We also believe debaters are a particularly promising audience for EA outreach, for the following reasons:

  • Prospective influence: Competitive debaters tend to be overwhelmingly represented in the high echelons of government, academia, media and business. A significant portion of high-ranking politicians in the US and the UK have a debating background. As a recent example of an influential community alum, Jake Sulivan was named the National Security Advisor of the Biden Administration; beyond him there are many other highly influential graduates that are positioned in key global roles in numerous important states. Thus, we think that making the community more EA-aligned presents an opportunity to instill important values that can propel large scale plans in the following decades.
  • Potentially high EA affinity: Competitive debaters tend to look at the world from a neutral perspective and ask what is most beneficial to humanity at-large when discussing a topic. We believe that a willingness to constantly engage with people from all over the world, to seek knowledge on policies, and to have logical discussions are indicators that show that EA values can resonate with the target audience.

In addition to our general view that this type of outreach may be promising, we believed we had a particularly promising opportunity at this time, for several reasons:

  • Collaborating with key members of the world championship and the debating elite to reach a wider audience.
    • There are globally prominent debaters who identify as EAs (e.g. former world champions who are active in local EA branches).
    • In particular, this year presented an opportunity to collaborate with the World Universities Debating Champions (WUDC) - the most prestigious academic competitive debating event in the world. The opportunity existed since the Chief Adjudicator, who is the professional director of the World Championship, is also a part of EA Israel.
    • The opportunity allowed the EA tournament to be a major event on the international debate calendar that attracted a lot of global engagement.
  • The team running the tournament had experience in reaching large audiences with similar past projects.
    • We were able to assemble a team that had the experience of running over 300 debating events, and recruit many key members of the debating community to help.
    • In particular, the team also ran large educational programs before (e.g. a Distinguished Lecture Series on feminism that had a global audience of thousands of dedicated participants).

Finally, we believed such an event could present a unique opportunity for improving the state of EA advocacy research:

  • Outreach is a key component of EA, and yet, the research on the effectiveness of different types of EA advocacy is still limited. We believe this is an important and underinvested research area within EA.
  • Debate, being a structured discussion format where judges and speakers are randomly assigned positions, is a natural ground for research on how engagement with EA ideas can change stances on EA values.

Program Description

The program included three distinct components, which complemented each other. Below we describe each of these components, followed by the total costs of running this program.

Distinguished Lecture Series:

  • The championship featured a Distinguished Lecture Series (DLS) as non-mandatory preparation material. We have recorded 8 lectures, 3 debate exercises and 1 Q&A session.
  • The content focused on main EA causes and centred on: introduction to EA, cause prioritization, global health and development, animal welfare, and existential risk. You can find the links to the lectures at the end of this section.
  • The exercises featured elite debaters that analyzed debate topics using the information from the corresponding lectures. We recorded 3 exercises that followed the topic-specific lectures - on global health and development, animal welfare, and existential risk.
  • Each lecture was between 20 mins to 90 mins at the discretion of the speaker. The lectures combined can serve as a crash course to EA and the materials totaled in ~10 hours.
  • The lectures were given in advance to the participants of the tournament so they can watch them before the beginning of the tournament . This was not mandatory as that is not a custom in debating. However, our analytics suggest that between a third to a half of the participants have watched the lectures to deepen their EA understanding.
  • We later released the lectures to the debating community at large, using the formal World Championship channels, so that more people could benefit from them.
  • The full program and speakers can be found here.

Debate championship:

  • How do debate tournaments work? The championship used the British Parliamentary debate format, which is the standard format of international competitive debating. In the format there are 4 teams, each consisting of 2 people, in any given debate. The teams are arbitrarily allocated to be for, or against, a predetermined topic, such that 2 teams support the topic and 2 teams oppose it. All teams compete with each other, even teams that are “on the same side” (e.g. both support the topic), and are ranked from 1st to 4th. The topics are decided by a committee, and are not known in advance to any of the speakers. When a topic is announced teams have 15 minutes to prepare their case before the debate starts. In the EA championship there were 5 preliminary rounds that were debated by all speakers. After each round teams were assigned scores by their adjudicators per the standard manual. At the end of the preliminary rounds the teams with the highest accumulative scores progressed to knockout rounds that determined the champions. There are separate knockout tracks for teams based on their English proficiency, to promote inclusivity. If you are interested to learn more you can visit this page.
  • How many people attended, and from where? The tournament had over 150 participants. During the course of the weekend 584 speeches on EA-related topics were given. Attendance was truly global - participants attended from roughly 25 countries that span 6 continents. Many top universities (e.g Oxford, Cambridge, Stanford etc.) had attended and sent several members.
  • How did the collaboration with the World Universities Debate Championship (WUDC) work? WUDC collaborated on producing, designing & distributing the Distinguished Lecture Series. The competition undertook similar initiatives in the past centred on other themes, and had the existing expertise and channels to broadcast the series. Additionally, several of the people that manage WUDC were also part of the organizational core of the EA tournament. Such a structure guaranteed that the content could be spread to the entirety of the global debating community and not be limited strictly to the participants of the tournament.
  • What were the debate topics? See Effective Altruism Debating Championship Motions.

Advocacy research:

  • To explore how debaters engage with EA ideas and values we set a research program that ran concurrently with the tournament.
  • Willing participants filled in questionnaires on EA positions before, during, and after the tournament. About a one third of the tournament participated in the research program and received compensation for their time.
  • The amount of data that was available to analyze makes it hard to establish causality and prove statistical significance for our research hypothesis. However, interim analysis suggests that the interaction with the competition, and/or lectures, made the participants change their views on some EA related issues towards the EA position. We have reasons to believe that the debate dynamics played a role in the positions that participants converged to. Some metrics are shared in the next section. However, we would like to stress that we are unsure if we have enough data to prove our hypothesis, and we may need to conduct followup research to do so. The analysis is still a work in progress and we will release our conclusions in a future post (should they be informative).

The costs and funding for the program were as follows:

  • The tournament and the DLS components were done on a voluntary basis.
  • The advocacy research totalled at $4,000 USD paid as incentive/compensation for people filling out the questionnaire.
  • The tournament also had a donation matching scheme. During the course of the tournament debaters donated ~$2,100 to effective charities (mostly to AMF and Givewell). The money was doubled via donation matching.
  • Funding for the research and donation matching was provided by an Open Philanthropy grant.

Engagement & Outcomes

Overall we were pleased with the engagement, response, and follow-ups to the tournament. Below we share some of the easier-to-quantify metrics for engagement and influence:

  • The championship had 150+ participants, from roughly 25 countries, that span 6 continents.
  • Participants were very satisfied with the tournament - there is a strong interest in attending a similar event in the future (an average of over 6, on a scale of 1-7), an interest in engaging with EA materials further (average of 5.85 out of 7), and to advocate EA to others (average of 5.66 out of 7). Note: an analysis of how much of these positive reactions is due to the lectures, the championship, or neither, is still pending. We are looking at our surveys to analyze that and will write a future post about this issue.
  • Hundreds of people from all over the world watched the full ~10 hour lecture series.
  • About 100 people joined the group Effective (Altruism) Debating that was designed to introduce more EA background and content to debaters. Due to a lack of personnel to promote and moderate the group it became less active after the tournament, but we do see the group as an opportunity for continuing engagement.
  • As mentioned ~$4200 were donated in the context of the tournament (50% via participants, 50% via donation matching). The vast majority of the money was donated to AMF and Givewell.
  • Anecdotally, several of the organizers have had multiple debater friends reach out to them for donation advice or with questions about EA in general.
  • Several other major debating tournaments adopted norms of donating proceeds to effective charities. The tournament model was also fondly mentioned in the context of doing debating for good several times. It remains to be seen whether this growing norm sustains in the debating community. However, we believe this shift in perception both has value in and of itself, and is a signal that the tournament was successful in influencing the broader debating community.
  • A few debating programs showed interest in adopting EA content to their curriculums. We have set up a pilot program for seniors in high schools that is based on the materials that we have filmed for the Distinguished Lecture Series and the competition. The pilot includes a small group of students 10-20 and is expected to conclude in the summer of 2021. Should it be a success we have great scale up opportunities - we are in contact with several schools that teach elite debate classes to thousands of teenagers globally.

Lessons Learned 

  • Lesson 1: EA branding. There is a huge gap between what EA is about and what people think is EA about. For example, we heard comments about not wanting to do 5 debates about which charity to donate to. This should definitely be addressed before another such tournament or program is run, but may also be relevant for EA outreach in general. The problem seemed pretty universal, and people’s interest in hearing more about EA seemed to be heavily influenced by its branding.
  • Lesson 2: Increasing the organizational timeline would have been helpful. Our timelines ended up being very short, especially with regards to publishing the lecture series before the tournament (which was done in the week prior). We believe if we had published the materials earlier more people would engage with them. We did start organizing the event 4 months in advance, however, future similar events with multiple components running simultaneously may require more time.
  • Lesson 3: More manpower. Running the EA tournament, Q&A with Will MacAskill, and research survey simultaneously was challenging, even though our team has extensive experience in running debate events. We successfully remained close to the planned schedule throughout the event, but in potential future events it would make sense to recruit a larger team and distribute responsibilities in a clearer fashion to ease organizational burdens.
  • Lesson 4: Time investment and funding. The competition required a lot more time than we initially thought it would. The main cause was the ambitious program involving lectures, research and debating that we developed along the process. Perhaps providing grants to the organizing committee for future similar outreach events to offset the time costs will make sense if we choose to do this again in the future. Such grants would have allowed us to recruit a larger operating team and to potentially brand the tournament better.
  • Lesson 5: It may be helpful to design a formal EA-advocacy framework and research agenda. Debate can be a useful case-study for EA-advocacy for the reasons mentioned in this post.  However, even with the help of fellow EAs, it took us a while to understand how best to measure engagement with EA content. It can be worthwhile to have a work group that tackles these questions specifically. Figuring best practices for such research can be valuable in reaching insights on what works, and what does not, helping the community grow in relevant target audiences.

Conclusions

  • Conclusion 1:  The event was effective at positively influencing highly-engaged participants. We learned valuable lessons on how to engage crowds with EA-content, and the championship seemed to spark genuine interest among some members of the community.  Overall, we feel we managed to achieve deep and positive engagement with tournament participants, but haven’t yet cracked the question of how to properly engage (more shallowly) with the broader debating community. It is also clear from the outcomes that these kinds of events are much more effective at outreach and advocacy than direct fundraising among the target audience (which makes sense given that the audience is students).
  • Conclusion 2: The debate community is a good “fit” for EA. A significant portion of participants we talked to found EA principles obvious and appealing once properly exposed to them. We don’t think this is unique to this type of outreach - we have seen similar results in other circumstances when communities with good “EA fit” are exposed to EA content. The participants showed a very high desire to attend similar events in the future, support EA actions, and promote EA values. Anecdotally, some major debate tournaments (after this project) referred participants to donate to effective charities, and people continue to engage with the recorded DLS lectures. These outcomes, the relatively low cost of running debate events, and the high probability that within a decade or so we can expect members of this audience to be in global positions of influence makes the community a great outreach target.
  • Conclusion 3: There is an appetite for more of these events. As mentioned, participants showed a high willingness to attend similar events in the future. Many top universities have both debate societies and EA societies, so perhaps promoting collaborations between them on a local level is worthwhile. An approach to promoting such a collaboration may be to build a formal EA crash course for debaters that can be circulated globally.  We recommend exploring these options, and are happy to assist in such efforts.

Next Steps

  • The analysis of the survey results is still ongoing. We expect to reach results and share them (if they are valuable) later in the year.
  • As mentioned we are looking into the prospect of turning the lecture series into an EA & Debate crash course for schools debating. We are starting a pilot program with a small group of high school seniors that is expected to conclude in the summer of 2021. The materials are based on the Distinguished Lecture Series and other content we have gathered for the competition. We believe this direction has potential, because if done well it can scale to thousands of schools globally - and some school networks have already expressed interest in collaborating with us. When the pilot is concluded in a few months we intend to evaluate the results, and decide if we should pursue a scale up (based on how effective we find the program and whether we have capacity to grow it).
  • We are considering running such a tournament annually. However, we are unsure we have the available resources to pull this off again, at least as of yet. Overall there is interest from both the organizers and the community, but we will probably make a final decision only after all efforts around this instance are complete.
  • We continue to distribute EA content on the relevant debate channels. In particular, we found dedicated Facebook groups to be a good medium to increase engagement (e.g. the Effective (Altruism) Debating group), but currently we do not have anyone with the time and/or social media skills to keep that going. If you’re interested in volunteering to do so, please contact us.

Final Details & Discussion

  • In order to reach us please feel free to write us here, via LinkedIn (Dan; Sella), or via email: danlahav (at) gmail (dot) com, sellanevo (at) gmail (dot) com.
  • We’d love to hear your thoughts and feedback.
  • If you have extensive background in running outreach programs, in the educational spheres, or in video editing and production, we may also be looking for volunteers to help us to run the EA-schools program (should the pilot be a success).
  • If you are considering running similar outreach events, or if you also want to promote EA-debate relations, feel free to also approach us for advice.

135

47 comments, sorted by Highlighting new comments since Today at 9:57 AM
New Comment

This seems great to me, please do more.

I... feel pretty conflicted and hesitant about this. Overall, I have a sense that debate wasn't a healthy sport for me, and is unlikely to be healthy for others. And I guess I don't really want us to end up adopting much of its culture,  or use the techniques used in competitive debating to convince people. So I am not sure whether I think this is a good idea. 

As long as we are all just treating debate as a game that has as much relationship to figuring out the truth as playing soccer, I am of course OK with that, but I am pretty concerned that it's actually pretty hard to maintain that relationship to the sport at an event like this. 

I used to participate in debate in high school and think it taught me a lot of really bad epistemic habits that took quite a while to unlearn, and most of the American debate tradition seems even more broken than what I was used to in Germany. 

I think it's hard to understate how far away the common practices of competitive debate are from actually doing anything that helps you better understand how the world works, or has much to do with truth-seeking. And at the same time, my experience of the debate community is that they do think that they are learning valuable skills that help them better navigate the world and communicate to each other. 

Overall, my sense is that it's very hard to organize an event like this and actually have the narrative of "To be clear, please don't ever actually talk to other people in EA the way you talk to other people on stage here. I would consider that quite rude and probably actively harmful, and also, I really don't think the sport we are practicing here is helping you understand the world better, please don't take this seriously". 

Like, I would be really surprised if anything like that was said at the opening talk of this event, but like, I do think that is the actually right attitude to have towards debate if you don't want it to hurt you. I've talked to a bunch of people in the EA community who have a debating background, and all of the ones I've talked to thought that overall the habits they learned were probably bad, and the whole process really didn't have much to do with truth-seeking. 

For people who are unfamiliar with the degree to which various competitive debating practices have created to me quite horrifying abominations, here is an example of a very high-level competitive debate: 

I do think British Parliamentary Debate style is a bit less broken than this, but like, not that much. I think overall, the sport really doesn't have much to do with even just real and normal political debate, which is already a bad thing to imitate. 

This again, doesn't make me totally confident that the above is a bad project, but I do sure feel like I would warn people against attending events like this, and am pretty worried about adopting more of the competitive debating culture with things like this, and also don't super think that given how far the sport has deteriorated, that skill in it is really predictive at all in truth-seeking ability, after you control for g (and my guess is negatively correlated after you control for g, though I am less confident of that).

I think these concerns are all pretty reasonable, but also strongly discordant with my personal experience, so I figured it would help third parties if I explained the key insights/skills I think I learned or were strongly reinforced by my debating experience. 

Three notable caveats on that experience:

  • I spent more time judging debates than I did speaking in them, which is moderately unusual. It's plausible to me that judging was much more useful.
  • It was 8-12 years ago, and my independent impression is that the top levels of the sport have degenerated somewhat since (e.g. I watched world-class debaters speak and while they spoke fast, I've never seen anything like the link Oli posted).
  • I approached debating with a mindset of 'this is an area I am naturally weak in and want to get better at', so it was always more likely to complement my natural quantitative approach to figuring things out, rather than replacing it.

(Edit: Since some other discussions on this thread are talking about various formats, I should also add that my experience is entirely inside British Parliamentary debate.)

All in all, I think it's very plausible Oli's experience was closer to a typical 2021 experience than mine. But mostly I'm just not sure, for one thing I'd bet that the 'cram as many points in as possible' strategy is still much less prevalent at lower levels. 

With that out of the way, here are things I picked up that I think are important and useful for truth-tracking, as opposed to persuasion.

  • Actually listening to the arguments that have been made, in a way that means I could repeat them back with at-least-comparable eloquence to the speaker. Put another way, I think debating made me much better at ideological Turing tests.
  • A healthy skepticism of the power of arguments and inner-sense-of-conviction as a truth-tracking device, particularly whenever you are talking to someone smarter and more charismatic than yourself, or whenever you've just done something like give a speech (or write a blog post/comment!) in favour of a particular conclusion, or whenever you are surrounded by a group of people who all think the same way. This is very closely related to Epistemic Learned Helplessness. It seems like Scott realised this by reading pseudohistory books, see below quote, but my parallel 'oh shit' moment was being thoroughly out-argued and convinced by much better debaters in favour of A, and then being equally out-argued by debaters in favour of not-A. Unlike Scott's experience, I think those people could argue circles around me on virtually every topic. Which just makes it even more obvious you need a better approach.
  • Being able to generate (some) strong arguments against things I strongly believe and being able to do it independently. It's pretty common for novice debaters who are highly committed socialists to be unable to come up with any arguments for free markets, or vice-versa. I often see similar patterns, including on that exact issue but also on many other issues, within EA groups. I think getting better at this is critical if we want to do more policy work. Closely related: Policy debates should not appear one-sided. I'm also reminded of Haidt's work on moral foundations and how liberals tend to ignore some of the foundations.
  • Identifying critical disagreements, areas that if they resolved one way would likely result in a win for one side, and if they resolved the other way would win for the other side. These are very close to, though not quite the same as, CFAR's concept of a double crux.

To state the hopefully-obvious, I doubt debating is the optimal way to learn any of this. If I was talking to an EA without debating experience who really wanted to pick up the things I picked up, I'd advise them to read and reflect on the above links, and probably a few other related links I didn't think of, rather than getting involved in competitive debating, partly for reasons Oli gives and partly for time reasons. I did it primarily because it was fun and the fact it happened to be (imo) useful was a bonus, not unlike the reasons I played Chess or strategy games. That and the fact that half those posts didn't even exist back in 2009. 

At the same time, if I want to learn things from a conversation with someone that I disagree with, and all I know is that I have the choice between talking to someone with or without debating experience, I'm going with the first person. Past experience has taught me that the conversation is likely to be more efficient, more focused on cruxes and falsifiable beliefs, and thus less frustrating.

And there are people who can argue circles around me. Maybe not on every topic, but on topics where they are experts and have spent their whole lives honing their arguments. When I was young I used to read pseudohistory books; Immanuel Velikovsky’s Ages in Chaos is a good example of the best this genre has to offer. I read it and it seemed so obviously correct, so perfect, that I could barely bring myself to bother to search out rebuttals.

And then I read the rebuttals, and they were so obviously correct, so devastating, that I couldn’t believe I had ever been so dumb as to believe Velikovsky.

And then I read the rebuttals to the rebuttals, and they were so obviously correct that I felt silly for ever doubting.

And so on for several more iterations, until the labyrinth of doubt seemed inescapable...

I don't have time to respond in super much depth because of a bunch of competing commitments but I want to say that all of these are good points and I appreciate you making them.

You did give some responses elsewhere, so a few thoughts on your responses:

But this is really far from the only way policy debate is broken. Indeed, a large fraction of policy debates end up not debating the topic at all, but end up being full of people debating the institution of debating in various ways, and making various arguments for why they should be declared the winner for instrumental reasons. This is also pretty common in other debate formats.

(Emphasis added). This seems like a classic case for 'what do you think you know, and how do you think you know it?'. 

Here's why I think I know the opposite: the standard in British Parliamentary judging is to judge based on the 'Ordinary Intelligent Voter', defined as follows:

In particular, judges are asked to conceive of themselves as if they were a hypothetical ‘ordinary intelligent voter’ (sometimes also termed ‘average reasonable person’ or ‘informed global citizen’). This hypothetical ordinary intelligent voter doesn’t have pre-formed views on the topic of the debate and isn’t convinced by sophistry, deception or logical fallacies. They are well informed about political and social affairs but lack specialist knowledge. They are open-minded and concerned to decide how to vote – they are thus willing to be convinced by the debaters who provide the most compelling case for or against a certain policy. They are intelligent to the point of being able to understand and assess contrasting arguments (including sophisticated arguments), that are presented to them; but they keep themselves constrained to the material presented unless it patently contradicts common knowledge or is otherwise wildly implausible.

This definition is basically designed to be hard to Goodhart. It's still easy for judging cultures to take effect and either reward or fail to punish unhelplful behaviour, and personally I would list 'speaking too fast' under this, but nothing in that definition is likely to lead to people 'debating the institution of debating'. So unsurprisingly, I saw vanishingly little of this. Scanning down recent WUDC finals, the only one where the speakers appear to come close to doing this is the one where the motion itself is "This house believes that University Debating has done more harm than good". Correspondingly, I see no cases where they end up 'not debating the topic at all'. 

The debates I participated in in high-school had nobody talking fast. But it had people doing weird meta-debate, and had people repeatedly abusing terrible studies because you can basically never challenge the validity or methodology of a study, or had people make terrible rhetorical arguments, or intentionally obfuscate their arguments until they complete it in the last minute so the opposition would have no time to respond to it.

I mean, I'm sorry you had terrible judges or a terrible format I guess? I judged more high school debates than virtually anyone during my time at university, and these are not things I would have allowed to fly, because they are not things I consider persuasive to the Ordinary Intelligent Voter; the 'isn't convinced by sophistry, deception or logical fallacies' seems particularly relevant. 

On that note, I don't think it's a coincidence that a significant fraction of my comments on this forum are about challenging errors of math or logic. My rough impression is that other users often notice something is wrong, but struggle to identify it precisely, and so say nothing. It should be obvious why I'm keen on getting more people who are used to structuring their thoughts in such a way that they can explain the exact perceived error. Such exactness has benefits even when the perception is wrong and the original argument holds, because it's easier to refute the refutation. 

I might be wrong here, but I currently don't really believe that recruiting from the debate community is going to increase our cognitive diversity on almost any important dimension.

The Oxbridge debating community at least is pretty far to the right of the EA community, politically speaking. I consider this an important form of cognitive diversity, but YMMV. 

***

Overall, I'm left with the distinct impression that you've made up your mind on this based on a bad personal experience, and that nothing is likely to change that view. Which does happen sometimes when there isn't much in the way of empirical data (after all, there's sadly no easy way for me to disprove your claim that a large fraction of BP debates end up not debating the topic at all..), and isn't a bad reasoning process per se, but confidence in such views should necessarily be limited. 

To be clear, I think very little of my personal experience played a role in my position on this. Or at least very unlikely in the way you seem to suggest.

A good chunk of my thoughts on this were formed talking to Buck Shlegeris and Evan Hubinger at some point and also a number of other discussions about debating with a bunch of EAs and rationalists. I was actually pretty in favor of debate ~4-5 years ago when I remember first discussing this with people, but changed my mind after a bunch of people gave their perspectives and experiences and I thought more through the broader problem of how to fix it.

I also want to clarify the following 

after all, there's sadly no easy way for me to disprove your claim that a large fraction of BP debates end up not debating the topic at all..

I didn't say that. I said "This is also pretty common in other debate formats". I even explicitly said I am less familiar with BP as a debate format. It seems pretty plausible to me that BP has less of the problem of meta-debate. But I do think evidence of problems like meta-debate in other formats is evidence of BP also having problems, even if I am specifically less familiar with BP.

I even explicitly said I am less familiar with BP as a debate format.

The fact that you are unfamiliar with the format, and yet are making a number of claims about it, is pretty much exactly my issue. Lack of familiarity is an anti-excuse for overconfidence.

The OP is about an event conducted in BP. Any future events will presumably also be conducted in BP. Information about other formats is only relevant to the extent that they provide information about BP. 

I can understand not realising how large the differences between formats are initially, and so assuming information from other formats has strong relevance at first, which is why I was sympathetic to your original comment, but a bunch of people have pointed this out by now.

I expect substantiated criticisms of BP as a truth-seeking device (of which there are many!) to look more like the stuff that Ben Pace is saying here, and less like the things you are writing. In brief, I think the actual biggest issues are:

  1. 15-minute prep makes for a better game but for very evidence-light arguments.
  2. Judges are explicitly not supposed to reward applause lights, but they are human, so sometimes they do.
  3. It's rarely a good idea to explicitly back down, even on an issue you are clearly losing. Instead you end up making a lot of 'even if' statements. I think Scott did a good job of explaining why that's not ideal in collaborative discussions (search for "I don’t like the “even if” framing.").

(1) isn't really a problem on the meta (read: relevant) level, since it's very obvious; mostly I think this ends up teaching the useful lesson 'you can prove roughly anything with ungrounded arguments'. (2) and (3) can inculcate actual bad habits, which I would worry about more if EA  wasn't already stuffed full of those habits and if my personal experience didn't suggest that debaters are pretty good at dropping those habits outside of the debates themselves. Still, I think they are things reasonable people can worry about.

By contrast, criticisms I think mostly don't make sense:

  • Goodharting 
  • Anything to the effect of 'the speakers might end up believing what they are saying', especially at top levels. Like, these people were randomly assigned positions, have probably been assigned the roughly opposite position at some point, and are not idiots. 

Finally, even after a re-read and showing your comment to two other people seeking alternative interpretations, I think you did say the thing you claim not to have said. Perhaps you meant to say something else, in which case I'd suggest editing to say whatever you meant to say. I would suggest an edit myself, but in this case I don't know what it was you meant to say.

Finally, even after a re-read and showing your comment to two other people seeking alternative interpretations, I think you did say the thing you claim not to have said. Perhaps you meant to say something else, in which case I'd suggest editing to say whatever you meant to say. I would suggest an edit myself, but in this case I don't know what it was you meant to say.

I've edited the relevant section. The edit was simply "This is also pretty common in other debate formats (though I don't know how common in BP in particular)".

By contrast, criticisms I think mostly don't make sense:

+ Goodharting
+ Anything to the effect of 'the speakers might end up believing what they are saying', especially at top levels. Like, these people were randomly assigned positions, have probably been assigned the roughly opposite position at some point, and are not idiots.

Alas, those are indeed my primary concerns. It's of course totally OK if you are not compelled, but I have no idea how you are so confident to dismiss them. Having talked to multiple people who have participated in high-level debate in multiple formats, those are the criticisms that they level as well, including for formats very similar to BP, and for BP in-particular. 

I have now watched multiple videos of BP debate, and I wish I didn't because my guesses of what it would look like were basically right, and I feel like I wasted two hours of my time watching BP debates because you insisted that for some reason I am not allowed to make claims from very nearby points of evidence, even though as far as I can tell after spending those two hours, most of my concerns are on-point and BP looks just like most other forms of debate I've seen. 

I knew when I wrote the above comment that BP debates look less immediately goodharted. But after engaging more deeply with it, I would be surprised if it is actually much less goodharted. Of course, 4 debates at 2x speed isn't really enough to judge the whole category, but given that I've watched dozens of other debates in multiple other formats, I feel like I've pinpointed the type of debate that BP is pretty well in debate space. 

Of course, you can insist that only people intimately familiar with the format participate in this discussion, in which case I of course cannot clear that bar, and neither can almost anyone else on the forum (and this will of course heavily select against anyone who is critical of debate).


Let me take a step back. Look... I feel super frustrated by your comment above. I am trying to contribute a number of points to this discussion that feel important, and you now twice just kind of insinuated that I am unfairly biased or being unreasonable without really backing up your points? Like, your comments have been super stressful, and the associated downvotes have felt pretty bad. I think the arguments I've made in my comments are pretty straightforward, I stayed civil, and I don't think I am being particularly irrational about this topic. 

I've thought about it for 2-3 dozen hours over the years and at multiple points in the last few years have spent full-time work weeks evaluating whether we should have a debate tradition inside of EA as well, which caused me to think through a lot of the relevant considerations and investigate a substantial number of debate formats. I've talked to something like 8 people with extensive debate experience in EA for at least an hour and tried to get a sense of what things worked and what didn't. 

And then you come along and just assert: 

Overall, I'm left with the distinct impression that you've made up your mind on this based on a bad personal experience, and that nothing is likely to change that view. 

And this... just feels really unfair? Indeed, phenomenologically, my debate experiences were great. I didn't have a random bad experience that somehow soured me towards this whole sport. I was positive on it, and then thought about it for at least a dozen hours in total and overall came to a complicated high-level position that overall was a lot more hesitant. I have separately also thought for at least a literal 1000 by-the-clock-hours about our talent funnels and the epistemic norms I want the community to have and how the two interact. 

My position also isn't categorically opposed to debate at all. Indeed, I am personally likely to cause EA and the Rationality community to have more of a debating institution internally, and continue to feel conflicted about this project. I think it's quite plausible it's good, but I would want the organizers to think hard about how to avoid the problems that seem pretty deeply embedded in debate, and how to avoid them damaging the social institutions of EA, or attract people that might otherwise cause harm.

I don't know. It's fine for you to think I am being irrational about this topic, or for some reason to categorically dismiss the kind of concern that I am having, but I don't feel like you've really justified either of those assertions, and I perceived them both coming with some kind of social slap-down motion that made participating in this thread much more stressful than necessary. I will disengage for now. I hope the people involved in this project make good choices.

I just don't think this is very relevant to whether outreach to debaters is good. A better metric would be to look at life outcomes of top debaters in high school. I don't have hard statistics on this but the two very successful debaters I know personally are both now researchers at the top of their respective fields, and certainly well above average in truth-seeking.

I also think the above arguments are common tropes in the "maths vs fuzzies" culture war, and given EA's current dispositions I suspect we're systematically more likely to hear and be receptive to anti-debate than to pro-debate talking points. (I say this as someone who loved to hate on debate in high school, especially as it was one of the main competitors with math team for recruiting smart students. But with hindsight from seeing my classmates' life outcomes I think most of the arguments I made were overrated.)

Hi, thank you for your thoughtful response, I appreciate you taking the time to write it. You presented a complicated issue, and I think discussions like this are further complicated by the fact that debating can vary wildly depending on how it’s taught and practiced, so different people can have extremely different experiences (like in the different comments on this thread). Many people already responded to parts, but I’d like to stress a few things that I believe are worth highlighting. 

1) The effect of the tournament. The question of the quality of debating as a tool to evaluate the truth is somewhat orthogonal to the question of whether it is worthwhile to engage the debate community with EA-related concepts. While there is value in discussing the former (and I’ll address it later in the comment), the main purpose of the project was the latter. That is to say that rather than convincing EAs to adopt debating as a tool, the aim of the championship was to engage people that already debate with EA. We believe it is a worthwhile effort for all of the reasons stated in the post - e.g. there is potentially a big future gain due to the prospective influence members of the community are going to carry, the project is positively positioned to generate insights for EA advocacy that may be beneficial broadly, etc. It is of course very valid to hold an opinion that there are issues with debating itself, but somewhat like issues with the conducts of tech companies should not prevent us from nudging them towards an EA direction, we think the same applies here. The way we’ve structured the tournament is focused on this - we’ve publicized this tournament to debaters (not non-debating EAs), the process included lots of exposure to EA content, and so on. Therefore, when evaluating the project we primarily took the prism of the value of engaging the debating community. 

2) The opportunities of the EA championship. There is truth to the claim that when debating is taught poorly it can lead to suboptimal habits. However, we think these issues are not inherent to the format. Specifically, in this project we assembled a team with a specialty in global debate education that had successfully organized hundreds of international events, built the complimentary lecture series that allowed people to explore EA more deeply before/after the tournament, created additional channels so that people can reflect on content (e.g. a FB group and Discord channels that were used during the tournament), and had a dedicated team of people that were tasked with making sure that the atmosphere in the tournament is positive & inclusive. While it is hard to prove that such an effort was successful, the anecdotal evidence of participants indicating that they wish to learn more on EA, the adoption of donation norms in other tournaments, and the very positive feedback participants provided serve as evidence that (hopefully) the message was conveyed well and that participants were responsive to the educational element of the tournament. 

3) Norms of learning. The norms of debating have shifted quite heavily in recent years and the BP-community is very different from the policy-community (the one featured in the video). There are ample discussion groups for debaters that seek to deepen their knowledge, there is a lot of emphasis on inclusion, and extracurricular educational videos are a rising trend. Therefore, most debaters operate in ecosystems where exploring complexity is a virtue. 

4)  On debating itself. I think it is fair to say that many debates do not result in figuring the truth on the topic at hand due to the complicated nature of the policies & ideas that are discussed and the limited time per discussion. However, there are multiple benefits to the activity that, in my opinion, make debating a useful tool. To name a few:

a) To debate well one needs to carefully listen to the arguments the other side is making, be able make rapid yet thoughtful responses, build arguments and understand how they relate to each other, predict critical points of disagreements and update them dynamically as the discussion progresses, understand how to work well in a team, etc. There are of course other ways to pick up these skills, but debate provides a useful way to practice them. Further, since debating improves your ability to understand how arguments relate to one another, these skills can aid in figuring which position makes more sense in complicated discussions in real life, which can be helpful in seeking the truth, or at the very least in identifying falsehoods.       

b) Debating incentivizes you to learn more about the world. Even if winning is one's key motivation, being competitive requires a lot of preparation between tournaments. This usually involves researching ideas, discussing thoughts with peers, reflecting on previous rounds, etc. From my experience, the general culture is one where debaters are nudged towards challenging their current perceptions and to enrich their world-views. 

c) Debating forces you to engage with multiple perspectives. Positions in debating, i.e. being for or against the topic, are randomly allocated. This feature compels you to think on a topic in ways you might not have otherwise, and ultimately assists in developing a more nuanced world view.   

d) The debating community is truly global. In competitions you can hear voices that are hard to find in other places. The ability to gain the perspectives of people from around the world on a plethora of important issues has benefits for those that hold EA values.

The bottom line is that I agree that the tool is not perfect. However, I think that since the competition primarily introduces EA to debaters rather than promotes debating as a tool, that should be the main consideration in prioritizing this project. I also tried to note in the above why I think debating as a whole, and these kinds of competitions in particular, are potentially beneficial.

I appreciate the response!

However, we think these issues are not inherent to the format. 

I do basically think the problems are kind of inherent to the format and pretty hard to fix. Like, I don't think it's physically impossible to fix these issues, but I am very skeptical of any efforts that try to fix them that stay within the existing debate context. 

Overall, I am not sure where the key causes of our disagreement lies. The above didn't really feel like it addresses my core concerns with the sport, and while the list of benefits is nice, it feels like a list that I can basically construct for an arbitrary sport (like, not this specific list of debates, but something of equal benefit), and below I give some pointers why I think some of them don't hold, or at least don't hold with the forcefulness that one might expect based on your descriptions.

I think it is fair to say that many debates do not result in figuring the truth on the topic at hand due to the complicated nature of the policies & ideas that are discussed and the limited time per discussion.

To be clear, neither the complexity of the topics or the limited time are at all anywhere close to the central reason why I think debate isn't very truth-seeking. I can totally have a meeting with a bunch of friends of mine about a complicated issue with only an hour of time, and we can easily make good approximations and solid progress on understanding it. I am quite confident we would not if we instead spent that time in any competitive debating context. Indeed, it seems likely to me that we would leave the competitive debating context with worse beliefs than we entered it on the relevant topic.

There are ample discussion groups for debaters that seek to deepen their knowledge, there is a lot of emphasis on inclusion, and extracurricular educational videos are a rising trend. Therefore, most debaters operate in ecosystems where exploring complexity is a virtue. 

None of these (inclusion, extracurricular education or "exploring complexity is a virtue) have really much to do with my concerns for debate, so at least from my perspective, describing these as trends that are gaining momentum in the debate community does very little to make me less concerned. Some of these seem mildly bad to me.

Further, since debating improves your ability to understand how arguments relate to one another, these skills can aid in figuring which position makes more sense in complicated discussions in real life, which can be helpful in seeking the truth, or at the very least in identifying falsehoods.       

I don't really think debate is really helping you understand how arguments relate to each other, at least not in a truth-tracking way. In most debate formats, it's usually much less about actually making good arguments, but much more about abusing the way judges are told to score various arguments, in a way that has very little to do with the cognitive patterns I would encourage someone to use if they were trying to figure out whether an argument makes sense or not. 

c) Debating forces you to engage with multiple perspectives. Positions in debating, i.e. being for or against the topic, are randomly allocated. This feature compels you to think on a topic in ways you might not have otherwise, and ultimately assists in developing a more nuanced world view.   

I think this is actually useful, and learning the skill of generating steelmanned-arguments for positions you don't believe is quite useful. Though because of the problem I pointed out above with the arguments that you are generating having very little to do with actual truth-trackingness, this benefit does fall quite a bit short from the ideal you describe here. 

In my experience it creates a kind of "fallacy-of-grey"-like mindset where you are avoiding having any beliefs on these issues at all, or don't really think of it being your job to actually decide which side is right, which I think is quite bad. Ultimately the goal of understanding both sides of an argument is to still judge which side is right (or of course to do a more complicated synthesis between the two, though that's I think pretty actively discouraged in the debating format).

d) The debating community is truly global. In competitions you can hear voices that are hard to find in other places. The ability to gain the perspectives of people from around the world on a plethora of important issues has benefits for those that hold EA values.

I don't really believe this? The debating community is overall really insular and narrow, as far as I can tell, being really heavily selected for being full of all the standard ivy-league people that we already have a ton of. I like cognitive diversity, but I don't really think the debating community is very exciting from that perspective. Indeed it seems to have very similar selection filters to the way the EA community is already filtered for. I might be wrong here, but I currently don't really believe that recruiting from the debate community is going to increase our cognitive diversity on almost any important dimension.

Following up on this, as part of me trying to understand the format of BP more, I was watching this video, which is the most watched WUDC video on Youtube. And... I find it terrifying. I find it in some sense more terrifying than the video where everyone talks super fast: 

I encourage other people who are trying to evaluate debate as a method for truth-seeking to watch this themselves. 

There is no super-fast-talking here, but all the arguments in the opening speech are terrible rhetorical argument. The speaker leverages the laughs and engagement of the audience to dismiss the position of his opponents, and this overall really felt more terrifying to me than many of the big political speeches I've seen this year. 

Like, I... think I am more terrified of the effect this would have on epistemics than the effect of the super-fast-talkers in policy debate? Like, at least in policy-debate it's somewhat obvious you are playing a game. In the above, I wouldn't be surprised if the participants actually come to believe the position they are trying to defend. 

I was surprised, this video was much less goodharted than I expected (after having been primed with the super-fast talking example). I was expecting more insane things.

Though overall it had the level of much broad public debate/discourse I’ve seen. I watched the first three speakers, and didn’t learn anything. In good debates I’ve seen I’ve felt that I’ve learned something from the debaters about their fields and their unique world views, these felt like two opposing sides in a broader political debate with kind of no grounding in reality. They were optimized for short-scale (e.g. <30 seconds) applause lights for the audience, when objected they’d make it a fight saying things like “Don‘t even try to win that example”, their examples seemed false yet rewarded (primarily attributing China’s rise out of poverty in the last 50 years to ‘redistribution’ and getting applause for it, which, correct me if I’m wrong, is not at all the primary reason, they had massive growth in industry in part by copying a lot of the west). I wouldn’t expect to learn anything, it just seemed like nobody understood economics and they were indexed off what was like 0-1 inferential steps from what the audience as a whole understood. I guess that was the worst part, how can you discuss interesting ideas if they have to be obvious to an audience that big and generic within 10-20 seconds?

I did competitive college debate for four years (American Parliamentary format, which is similar to the BP format used in the EA Debate Championship but not identical) and I think that the extent to which it does/doesn’t encourage truth-seeking is less important than the way it pushes people to justify their values.

Oversimplifying broadly, debate has two layers: one is the arguments about what the impacts of a certain idea/policy are likely to be, and one is arguments about which impacts are more important (known as “weighing”). In order to win rounds, you have to win arguments at both levels. This means that debate requires people to engage with one of issues that is most central to EA — a relatively consequentialist understanding of which issues matter most. In regular life you can say, “I support government funding for the arts because art is good” and not think very hard about how that trades off with, say, funding for healthcare. But if you do that in a debate round, the other team will point out the tradeoff, estimate the number of people who will die as a result of there being less funding for healthcare, and you will lose the round.

I think this is the main benefit of debate from an EA perspective, and I suspect that it has meaningful impacts on people who are forced to confront, over and over again in countless debate rounds, the actual effects (in lives lost and other very serious harms) of different ways of weighing between issues. Anecdotally, a higher-than-average percentage of the debaters I know are EAers, or at least interested in EA. And even debaters who don’t personally support EA very often use EA weighing arguments in rounds. As a result, for some people (I suspect many), debate is the first place they hear about EA. To me, this makes debate leagues a fertile recruiting ground for EA.

People shared so many bad experiences with debate…

I had a great time debating (BP style) in Russia a few years ago. I clearly remember some moments which helped me to become better at thinking/speaking and world modeling:

  • The initial feedback I got during the practice session is basically don't be a guy from the terrible video you shared :-). Make it easy for a judge to understand your arguments: improve the structure and speak slower. Focus on one core argument during your speech: don't squeeze multiple half-baked ideas in; deliver one but prove it fully.

  • At my first tournament for newbies, an experienced debater gave a lecture on playing something-something resolutions and concluded with strongly recommending reading up on game theory (IIRC, The Strategy of Conflict and Governing the Commons).

  • My second tournament was in Jedi format: I, an inexperienced Padawan, played with a skilled Jedi. I matched with a person because both of us liked LessWrong. I think we even managed to use "belief should pay rent" as part of an argument in a debate on the tyranny of the majority. I think it's plausible that we referred to Moloch at least once.

  • Later on, improvement came from managing inferential distances during speeches; and grounding arguments in reality: being specific about harms and benefits, delivering appropriate ~examples to support intermediate claims.

I think the experience was worth it. It helped me to think more in-depth and about much more issues than I would have overwise (kinda like forecasting now). I quit because (a) tournaments are time-consuming; (b) I got bored playing social issues & identity politics.

While competitive debating is not about collaborative truth-seeking, in my experience, debtors are high cognitive decouplers. Arguing with them (outside of the game) felt good, and we were able to touch topics far outside of the default Overtone window (like taking the perspective of ISIS).

The culture was healthy because most people were just passionate about debating/grokking complex issues (like investor-state dispute settlements), and their incentives were not screwed up because the only upside to winning debate tournaments in Russia is internet points.

Upd: I feel that one of your main concerns is Goodharting. I think the BP system as we played it basically encouraged maximizing the expected utility of impacts of arguments you brought to the table i.e. harm/benefit to individual × scale × probability occurring × how well you proved it (which can be seen as the probability that your reasoning is correct). It's a bit harder to fit the importance of framing the issue and principled arguments into my simplification. But the first can be seen as prioritizing based on relative tractability (e.g. in almost all of the debate arguing that "we will save money by not implementing a policy" is a bad move because there are multiple other ways to save money and the benefits of the policy might be unique). The second is about the importance of metagame, incentive structures, commitments, and so on.

I want to echo this. I think my own experience of debating has been useful to me in terms of my ability to intelligence-signal in person, but was pretty bad overall for my epistemics. One interesting thing about BP (which was the format I competed in most frequently at the highest level) was the importance in the 4th speaker role identifying the cruxes of the debate (usually referring to them as "clash"), which I think is really useful. Concluding that the side you've been told to favour has then "won" all of the cruxes is... less so.

I was confused about the situation with debate, so I talked to Evan Hubinger about his experiences. That conversation was completely wild; I'm guessing people in this thread might be interested in hearing it. I still don't know exactly what to make of what happened there, but I think there are some genuine and non-obvious insights relevant to public discourse and optimization processes (maybe less to the specifics of debate outreach). The whole thing's also pretty funny.

I recorded the conversation; don't want to share publicly but feel free to DM me for access.

...that video is deeply absurd, and I am caught between revulsion and laughter. They are clearly way past the point where there is even a gesture at real usefulness; this might be the most heavily gamed thing I have ever seen.

However, if the video is an example I am confident that this activity can't do any real harm to our objectives. Who could do this with even the pretense of thinking they would talk to people in the real world that way? It's just a verbal luge.

Yeah, it’s kinda hilarious. Speaking so fast that your opponents can’t follow your arguments and therefore lose the round is common practice in some forms of competitive debate. But in other debate categories, using this tactic would immediately lose you the round. In my own personal experience of high school debate, the quality of competitive debate depends very heavily on the particular category of debate.

The video above is Policy Debate, the oldest form of debate which degenerated decades ago into unintelligible speed reading and arguments that every policy would result in worldwide nuclear annihilation. In the 1980s, the National Speech and Debate Association instituted a new form of debate called Lincoln Douglas that attempted to reground debate in commonsense questions about moral values; but LD has also fallen victim to speed reading and even galaxy-brained “kritiks” arguing that the structure of debate itself is racist or sexist and therefore that the round should be abandoned.

Public Forum debate, invented in 2002 as an antidote to Lincoln Douglas, is IMO a very healthy and educational form of debate. Here is the final round of the 2018 national championship (starting at 4:05) on the resolution, “On balance, the benefits of United States Participation in the North American Free Trade Agreement outweigh the consequences.” https://m.youtube.com/watch?v=MUnyLbeu7qU&feature=youtu.be

British Parliamentary debate is another form of debate that, in my experience, is more civil and less “game-able” than other forms of debate (though Harrison D disagrees below, with specifics about its pitfalls). One key difference is that, while Public Forum allows and encourages debaters to spend weeks researching and debating a single specific resolution, Parliamentary debate typically a involves generalized preparation on a subject or theme and only reveals the specific resolution a few minutes before the round begins. Because of this, I think Public Forum is more educational for debaters, but Parliamentary is probably easier to run a one-off tournament because debaters won’t be expected to have done as much preparation.

Extemporaneous Speaking is another category involving less preparation, where participants are asked a question about current affairs or politics and have 30 minutes to prepare a 7 minute off-the-cuff speech. There is no “opponent” in Extemp, perhaps limiting the level of discourse, but it might be possible to easily introduce EA-related topics because participants are expected to be conversant in a wide range of topic areas.

On the whole, I’m very glad to see this EA debate tournament being run, and would be very excited to see further work bringing EA topics into debate. I can understand why many people might find some debate tactics toxic and counterproductive, particularly in categories like Policy and LD, but I do think this is the failure of specific categories and tactics and not an indictment of all adversarial debate. Learning the best arguments for both sides of a resolution certainly teaches a bit of an “arguments as soldiers” approach, but I believe the greater effect is to lead debaters to real truths about which arguments are stronger and improve their personal understanding of the issues. In future EA debate events, I would only suggest that organizers be very conscious of these standards and norms when choosing a specific category of debate.

Just to clarify my position:

  • I think that the culture of British Parliamentary has made it significantly less game-y and more civil than most if not all other formats of collegiate debate, including both prepared formats (e.g., Policy Debate, Lincoln-Douglas, Public Forum) and limited preparation formats (e.g., other forms of parli such as American Parliamentary).
  • I think that the limited topic prep nature of parliamentary debate makes those formats significantly less game-y and more civil than most other formats of collegiate debate.
  • My main issue with BP is really just two individual characteristics in the format that represent stark differences from the format I did in high school (American Parli, in the Stoa league): the 4-teams-of-2 (instead of 2v2) combined with the lack of access to published sources on the internet when in prep. Really, most if not all of the main issues I highlighted in my comment relate to the first thing, which I think is more fundamental. So ultimately, I'm not trying to compare BP to policy debate, nor am I trying to compare it to the actual (culturally-driven) practice of American Parli in collegiate leagues (which I'm not as familiar with), it's really just me comparing it to what I think an ideal format would be when given a decent culture that isn't so acceptive of gamification.

I appreciate this nuanced comparison between formats. I increase my estimation of the goodness of the EA debate project, because this introduces a new pathway to victory:

The creation of an EA debate format, which can be designed against the backdrop of all the poisoned formats to encourage skills and norms more like those we want to develop.

It does feel like the general period for research before the debate would work well for these topics. Do you know if any format has a mechanism for awarding points on the grounds of agreeing with the opponent's points, or otherwise acknowledging when arguments or data weigh against you?

+1 from my own experience in debate (also british style). Truth-seeking / identifying and meaningfully resolving points of disagreement between different positions is a very different skill to trying to win a debate, and the skillsets/mindsets developed in the latter seem like they might actively work against the former unless the people doing it are very careful and self-aware.

It's unfortunate to hear that you had such a negative experience from debate. As someone who has judged public-school high school policy and public forum debate, I will say that I am not that surprised. That being said, I do take issue with your characterization of all/BP debate with the video plus the statement "I do think British Parliamentary Debate style is a bit less broken than this, but like, not that much."

I cannot speak to every BP league/competition in the world, but I have never seen nor heard of such drastic gamification of debate in BP--or anything even come close to it--in the four years that I did BP in college. In fact, I have often seen people hold up BP and collegiate policy debate as polar opposites, with BP being one of the least toxic/gamified formats (at least among the major formats) and policy debate being the most. (BP definitely has some problems with left-leaning judge bias, but it could be a lot worse and that's not really that unique to BP.) Ultimately, I don't want to be rude/abrasive, but I feel that the video really gives a deeply misleading picture of BP, even if it is only a mild-moderate exaggeration of collegiate policy debate. I think it's unfortunate to think that many people who are unfamiliar with debate (let alone BP specifically) may come away with such a misleading picture of BP/debate in general based on this extreme example of a different format. I'm not sure how to say this in a non-confrontational way, but I personally think that some kind of revision/redaction (e.g., a disclaimer saying that the video is not of BP, acknowledgement that BP is different) may be in order.

I will just add the following video to illustrate that the gamifying (e.g., speed and spread tactics) of debate is not so inherent to the sport or even specific formats themselves, but rather are heavily determined by league culture (e.g., what kinds of judges are used, how debaters are taught to debate): https://www.youtube.com/watch?v=TvhNvumnZ1U&t=23s (Although, do excuse the pathos-heavy story at the beginning, and remember these are just high school students)

I just want to say I, Ben Pace, feel attacked every time someone criticizes “BP” in this comment thread.

A lot of my models here come from talking to Evan Hubinger about this, who has a lot of thoughts on debate and competed at the national level in Policy debate in college.

My guess is that overall debate is really badly goodharted. All of the different variants. One of the ways policy-debate in particular is really badly goodharted is that everyone talks so fast nobody can comprehend them. But this is really far from the only way policy debate is broken. Indeed, a large fraction of policy debates end up not debating the topic at all, but end up being full of people debating the institution of debating in various ways, and making various arguments for why they should be declared the winner for instrumental reasons. This is also pretty common in other debate formats (though I don't know how common in BP in particular). Evan won a bunch of his highest-level debates by telling stories about pirates for 15 minutes, and then telling everyone that the institution of debate is so broken and useless and wouldn't it just be better if we would use this time to learn cool facts about history like the pirate anecdotes I told you in the first 80% of the debate?

If people were just talking fast, I do think that would be a problem, but not a super big one. I can imagine a kind of hyperoptimized court system that functions fine with everyone talking at 2-3x normal speed. The problem is that it's evidence that the system at large has very little defenses against goodharting and runaway competition effects. 

As far as I can tell, some other debate formats do not have the problem of everyone talking at 2-3x speed, but basically have many of the other problems, and importantly, all share the fundamental attribute that they have very few successful defenses against goodhart's law. And so somehow, even though it might differ from debate format to debate format, the actual thing the competitors are doing has very little to do with seeking the truth. 

The debates I participated in in high-school had nobody talking fast. But it had people doing weird meta-debate, and had people repeatedly abusing terrible studies because you can basically never challenge the validity or methodology of a study, or had people make terrible rhetorical arguments, or intentionally obfuscate their arguments until they complete it in the last minute so the opposition would have no time to respond to it.

I watched about 20 minutes of the video you linked, and I do think just from that slice it seems much less obviously broken than the video I linked, and that does seem important to recognize. 

I do also think that I can only really interface with that whole video in a healthy way by repeatedly forcing myself to take a step back and basically throw away all the information that is thrown at me, because I don't trust it, and I don't trust the process that produced it. Like, I don't think I left that video having a better understanding of seatbelt laws (the topic of debate). I learned some useful relevant facts, but I am still worried that I left that debate with worse beliefs about seatbelt laws, and very likely much worse than if I had just read the Wikipedia article on it. Of course, the point of the debate is not for me to learn about seatbelt law, but I do also think the same is probably true for the participants. 

"The problem is that it's evidence that the system at large has very little defenses against goodharting and runaway competition effects." Although I acknowledge that there will always be some level of misalignment between truth-seeking and competition, I would push back on the idea that the system has little defense against drastic goodharting like is seen in both high school and collegiate policy debate: the experience of Stoa (the league in which I debated) and NCFCA are evidence of that. In my view and in the view of some others (see e.g., https://www.ethosdebate.com/community-judges-1-necessity-community-judges/), it seems that one of the important front-line defenses against gamification of debate is the use of community judges who recoil at nonsense and speed. Of course, that introduces tradeoffs that debaters (myself included) sometimes huff about, such as biased decisions, but it still seems worth it. Additionally, I feel fairly confident that there are other important factors that explain the stark cultural differences between Stoa/NCFCA and most public-school/collegiate leagues (e.g., the debaters' personalities/background, parental involvement, the Christian ethos, the observation of and opportunity for self-differentiation from public-school/collegiate practices).

To address your broader point about the truth-seeking vs. competition drive (goodharting): I and many others in my league have considered this question. (For a brief example article from someone I know, see https://www.ethosdebate.com/art-persuasion-vs-pursuit-truth/) I could be wrong/exaggerating, but I get the sense from you that debate should be really strict about promoting truth-seeking above other things--perhaps even to the extent that debate should almost never sacrifice truth-seeking for other goals. Perhaps that is not what you are saying, but regardless, I would push back and emphasize that debate has a wide variety of purposes, crucially including skills education in general (as opposed to topical education). (I actually recently finished a blog series which I started by outlining some of the major purposes of debate: see https://www.ethosdebate.com/purposes-of-debate-pt-1-the-goals-and-anti-goals-of-debate/ ). In short, I think that the experience of Stoa/NCFCA shows that with reasonable safeguards (e.g., including community judges in the judging pool) debate can be at least neutral if not more positive than negative in promoting truth-seeking, while at the same time is a great way to get youth excited about studying topics, scrutinizing their own views, and learning to persuade others. That last part applies to that NITOC final (regarding seatbelt policy), which focused on a case that was known for being somewhat pathos-heavy (as opposed to, for example, the case for cutting funding for air marshals, which I and many other debaters would likely have never come to see if it were not subject to the adversarial scrutiny of a competitive season of debate): debate shouldn't be entirely/solely about truth-seeking; teaching persuasion skills is also really important, because if you have the truth but cannot persuade others, then your ability to act on it is sorely limited.

Also: "people repeatedly abusing terrible studies because you can basically never challenge the validity or methodology of a study" -- my experience in Stoa was fairly different: I repeatedly had to defend the methodology of some of the studies I relied on, and was able to challenge the methodology of sources.

Sure, debate may involve a lot of techniques that are counterproductive to truth-seeking, and I wouldn't want people to write on the EA Forum like it's a debate, for example. However, I think there are many places where it would help to be able to convey more convincing arguments even if being more convincing doesn't improve truth-seeking—speaking with non-EAs about EA, for example.

I generally want us to use truth-seeking methods when engaging with outsiders as well. Of course, that isn't always possible, but I also really don't want us to have a reputation for using lots of rhetorical tricks to convince others (and generally think that doing so is pretty bad).

As someone who did debate in high school and throughout college, I am really excited to see this + I think it makes a lot of sense. As you noted, debate often involves evaluating choices in more-neutral ways, seeing both sides of arguments, etc. I'd love to hear more about how this project/idea develops.

The only thing I would note is my moderate dislike for the British Parliamentary (BP) format. Of course, I recognize that it may not be feasible to choose a different format and/or that there may be other justifications for using it (e.g., having more people per round, the league's culture is not as wacky/out-of-touch as some other leagues', a greater breadth of perspectives in each round). 
Still, in my experience/analysis, BP's 4-teams-of-2 format (instead of the traditional "one team vs. one team" format), wherein teams that are ostensibly supposed to be  working together to support their side of the motion are actually partially pitted against each other to get a higher rank in a round, leads to numerous problems that undermine the educational value of the round: knifing* (where one of the "back half" teams undercuts something that the "opening" team on their  own side said), abandoning (where one of the back half teams lets the other side strawman or otherwise unfairly attack their opening team's arguments), the fact that closing government (back-half team for the motion) can really suffer if opening government sets up the round poorly (e.g., when opening government uses really bad definitions), the fact that closing teams are often incentivized to focus on "new" arguments rather than focusing on the "good" arguments (since those will usually already have been taken by the opening teams), etc. 
(Honestly, this is just a few of the highlights: for a few months off-and-on I've been outlining a blog article on why I dislike certain aspects of BP. Who knows, maybe I'll finish it sometime this month?)

*Although hard knifing is rarely an effective strategy (usually, judges aren't blind to what's going on and they'll punish the knifer if it was bad/uncalled for), it's maddening how effective soft abandonments are (e.g., only giving half responses then saying something like "we want to focus on new matter on back half").

 Thank you for the comment! Choosing BP was “easy” in the sense that it is the most international format and we were aiming for an event that will be major on the global debate calendar. We think the points you raised are interesting, and even though we believe that BP is actually the most fitting format for the project, the pilot we are running also features some 3vs3 debates, so we will have some ability to compare. If an insight comes up from that comparison we will update.

Sounds good! Like I said, I do recognize that choosing BP probably has quite a few advantages on the (meta?) level in the sense that it seemingly has a more-global audience and topic scope, perhaps a better competitive culture, etc. (Update/clarification: I would say that all of the "flaws" with BP's format are minor in comparison with the advantages from the BP league culture, which crucially does not have the ridiculous speed and spread from policy debate, as exhibited in the video from Habryka.) If I ever get around to finishing that article about its downsides I might share a link to it here...

Was so surprised and happy to see this when just scrolling through EA Forum today! Thanks a lot for conducting it, I really enjoyed competing. The high school pilot program looks incredible!

This is a really neat idea. I used to debate a lot during my undergrad, and while I have quite a lot of negative feelings towards the sport from my own experience, I do agree that debaters are a very receptive audience for EA ideas for the reasons you mention.

One major challenge I see is getting debaters to take more than a superficial interest in ideas that sound interesting. Debates about AI, global health, animal rights and many other EA issues were fairly popular motions when I debated, but didn't necessarily lead to deeper consideration than simply asking "how can I win a debate about this issue?" 

Obviously, the lecture series was useful for introducing people to EA who were unfamiliar, but hopefully, it also prompted participants to think about the issues in a deeper way - not just in the context of debating the topics. I think combining debate with these additional engagement resources is a smart way to run these kinds of events going forward.

 

P.S. Hi Dan, I think I adjudicated a final with you back in Belgrade 2016 - very cool to see an old familiar name on here.

Thanks! The indicators we have are quite positive on whether people took a more profound interest than purely trying to win, and we are trying to figure ways to further deepen the engagement.

It is also great to hear from you Billy, we should definitely schedule a catch up!

I hope you find finding to pay someone to organise this as I suspect this program could be extremely impactul.

I would also love to see some amount of prize money funded for this. I wouldn't be surprised if a relatively small amount of money by philanthropic standards could tempt more of the top debaters to enter.

Thanks for the excellent writeup!

A few questions:

  1. How might you use additional funding to boost the program? I think this would be a really strong candidate for the EA Infrastructure Fund, and I'd also be open to providing a small grant ($3,000 or less) if you had a need that couldn't be filled by the Fund or by Open Philanthropy.
  2. If someone wanted to help with your social media, would they need to have debate experience? I know at least one person who might be interested, but I'm not sure they've done debate before.
  3. Does your post-event survey ask about outcomes aside from interest — for example, people taking a giving pledge or joining a local group?
  4. It sounds like the donation match may not have been counterfactual if the money came from Open Phil — did you try to communicate this to the debaters? Or was it in some sense counterfactual? (This is a tiny nitpick in the broader context of the event.)

Thanks for the comment!

  1. The key budget item we’d need is recruiting a team member that will help in running the organizational aspects of our endeavours. If we find the pilot successful in a few months, or if we want to repeat the event in the future, having a paid member that can dedicate more time to run the program would be extremely useful. 
  2. Having a debate background is a big advantage, but we don’t think it is a must. We are happy to follow up to see whether the person is a good fit.
  3. The survey was actually quite long and we asked for many more things (e.g. whether the positions of the participants changed on some EA related topics). In terms of direct outcomes we also asked about the willingness to share EA content with others and the willingness to donate. We are still processing this data, and if we find interesting conclusions we will report them.
  4. We made no statements about how conterfactual the donation was (and most of our audience would not think of it in these terms). There’s some additional relevant context that we haven’t included in the above summary though. The matched funds were allocated to an effective charity chosen by the winners of the tournament. We did this as we believed this would seak to the competitive spirit of participants. As a result, most of our emphasis in our communications was not about the fact that these are additional funds (that would not have been donated effectively otherwise), but rather that these funds would empower whoever wins the tournament to choose where the funding goes - which is true (even in the counterfactual sense).

Thanks for hosting this event! It was a pleasure to participate. 

This is very cool. Seems like high fidelity outreach to a highly promising group, done well.

My main concern: how often can this general debate theme be repeated among major debate tournaments? I wonder if this will now not be repeatable for several years.

<<Lesson 5: It may be helpful to design a formal EA-advocacy framework and research agenda. Debate can be a useful case-study for EA-advocacy for the reasons mentioned in this post.>> I have often thought this. There is a lot of research that seems like it could be useful for EA outreach, e.g. testing the effectiveness of various messaging strategies. There's some cause area-specific research, but not much that I'm aware of relating to more general EA principles.

<< However, even with the help of fellow EAs, it took us a while to understand how best to measure engagement with EA content. >> I have also thought this! I created some questions to use in an RCT we are running at Animal Advocacy Careers and lamented that I had a scale to use for "Animal Farming Opposition" (based on factor analysis of Sentience Institute's surveys) but not "Effective Altruism Inclination" or something similar.

Would be happy to discuss the EA outreach research ideas a bit more if anyone reading this is interested in pursuing (/ collaborating on?) that.

Thanks for the comment!

WRT the main concern - we think EA is broad enough to be able to generate a distinct enough set of topics on a yearly basis (or a two years basis).

On discussing the EA outreach research - we would like too! This would probably be most relevant when we conclude the pilot program this summer, so we should keep in touch :)

I saw after writing this comment that Jonas Vollmer's recent "Some quick notes on 'effective altruism'" post is filled with people calling for more empirical testing on EA messaging. So perhaps there is both more interest and intent to carry out this sort of research than I previously believed.

This is a really great idea. Thanks for organizing this and writing up the results. A couple questions:

Overall, we feel we managed to achieve deep and positive engagement with tournament participants, but haven’t yet cracked the question of how to properly engage (more shallowly) with the broader debating community.

I'm curious about the "deep" engagement aspect. You mentioned that there was the Facebook group – have people continued to engage in other ways? E.g. attending meet ups or reading this forum.

high probability that within a decade or so we can expect members of this audience to be in global positions of influence makes the community a great outreach target

Do you know of anything which has measured this? I'm imagining something similar to how 80 K analyzed predictors of becoming an MP. It seems plausible to me that top debaters are disproportionately likely to gain positions of influence, but I'm not well calibrated on how big of an effect this is.

Hi Ben, thanks for these questions.

Regarding whether we achieved “deep” engagement. We have not formally followed up with participants to be able to answer this meaningfully. I can say anecdotally that a couple of participants I know personally have since been active on EA-related Facebook groups, but I don’t know if this generalizes. We’ve also collected the contact information of participants in the study and are able to follow up with additional surveys (of those interested) in the future, exactly for analysis such as this. Also, just a minor clarification - what we meant by the original sentence was a “deep” engagement in a rather informal sense - we felt people meaningfully engaged with the material. We did not mean to claim we achieved deep engagement in a stronger sense than that, e.g. as the term is used in the model of an EA group.

Regarding analysis of how likely debaters are to reach global positions of influence - I’m not aware of any proper measurement of this. My impression that this is true comes not from looking at the distribution of careers of past debaters, but rather looking at the portion of top political leaders with experience as successful debaters, and getting the impression this is substantially higher than the portion of debaters in general (but again - just an impression, no proper analysis here). While quickly searching for any analysis on this I haven’t found anything truly reliable, but I did aggregate the examples I found in this document in case it’s helpful to anyone. I’d be interested if anyone has done or knows of more reliable data on this.

Are there recordings of the debates? I'd be interested in watching them.

I would also be interested in this.

I actually found the Facebook group very difficult to search for - link is here.