Habryka

Project lead of LessWrong 2.0, often helping the EA Forum with various issues with the forum. If something is broken on the site, it's a good chance it's my fault (Sorry!).

Comments

Concerns with ACE's Recent Behavior

and actively campaigning against racism has nothing in common with sexual harassment.

Universal statements like this strike me as almost always wrong. Of course there are many similarities that seem relevant here, and a simple assertion that they are not doesn't seem to help the discussion.

I would really quite strongly prefer to not have comments like this on the forum, so I downvoted it. I would have usually just left it at the downvote, but i think Khorton has in the past expressed a preference for having downvotes explained, so I opted on the side of transparency. 

EA Debate Championship & Lecture Series

Finally, even after a re-read and showing your comment to two other people seeking alternative interpretations, I think you did say the thing you claim not to have said. Perhaps you meant to say something else, in which case I'd suggest editing to say whatever you meant to say. I would suggest an edit myself, but in this case I don't know what it was you meant to say.

I've edited the relevant section. The edit was simply "This is also pretty common in other debate formats (though I don't know how common in BP in particular)".

By contrast, criticisms I think mostly don't make sense:

+ Goodharting
+ Anything to the effect of 'the speakers might end up believing what they are saying', especially at top levels. Like, these people were randomly assigned positions, have probably been assigned the roughly opposite position at some point, and are not idiots.

Alas, those are indeed my primary concerns. It's of course totally OK if you are not compelled, but I have no idea how you are so confident to dismiss them. Having talked to multiple people who have participated in high-level debate in multiple formats, those are the criticisms that they level as well, including for formats very similar to BP, and for BP in-particular. 

I have now watched multiple videos of BP debate, and I wish I didn't because my guesses of what it would look like were basically right, and I feel like I wasted two hours of my time watching BP debates because you insisted that for some reason I am not allowed to make claims from very nearby points of evidence, even though as far as I can tell after spending those two hours, most of my concerns are on-point and BP looks just like most other forms of debate I've seen. 

I knew when I wrote the above comment that BP debates look less immediately goodharted. But after engaging more deeply with it, I would be surprised if it is actually much less goodharted. Of course, 4 debates at 2x speed isn't really enough to judge the whole category, but given that I've watched dozens of other debates in multiple other formats, I feel like I've pinpointed the type of debate that BP is pretty well in debate space. 

Of course, you can insist that only people intimately familiar with the format participate in this discussion, in which case I of course cannot clear that bar, and neither can almost anyone else on the forum (and this will of course heavily select against anyone who is critical of debate).


Let me take a step back. Look... I feel super frustrated by your comment above. I am trying to contribute a number of points to this discussion that feel important, and you now twice just kind of insinuated that I am unfairly biased or being unreasonable without really backing up your points? Like, your comments have been super stressful, and the associated downvotes have felt pretty bad. I think the arguments I've made in my comments are pretty straightforward, I stayed civil, and I don't think I am being particularly irrational about this topic. 

I've thought about it for 2-3 dozen hours over the years and at multiple points in the last few years have spent full-time work weeks evaluating whether we should have a debate tradition inside of EA as well, which caused me to think through a lot of the relevant considerations and investigate a substantial number of debate formats. I've talked to something like 8 people with extensive debate experience in EA for at least an hour and tried to get a sense of what things worked and what didn't. 

And then you come along and just assert: 

Overall, I'm left with the distinct impression that you've made up your mind on this based on a bad personal experience, and that nothing is likely to change that view. 

And this... just feels really unfair? Indeed, phenomenologically, my debate experiences were great. I didn't have a random bad experience that somehow soured me towards this whole sport. I was positive on it, and then thought about it for at least a dozen hours in total and overall came to a complicated high-level position that overall was a lot more hesitant. I have separately also thought for at least a literal 1000 by-the-clock-hours about our talent funnels and the epistemic norms I want the community to have and how the two interact. 

My position also isn't categorically opposed to debate at all. Indeed, I am personally likely to cause EA and the Rationality community to have more of a debating institution internally, and continue to feel conflicted about this project. I think it's quite plausible it's good, but I would want the organizers to think hard about how to avoid the problems that seem pretty deeply embedded in debate, and how to avoid them damaging the social institutions of EA, or attract people that might otherwise cause harm.

I don't know. It's fine for you to think I am being irrational about this topic, or for some reason to categorically dismiss the kind of concern that I am having, but I don't feel like you've really justified either of those assertions, and I perceived them both coming with some kind of social slap-down motion that made participating in this thread much more stressful than necessary. I will disengage for now. I hope the people involved in this project make good choices.

EA Debate Championship & Lecture Series

Following up on this, as part of me trying to understand the format of BP more, I was watching this video, which is the most watched WUDC video on Youtube. And... I find it terrifying. I find it in some sense more terrifying than the video where everyone talks super fast: 

I encourage other people who are trying to evaluate debate as a method for truth-seeking to watch this themselves. 

There is no super-fast-talking here, but all the arguments in the opening speech are terrible rhetorical argument. The speaker leverages the laughs and engagement of the audience to dismiss the position of his opponents, and this overall really felt more terrifying to me than many of the big political speeches I've seen this year. 

Like, I... think I am more terrified of the effect this would have on epistemics than the effect of the super-fast-talkers in policy debate? Like, at least in policy-debate it's somewhat obvious you are playing a game. In the above, I wouldn't be surprised if the participants actually come to believe the position they are trying to defend. 

EA Debate Championship & Lecture Series

To be clear, I think very little of my personal experience played a role in my position on this. Or at least very unlikely in the way you seem to suggest.

A good chunk of my thoughts on this were formed talking to Buck Shlegeris and Evan Hubinger at some point and also a number of other discussions about debating with a bunch of EAs and rationalists. I was actually pretty in favor of debate ~4-5 years ago when I remember first discussing this with people, but changed my mind after a bunch of people gave their perspectives and experiences and I thought more through the broader problem of how to fix it.

I also want to clarify the following 

after all, there's sadly no easy way for me to disprove your claim that a large fraction of BP debates end up not debating the topic at all..

I didn't say that. I said "This is also pretty common in other debate formats". I even explicitly said I am less familiar with BP as a debate format. It seems pretty plausible to me that BP has less of the problem of meta-debate. But I do think evidence of problems like meta-debate in other formats is evidence of BP also having problems, even if I am specifically less familiar with BP.

EA Debate Championship & Lecture Series

A lot of my models here come from talking to Evan Hubinger about this, who has a lot of thoughts on debate and competed at the national level in Policy debate in college.

My guess is that overall debate is really badly goodharted. All of the different variants. One of the ways policy-debate in particular is really badly goodharted is that everyone talks so fast nobody can comprehend them. But this is really far from the only way policy debate is broken. Indeed, a large fraction of policy debates end up not debating the topic at all, but end up being full of people debating the institution of debating in various ways, and making various arguments for why they should be declared the winner for instrumental reasons. This is also pretty common in other debate formats (though I don't know how common in BP in particular). Evan won a bunch of his highest-level debates by telling stories about pirates for 15 minutes, and then telling everyone that the institution of debate is so broken and useless and wouldn't it just be better if we would use this time to learn cool facts about history like the pirate anecdotes I told you in the first 80% of the debate?

If people were just talking fast, I do think that would be a problem, but not a super big one. I can imagine a kind of hyperoptimized court system that functions fine with everyone talking at 2-3x normal speed. The problem is that it's evidence that the system at large has very little defenses against goodharting and runaway competition effects. 

As far as I can tell, some other debate formats do not have the problem of everyone talking at 2-3x speed, but basically have many of the other problems, and importantly, all share the fundamental attribute that they have very few successful defenses against goodhart's law. And so somehow, even though it might differ from debate format to debate format, the actual thing the competitors are doing has very little to do with seeking the truth. 

The debates I participated in in high-school had nobody talking fast. But it had people doing weird meta-debate, and had people repeatedly abusing terrible studies because you can basically never challenge the validity or methodology of a study, or had people make terrible rhetorical arguments, or intentionally obfuscate their arguments until they complete it in the last minute so the opposition would have no time to respond to it.

I watched about 20 minutes of the video you linked, and I do think just from that slice it seems much less obviously broken than the video I linked, and that does seem important to recognize. 

I do also think that I can only really interface with that whole video in a healthy way by repeatedly forcing myself to take a step back and basically throw away all the information that is thrown at me, because I don't trust it, and I don't trust the process that produced it. Like, I don't think I left that video having a better understanding of seatbelt laws (the topic of debate). I learned some useful relevant facts, but I am still worried that I left that debate with worse beliefs about seatbelt laws, and very likely much worse than if I had just read the Wikipedia article on it. Of course, the point of the debate is not for me to learn about seatbelt law, but I do also think the same is probably true for the participants. 

EA Debate Championship & Lecture Series

I would also be interested in this.

EA Debate Championship & Lecture Series

I appreciate the response!

However, we think these issues are not inherent to the format. 

I do basically think the problems are kind of inherent to the format and pretty hard to fix. Like, I don't think it's physically impossible to fix these issues, but I am very skeptical of any efforts that try to fix them that stay within the existing debate context. 

Overall, I am not sure where the key causes of our disagreement lies. The above didn't really feel like it addresses my core concerns with the sport, and while the list of benefits is nice, it feels like a list that I can basically construct for an arbitrary sport (like, not this specific list of debates, but something of equal benefit), and below I give some pointers why I think some of them don't hold, or at least don't hold with the forcefulness that one might expect based on your descriptions.

I think it is fair to say that many debates do not result in figuring the truth on the topic at hand due to the complicated nature of the policies & ideas that are discussed and the limited time per discussion.

To be clear, neither the complexity of the topics or the limited time are at all anywhere close to the central reason why I think debate isn't very truth-seeking. I can totally have a meeting with a bunch of friends of mine about a complicated issue with only an hour of time, and we can easily make good approximations and solid progress on understanding it. I am quite confident we would not if we instead spent that time in any competitive debating context. Indeed, it seems likely to me that we would leave the competitive debating context with worse beliefs than we entered it on the relevant topic.

There are ample discussion groups for debaters that seek to deepen their knowledge, there is a lot of emphasis on inclusion, and extracurricular educational videos are a rising trend. Therefore, most debaters operate in ecosystems where exploring complexity is a virtue. 

None of these (inclusion, extracurricular education or "exploring complexity is a virtue) have really much to do with my concerns for debate, so at least from my perspective, describing these as trends that are gaining momentum in the debate community does very little to make me less concerned. Some of these seem mildly bad to me.

Further, since debating improves your ability to understand how arguments relate to one another, these skills can aid in figuring which position makes more sense in complicated discussions in real life, which can be helpful in seeking the truth, or at the very least in identifying falsehoods.       

I don't really think debate is really helping you understand how arguments relate to each other, at least not in a truth-tracking way. In most debate formats, it's usually much less about actually making good arguments, but much more about abusing the way judges are told to score various arguments, in a way that has very little to do with the cognitive patterns I would encourage someone to use if they were trying to figure out whether an argument makes sense or not. 

c) Debating forces you to engage with multiple perspectives. Positions in debating, i.e. being for or against the topic, are randomly allocated. This feature compels you to think on a topic in ways you might not have otherwise, and ultimately assists in developing a more nuanced world view.   

I think this is actually useful, and learning the skill of generating steelmanned-arguments for positions you don't believe is quite useful. Though because of the problem I pointed out above with the arguments that you are generating having very little to do with actual truth-trackingness, this benefit does fall quite a bit short from the ideal you describe here. 

In my experience it creates a kind of "fallacy-of-grey"-like mindset where you are avoiding having any beliefs on these issues at all, or don't really think of it being your job to actually decide which side is right, which I think is quite bad. Ultimately the goal of understanding both sides of an argument is to still judge which side is right (or of course to do a more complicated synthesis between the two, though that's I think pretty actively discouraged in the debating format).

d) The debating community is truly global. In competitions you can hear voices that are hard to find in other places. The ability to gain the perspectives of people from around the world on a plethora of important issues has benefits for those that hold EA values.

I don't really believe this? The debating community is overall really insular and narrow, as far as I can tell, being really heavily selected for being full of all the standard ivy-league people that we already have a ton of. I like cognitive diversity, but I don't really think the debating community is very exciting from that perspective. Indeed it seems to have very similar selection filters to the way the EA community is already filtered for. I might be wrong here, but I currently don't really believe that recruiting from the debate community is going to increase our cognitive diversity on almost any important dimension.

EA Debate Championship & Lecture Series

I generally want us to use truth-seeking methods when engaging with outsiders as well. Of course, that isn't always possible, but I also really don't want us to have a reputation for using lots of rhetorical tricks to convince others (and generally think that doing so is pretty bad).

EA Debate Championship & Lecture Series

I don't have time to respond in super much depth because of a bunch of competing commitments but I want to say that all of these are good points and I appreciate you making them.

EA Debate Championship & Lecture Series

I... feel pretty conflicted and hesitant about this. Overall, I have a sense that debate wasn't a healthy sport for me, and is unlikely to be healthy for others. And I guess I don't really want us to end up adopting much of its culture,  or use the techniques used in competitive debating to convince people. So I am not sure whether I think this is a good idea. 

As long as we are all just treating debate as a game that has as much relationship to figuring out the truth as playing soccer, I am of course OK with that, but I am pretty concerned that it's actually pretty hard to maintain that relationship to the sport at an event like this. 

I used to participate in debate in high school and think it taught me a lot of really bad epistemic habits that took quite a while to unlearn, and most of the American debate tradition seems even more broken than what I was used to in Germany. 

I think it's hard to understate how far away the common practices of competitive debate are from actually doing anything that helps you better understand how the world works, or has much to do with truth-seeking. And at the same time, my experience of the debate community is that they do think that they are learning valuable skills that help them better navigate the world and communicate to each other. 

Overall, my sense is that it's very hard to organize an event like this and actually have the narrative of "To be clear, please don't ever actually talk to other people in EA the way you talk to other people on stage here. I would consider that quite rude and probably actively harmful, and also, I really don't think the sport we are practicing here is helping you understand the world better, please don't take this seriously". 

Like, I would be really surprised if anything like that was said at the opening talk of this event, but like, I do think that is the actually right attitude to have towards debate if you don't want it to hurt you. I've talked to a bunch of people in the EA community who have a debating background, and all of the ones I've talked to thought that overall the habits they learned were probably bad, and the whole process really didn't have much to do with truth-seeking. 

For people who are unfamiliar with the degree to which various competitive debating practices have created to me quite horrifying abominations, here is an example of a very high-level competitive debate: 

I do think British Parliamentary Debate style is a bit less broken than this, but like, not that much. I think overall, the sport really doesn't have much to do with even just real and normal political debate, which is already a bad thing to imitate. 

This again, doesn't make me totally confident that the above is a bad project, but I do sure feel like I would warn people against attending events like this, and am pretty worried about adopting more of the competitive debating culture with things like this, and also don't super think that given how far the sport has deteriorated, that skill in it is really predictive at all in truth-seeking ability, after you control for g (and my guess is negatively correlated after you control for g, though I am less confident of that).

Load More