gwern on /r/machinelearning:
There's no comparison to prior full-press Diplomacy agents, but if I'm reading the prior-work cites right, this is because basically none of them work - not only do they not beat humans, they apparently don't even always improve over themselves playing the game as if it was no-press Diplomacy (ie not using dialogue at all). That gives an idea how big a jump this is for full-press Diplomacy.
There's a commentated video by someone who plays as the only human in an otherwise all-Cicero game, which at least makes it seem like the dialogue is doing a lot.
Worth noting that this was "Blitz" Diplomacy with only five-minute negotiation rounds. Still very impressive though.
Some behavioral traits such as general intelligence show very high heritability – over 0.70 – in adults, which is about as heritable as human height.
I'm very confused about what numbers such as this mean in practice, since the most natural interpretation ("70% of the trait is genetically determined") is wrong, but there aren't very many clear explanations of what the correct interpretation is. When I tried asking this on LW, the top-voted answer was that it's a number that's mostly useful if you're doing animal breeding, but probably not useful for much else.
You mention a lot of heritability numbers, could you clarify what it is that we're intended to infer from them? (It seems to me that the main thing we can infer from heritability numbers is that if a trait has heritability above zero, then there's some genetic influence on it, but since you mention some traits having "very high" heritability, I presume that you find there to be some other information too.)
I'm not sure if there is any reason that should be strongly persuasive to a disinterested third party, at the moment. I think the current evidence is more on the level of "anecdotally, it seems like a lot of rationalists and EAs get something out of things like IFS".
But given that one can try out a few sessions of a therapy and see whether you seem to be getting anything out of it, that seems to be okay? Anecdotal evidence isn't enough to strongly show that one should definitely do a particular kind of therapy. But it can be enough to elevate a therapy to the level of things that might be worth giving a shot to see if anything comes out of it.
If they are better, why haven't they been more widely adopted by mainstream medicine?
Part of it is that the effectiveness of therapy is often hard and slow to study, so it's hard to get unambiguous evidence of one being better than another. E.g. many therapists, even if working within a particular school of therapy, will apply it in an idiosyncratic style that draws upon their entire life/job experience and also knowledge of any other therapy styles they might have. That makes it hard to verify the extent to which all the therapists in the study really are doing the same thing.
My impression is that CBT got popular in part because it can be administered in a relatively standardized form, making it easy to study. But that means that it's not necessarily any better than the other therapies, it's just easier to get legible-seeming evidence on.
Another issue is that therapy can often take a long time, and lots of the quantitative measures (e.g. depression questionnaires, measures of well-being) used for measuring its effects are relatively crude and imprecise and it can be hard to know exactly what one should even be measuring.
So overall it can just be relatively hard to compare the effectiveness of therapies, and particular ones get more popular more slowly, e.g. by therapist A hearing that therapist B has been doing a new thing that seems to get better results that A does and then getting curious about it. Or by word of mouth through therapy clients when a new approach is found that seems to work better than conventional therapies, as has been happening with IFS.
Oh wow, that is a really great paper! Thank you very much for linking it.
why anyone would choose one big ritual like 'Doom circles' instead of just purposefully inculcating a culture of opennes to giving / receiving critique that is supportive and can help others?
These don't sound mutually exclusive to me; you can have a formal ritual about something and also practice doing some of the related skills on a more regular basis.
That said, for many people, it would be emotionally challenging if they needed to be ready to receive criticism all the time. A doom circle is something where you can receive feedback at such a time when you have emotionally prepared for it, and then go back to receiving less criticism normally.
It might be better if everyone was capable of always receiving it, but it's also good to have options for people who aren't quite there yet. A doom circle is a thing that can be done in less than an hour, whereas skill-building is much more of a long-term project.
low-context feedback is often not helpful;
That's true, but also: since people generally know that low-context feedback can be unhelpful, they might hold back offering any such feedback, even when it would be useful! Having an explicit context for offering the kind of critical feedback that you know might be incorrect gives people the chance to receive even the kinds of impressions that they otherwise wouldn't have the chance to hear.
feedback is ultimately just an opinion; you should be able to take and also discard it.
Yes, in any well-run doom circle, this exact thing would be emphasized in the beginning (levin mentioned this in their comment and it was probably done in the beginning of the circles I've participated in, as well; I don't remember what the exact preliminaries were, but it certainly matches my general sense of the spirit of the circle).
That's correct, most of the people in the circle (including the person with the wizard line, I think) I'd only met a couple of days before.
That's great, thank you :D