I'm having an ongoing discussion with a couple professors and a PhD candidate in AI about "The Alignment Problem from a Deep Learning Perspective" by @richard_ngo, @Lawrence Chan, and @SoerenMind. They are skeptical of "3.2 Planning Towards Internally-Represented Goals," "3.3 Learning Misaligned Goals," and "4.2 Goals Which Motivate Power-Seeking Would Be Reinforced During Training". Here's my understanding of some of their questions:
- The argument for power-seeking during deployment depends on the model being able to detect the change from the training to deployment distribution. Wouldn't this require keeping track of the distribution thus far, which would require memory of some sort, which is very difficult to implement in the SSL+RLHF paradigm?
I don't see why it would require memory, because the model will have learned to recognize features of its training distribution. So this seems like this just requires standard OOD detection/anomaly detection. I'm not familiar with this...
General information about people in low-HDI countries to humanize them in the eyes of the viewer.
Similar for animals (except not “humanizing” per se!). Spreading awareness that e.g. pigs act like dogs may be a strong catalyst for caring about animal welfare. Would need to consult an animal welfare activism expert.
My premise here: it is valuable for EAs to viscerally care about others (in addition to cleverly working toward a future that sounds neat).
I'll just continue my anecdote! As it happens, the #1 concern that my friend has about EA is that EAs work sinisterly hard to convince people to accept the narrow-minded longtermist agenda. So, the frequency of ads itself increases his skepticism of the integrity of the movement. (Another manifestation of this pattern is that many AI safety researchers see AI ethics researchers as straight-up wrong about what matters in the broader field of AI, and therefore need to be convinced rather than collaborated with.)
(Edit: the above paragraph is an anecdote, and ...
Thanks very much, that helps!
Adding more not to defend myself, but to keep the conversation going:
I think that many Enlightenment ideas are great and valid regardless of their creators' typical-for-their-time ideas.
Education increasingly includes rather radical components of critical race theory. Students are taught that if someone is racist, then all of their political and philosophical views are tainted. By extension, many people learn that the Enlightenment itself is tainted. Like Charles, I think that this "produces misguided perspectives".
I'm--a...
Yes, these are great reasons to take inspiration from the Enlightenment!
The point I most want to get across is that, by using Enlightenment aesthetics, EAs could needlessly open themselves up to negative perception.
Yeah, the magnitude of the problem depends on the empirical question of how many people associate the Enlightenment with racism and such.
Descartes’ moral circle issue is that he believed animals have no moral standing whatsoever, so he enthusiastically practiced vivisection (dissecting animals while they were still alive).
The Enlightenment led to good foundational ideas of EA, but it was also full of philosophers who ... excluded pretty much everybody except for white men from the moral circle, and advocated for constant growth with no regard for sustainability (e.g. Immanuel Kant, Rene Descartes, Adam Smith).
I think this is pretty unfair. In general I think we should judge historical figures by the ways they were unusual for their period, not the ways they were typical, but in this case we don't even need to make this distinction. Here is Adam Smith on slavery for ex...
I'm more inspired by the "altruistic" aesthetic than the "effective" aesthetic.
"Effective" blends into the Silicon Valley productivity/efficiency crowd. While there's a lot to appreciate about the Bay Area, I'd prefer not to tie EA to that culture.
On the other hand, there are truly beautiful exemplars of altruism throughout history and around the world.
Personally, I associate altruism with Avalokiteśvara. Art portraying him is colorful and full of details, which, to me, represents that Effective Altruism can bridge all kinds of cultures, theories, an...
Adding on: Increasing EA spending in certain areas could certainly support diversity, but it could have the opposite effect elsewhere.
I’m concerned that focusing community-building efforts at elite universities only increases inequality. I’m guessing that university groups do much of the recruiting for all-expenses-paid activities. In practice, then, students at elite universities will benefit, while students at state schools and community colleges won’t even hear about these opportunities. So the current EA community-building system quite accurately selects for privileged students to give money to.
Curious about any work to change this pattern!
Thanks for the great idea!
Here's an email script summarizing this article. I wrote it in ~5 minutes to send to my US Congressional representative, so it's not very polished, but I think it's good enough.
Hi! I'd like to encourage Rep. ___ to advocate for opening borders to Russians as much as possible. Any simplification of the visa process will help. This will weaken Russia and its onslaught on Ukraine while also strengthening our economy.
...First, Russian men who don't want to fight would avoid conscription or desert the army by immigrating to the US w
"Strong Towns is an international movement dedicated to making communities across the United States and Canada financially strong and resilient." Advocates for friendly human-scale dense cities over car-centric suburbia. I learned about Strong Towns through the similarly educational YouTube channel Not Just Bikes.
Of course, this is relevant to global development work. And I feel better qualified to vote/advocate for local urban planning.
Finally, from a rationalist perspective, it was fascinating to watch my mind change as I understood how my American surroundings were built for cars, not people.
I'd love to read! Female American voice here. I'm a trained singer, but not a trained voice actor. I have a Tascam DR-05 and might be able to finagle access to a recording studio.
This is great timing. I'm currently in the middle of reading Significant Digits aloud. Just this past week, I realized that voice acting is a ton of fun and I'd like to contribute to a project :)
Thanks for organizing, Fin!
I'm currently researching the related topic of the compassion-oriented Buddhist spiritual path, so my response will be from that perspective. Feel free to DM me if you want to chat.
John Makransky, of Boston College and Kathmandu University, has done great work on this question. He adapts Tibetan Buddhist practices for a secular Western context. See "Compassion Without Fatigue: Contemplative Training for People who Serve Others" (third link from the top). The main insight for me is that I am not alone in trying to alleviate suffering--so many people t...
I highly recommend the Bodhicaryavatara by Shantideva! It's the most significant ethical text of the Mahayana Buddhist tradition, with some serious Madhyamaka metaphysics sprinkled in. I'm currently writing my undergrad thesis on it, and I'd be happy to talk about it.
Here's a great guide: https://www.shambhala.com/guide-to-the-way-of-the-bodhisattva/. I took an intensive course on the Bodhicaryavatara in the traditional monastic style in Kathmandu, Nepal; see https://ryi.org/programs/degree-programs if you really want to dive deep. The school is currently ...
Why should I donate to international poverty relief when these people would just have more kids (contributing to overpopulation) and not do anything good in the world? Shouldn't I donate to scholarship funds for local college students instead, since they're more likely to make a difference?
(I suspect this is a common line of reasoning among well-off educated white people in wealthy countries who think people in third-world countries are selfish and unambitious, but won't say that outright.)
Absolutely, I hear this all the time. Here's some anecdotal advice:
In particular, there's a strong thread in my circles that privileged people need to give up their power (for example, this was recently posted in the math Discord server at my left-leaning university), and philanthropy allows privileged people to hold onto power while feeling good about themselves. Social justice folks and EAs agree that everyone is complicit in injustice, and we should each take life-changing steps to help. The difference is that EAs claim that throwing away one's power is...
Here's a compilation of ideas from 2015 called "What Can A Technologist Do About Climate Change?": http://worrydream.com/ClimateChange/
Hi! Thanks for this new way to get career advice.
I'd greatly appreciate ideas for where my skill set could be most useful.
My dream job would be some sort of research role at the intersection of philosophy, math, computer science, and religious studies. Lately, I've been curious about the risks of demographic shift toward religious fundamentalists.
What steps could I take toward a role like this? Where can I find EAs interested in the future religious landscape? Has there already been discussion in EA circles about the demographic shift toward fundamentalism...
As someone dubiously planning a career affiliated with the U.S. Department of Defense, I would really appreciate an analysis of working inside and outside of The System. Historically, have altruists been able to do good from within harmful governments (fascist dictatorships, military juntas, genocidal governments, etc.)? How? Which qualities do altruism-friendly systems have?
"I only ask of God
That I am not indifferent to the pain,
That the dry death won’t find me
Empty and alone, without having done the sufficient."
from https://lyricstranslate.com/en/Solo-le-pido-Dos-I-only-ask-God.html
"But those who fill with bliss
All beings destitute of joy,
Who cut all pain and suffering away
From those weighed down with misery,
Who drive away the darkness of their ignorance—
What virtue could be matched with theirs?
What friend could be compared with them?
What merit is there similar to this?"
..."The great should never be abandoned for the less
I can't resist mentioning that Mahayana Buddhism considers meditation to be an altruistic act because it fosters wisdom and compassion. Sam Harris' Waking Up app is particularly great at taking meditation seriously; plus, the company has taken the Giving What We Can pledge.
Many charities and hospitals accept knitted and crocheted donations, and they usually prefer super-affordable acrylic. When I was learning to knit and crochet as a little kid, I donated a lot of preemie- and newborn-sized hats. The great thing about these crafts is that they can be either easy and meditative or creative and engaging.
In the spirit of Aaron Gertler's expansion on calling elderly relatives, we can extend "feeding stray cats" to spending time with animals. This can be as small as giving some extra attention to local animals--in my case, I like to hang out with the cows and sheep at my university who are destined to become meat--or as significant as volunteering at a farm sanctuary.
Here's a looking-at-the-bright-side sort of progress:
I've been bewildered for most of this year about why I'm struggling so much to get things done. Just 2020-related stress doesn't explain it.
Well, I think I've figured out that I'm just really burned out (or, as Cal Newport puts it, in a state of "deep procrastination").
...in one of my two majors! So, I've changed the burned-out major to a minor. Now I'll graduate in just a few months, giving me more time to learn things and explore career options (which I'm suddenly more excited about).
My path ahead isn't exactly straightforward, but at least I've gained some valuable knowledge about what it could look like.
This seems like an incredibly interesting and important discussion! I don't have much time now, but I'll throw in some quick thoughts and hopefully come back later.
I think that there is room for Romy and Paolo's viewpoint in the EA movement. Lemme see if I can translate some of their points into EA-speak and fill in some of their implicit arguments. I'll inevitably use a somewhat persuasive tone, but disagreement is of course welcome.
(For context, I've been involved in EA for about six years now, but I've never come across an...
Ego depletion is quite a narrow psychological effect. If the idea that people's moment to moment fatigue saps moment to moment willpower is debunked, that's far from showing that akrasia isn't a thing in general.
In a world where general-sense akrasia was not a thing there would be a far higher rate of people being ripped like movie stars, a far lower rate of smoking, a much high rate of personal savings etc than there is in the world we inhabit.
Great, this is exactly the sort of response I was hoping for!
I do not have a personal connection to the award, and I don't know how many charities were nominated last year. I plan to nominate the organization that stands out in this discussion (thus far J-PAL). The website doesn't mention any kind of voting system, so one nomination should suffice.
Frustratingly, I think the requirement that the organization must serve North American women rules out ACE, SCI, CFAR, and Encompass. J-PAL may have a chance.
How about reducing the number of catered meals while increasing support for meals outside the venue? Silly example: someone could fill a hotel room with Soylent so that everyone can grab liquid meals and go chat somewhere--sort of a "baguettes and hummus" vibe. Or as @Matt_Sharp pointed out, we could reserve nearby restaurants. No idea if these exact plans are feasible, but I can imagine similarly scrappy solutions going well if planned by actual logistics experts.
Thanks so much for your work and this information!