Jenny K E

Topic Contributions

Comments

Bad Omens in Current Community Building

Thanks so much for writing this. As someone interested in starting to do community building at a university, this was helpful to read, especially the Alice/Bob example and the concrete advice. I do really think that EA could stand to be less big on recruiting HEAs. I think there are tons of people who are interested in EA principles but aren't about to make a career switch, and it's important for those people to feel welcome and like they belong in the community.

I was going to write "I kind of wish this post (or a more concise version) were required reading for community builders," and then I thought better of it and took actions about it -- namely, sent the link as feedback to the EA Student Group Handbook and made an argument that they should incorporate something like this into their guide for student groups.

What are the coolest topics in AI safety, to a hopelessly pure mathematician?

Elaborating on this, thanks to Spencer Becker-Kahn for prompting me to think about this more:

From a standpoint of my values and what I think is good, I'm an EA. But doing intellectual work, specifically, takes more than just my moral values. I can't work on problems I don't think are cool. I mean, I have, and I did, during undergrad, but it was a huge relief to be done with it after I finished my quals and I have zero desire to go back to it. It would be -- at minimum unsustainable -- for me to try to work on a problem where my main motivation for doing it is "it would be morally good for me to solve this." I struggle a bit with motivation at the best of times, or rather, on the best of problems. So, if I can find something in AI safety that I think is approximately as cool as what I'm currently doing, I'll do it, but the coolness is actually a requirement, because I won't be successful or happy otherwise. I'm not built for it (and I think most EAs aren't; fortunately some of them have different tastes than I do, as to what is or isn't cool).

 

What are the coolest topics in AI safety, to a hopelessly pure mathematician?

Thanks very much for the suggestions, I appreciate it a lot! Zoom In was a fun read -- not very math-y but pretty cool anyway. The Transformers paper also seems kind of fun. I'm not really sure whether it's math-y enough for me to be interested in it qua math...but in any event it was fun to read about, which is a good sign. I guess "degree of mathiness" is only one neuron of several neurons sending signals to the "coolness" layer, if I may misuse metaphors.

What are the coolest topics in AI safety, to a hopelessly pure mathematician?

The point about checking back in every now and then is a good one; I had been thinking in more binary terms and it's helpful to be reminded that "not yet, maybe later" is also a possible answer to whether to do AI safety research.

I like logic puzzles, and I like programming insofar as it's like logic puzzles. I'm not particularly interested in e.g. economics or physics or philosophy. My preferred type of problem is very clear-cut and abstract, in the sense of being solvable without reference to how the real world works. More "is there an algorithm with time complexity Y that solves math problem X" than "is there a way to formalize real-world problem X into a math problem for which one might design an algorithm." Unfortunately AI safety seems to be a lot of the latter!

What are the coolest topics in AI safety, to a hopelessly pure mathematician?

These are the sort of thing I'm looking for! In that, on first glance, they're a lot of solid "maybe"s where mostly I've been finding "no"s. So that's encouraging --thank you so much for the suggestions!

What are the coolest topics in AI safety, to a hopelessly pure mathematician?

I am not intellectually motivated by things on the basis of their impactfulness. If I were, I wouldn't need to ask this question.

What are the coolest topics in AI safety, to a hopelessly pure mathematician?

Absolutely agree with everything you've said here! AI safety is by no means the only math-y impactful work.

Most of these don't quite feel like what I'm looking for, in that the math is being used to do something useful or valuable but the math itself isn't very pretty. "Racing to the Precipice" looks closest to being the kind of thing I enjoy.

Thank you for the suggestions!

What are the coolest topics in AI safety, to a hopelessly pure mathematician?

Your points (1) and (2) are ones I know all too well, though it was quite reasonable to point them out in case I didn't, and they may yet prove helpful to other readers of this post.

Regarding Vanessa Kosoy's work, I think I need to know more math to follow it (specifically learning theory, says Ben; for the benefit of those unlucky readers who are not married to him, he wrote his answer in more detail below). I did find myself enjoying reading what parts of the post I could follow, at least.

Regarding the Topos Institute, someone I trust has a low opinion of them; epistemic status secondhand and I don't know the details (though I intend to ask about it).

Thanks very much for the suggestions!

What are the coolest topics in AI safety, to a hopelessly pure mathematician?

Ah, that's a good way of putting it! I'm much more of a "problem solver."

What are the coolest topics in AI safety, to a hopelessly pure mathematician?

The second and third strike me as useful ideas and kind of conceptually cool, but not terribly math-y; rather than feeling like these are interesting math problems, the math feels almost like an afterthought. (I've read a little about corrigibility before, and had the same feeling then.) The first is the coolest, but also seems like the least practical -- doing math about weird simulation thought experiments is fun but I don't personally expect it to come to much use.

Thank you for sharing all of these! I sincerely appreciate the help collecting data about how existing AI work does or doesn't mesh with my particular sensibilities.

Load More