All of nora's Comments + Replies

If LLMs could genuinely do writing well enough to do well on the EA Forum, there are lots of positive things we could do with that.

I agree there are those things, but I am overall probably more pessimistic than you; I think there is a (significant) assymmetry towards pollution-y and not-truth-conducive content production here.

(That said, I am not too concerned overall either; I think the solution of making it harder/require some form of verification to make an account.) 

Tangentially, this made me wonder whether the ppl running EAF/LW/etc are thinking about and "ready" wrt the risk of mass-produced BS from LLMs flooding online spaces, including potentially forums like these.

2
Ozzie Gooen
1y
I'm not sure if they are, but personally, I wouldn't be too concerned. If a bunch of new accounts joined that started to produce a bunch of maybe-fake-seeming content, that seems bad but not catastrophic. If LLMs could genuinely do writing well enough to do well on the EA Forum, there are lots of positive things we could do with that.

We did not consider the discussion on specific research projects to be within the scope of this post. As mentioned in the beginning, we tried to cover as much as we could that would be relevant to other field builders and related audiences.

It primarily focusses on information we think might be relevant for other people and initiatives in this space. We also do not go into specific research outputs produced by fellows within the scope of this post

 There are a few reasons for why it made sense this way.

As discussed in other parts of this post, a lot of ... (read more)

Answer by noraSep 12, 202220
0
0

TLDR; PIBBSS is hiring for a full-time Project Manager who will be responsible for running the second iteration of the PIBBSS Summer Research Fellowship

To apply, please complete this application form

---

PIBBSS aims to facilitate knowledge transfer from fields studying intelligent behaviour in natural systems to AI safety and alignment. 

The Project Manager will be supported by TJ and Nora  (who ran the fellowship in 2022) to help transfer learnings from last year’s fellowship, and work alongside (and manage) 1-3 team members... (read more)

What is the timeline of the Century Fellowship/application? Is there a time when applications will be closed?

2
abergal
2y
We’re currently planning on keeping it open at least for the next month, and we’ll provide at least a month of warning if we close it down.

Another thought in the gendre "consequentialism+": capabilitariansim à la Senn and Nussbaumer (e.g. here (h/t TJ) for an intro or SEP) seems attractive to me (among other reasons) because I believe it makes a practically useful abstraction from "what we believe ultimately matters" to "what are the best levers to affect that which we believe ultimatly matters". (In this case, the suggestion would be, while we might still think that some broad notion of utility is what we consider to ultimately matter morally, given the specific world we live in and its caus... (read more)

nora
2y24
0
0

FWIW I would be a regular reader of Nuno's monthly (or some other interval) forum digest. Also think that having a number of other people (potentially with complementary profiles) could be valuable. Given the depth d breadth of EA/EA Forum these days, trying to find the "common denominator of relevance" in the form of a single digest will result in a digest that is of limited usefulness for most readers.

Some of the section ideas are great, in particularly "underupvoted underdogs". 

 

9
NunoSempere
2y
Thanks Nora. Note that you can in fact sign up.

PIBBSS Summer Research Fellowship -- Q&A event

  • What? Q&A session with the fellowship organizers about the program and application process. You can submit your questions here.
  • For whom? For everyone curious about the fellowship and for those uncertain whether they should apply.
  • When? Wednesday 12th January, 7 pm GMT
  • Where? On Google Meet, add to your calendar

PIBBSS Summer Research Fellowship -- Q&A event

  • What? Q&A session with the fellowship organizers about the program and application process. You can submit your questions here.
  • For whom? For everyone curious about the fellowship and for those uncertain whether they should apply.
  • When? Wednesday 12th January, 7 pm GMT
  • Where? On Google Meet, add to your calendar

Somewhat related: What to do with people? https://forum.effectivealtruism.org/posts/oNY76m8DDWFiLo7nH/what-to-do-with-people

2
akrivka
3y
I reference this in the post. A conversation with Jan along the lines of his post about EA not being talent/vetting-constrained but infrastructure-constrained has been very influential on me.
nora
3y12
0
0

Context:  (1) Motivations for fostering EA-relevant interdisciplinary research; (2) "domain scanning" and "epistemic translation" as a way of thinking about interdisciplinary research
 

List of fields/questions for interdisciplinary AI alignment research

The following list of fields and leading questions could be interesting for interdisciplinry AI alignment reserach. I started to compile this list to provide some anchorage for evaluating the value of interdiscplinary research for EA causes, specifically AI alignment. 

Some comments on the ... (read more)

nora
3y15
0
0

Below, I briefly discuss some motivating reasons, as I see them, to foster more interdisciplinary thought in EA. This includes ways EA's current set of research topics might have emerged for suboptimal reasons. 


More EA-relevant interdisciplinary research : why?


The ocean of knowledge is vast. But the knowledge commonly referenced within EA and longtermism represents only a tiny fraction of this ocean. 

I argue that EA's knowledge tradition is skewed for reasons including but not-limited-to the epistemic merit of those bodies of knowledge. There are... (read more)

3
Davidmanheim
3y
I think RAND is a good case study for interdisciplinary approaches to problem solving, though I'm biased. The key there, as in industry and most places other than academia, but unlike Santa Fe and the ARPAs, is a focus on solving concrete specific problems regardless of the tools used. Also, big +1 to cybernetics, which is an interesting case study for 2 reasons, first because of what worked, and second because of how it was supplanted / coopted into narrow disciplines, and largely fizzled out as its own thing.
nora
3y14
0
0

The below provides definitions and explanations of "domain scanning" and "epistemic translation", in an attempt of adding further gears to how interdisciplinary research works.

 

Domain scanning and epistemic translation

I suggest understanding domain scanning and epistemic translation as a specific type of research that both plays (or ought to play) an important role as part of a larger research progress, or can be usefully pursued as “its own thing”. 

 

Domain Scanning

By domain scanning, I mean the activity of searching through diverse bodies a... (read more)

I think the "so that they become more predictable [to the recommender algorithm]" is crucial in Russel's argument. IF human preferences were malleable in this way, and IF recommender algorithms are strong enough to detect that malleability, then the pressures towards the behaviour that Russel suggests is strong and we have a lot of reasons to expect it. I think the answer to both IFs is likely to be yes. 

5
Eli Rose
3y
I just don't think we've seen anything that favors the hypothesis "algorithm 'intentionally' radicalizes people in order to get more clicks from them in the long run" over the hypothesis "algorithm shows people what they will click on the most (which is often extreme political content, and this causes them to become more radical, in a self-reinforcing cycle.)"

I agree something about influence is important. As a counterpoint, I think many manifestations of "having influence" don't store well (e.g. the fact that at a given time, a relatively large number of EAs have an "influential role" (whatever that means exactly) is only weakly related to how many EAs will have an influential role in t+1 (say a generation later).  

Wrt accumulation, influence also seems less straightforward to grow when you compare it to e.g. money (and to a lesser extent to knowledge) which, thanks to interest rates, accumulates at a cer... (read more)

This made me think of the way David Deutsch talks about knoweldge creation - where knowledge manifests physically in e.g. the way a species is adapted to its niche. The process of natural selection that lead to this adapation is a process of "exploratiin" and "error correction" that accumulates knoweldge. That degree of adaptation is the physical manifestation of kowledge.  DNA is an important substrate of this process - however, I expect that DNA won't be the most fruitful level of abstraction at which to think about the patient longtermist question.... (read more)

Moral progress

I largely agree with your assessment that and how automation puts a lot of pressure on the fate of democracy (although, as you acknowledge, there are ways automation could strengthen democracy, and the way this will cash out sure seems liek it's subject to strong path dependency.) 

When we compare pre-industrial times to post-industrial times, it is not only our economy and our arsenal of technologies that is different. Within these ~200-300 years, humanity has also undergone meaningful intellectual and moral progress. This includes thing... (read more)

What plausible outside views are there? How much to rely on which?

Here is another possible outside view one could take. Under this view, the question of how societies govern themselves is subject to evolutionary dynamics. (You allude to this a bit in one of your footnotes, when talking about economic determinism.) Different societies adopt different approaches, and societies with better approaches are more successful and become more dominant. Less successful societies either cease to exist or adopt the better approaches by imitation. Based on this view, we... (read more)

If democracy retreats, what will it be replaced by?

A lot of the time, people assume a natural dichotomy between democracy and authoritative regimes. While this is certainly a useful shorthand when looking at history, I think it is likely to be misleading when thinking about the future. 

This "false dichotomy" between democracy and authoritative regimes often contrasts "my values and needs are adequately taken into account" (<> democracy) with "my values and needs basically don't matter" (<>authoritative regimes). By putting these things int... (read more)

Thanks, I enjoyed reading this. 

Here are a few thoughts; they aren’t meant as critiques of things you say, but simple thoughts triggered by, building on or attempting to complement your analysis. 

5
nora
3y
Moral progress I largely agree with your assessment that and how automation puts a lot of pressure on the fate of democracy (although, as you acknowledge, there are ways automation could strengthen democracy, and the way this will cash out sure seems liek it's subject to strong path dependency.)  When we compare pre-industrial times to post-industrial times, it is not only our economy and our arsenal of technologies that is different. Within these ~200-300 years, humanity has also undergone meaningful intellectual and moral progress. This includes things like coming to think that women and people of colour are full members of society, or spelling out values such as freedom, self-realization, etc. If automatisation will lead to power being concentrated in the hands of a small elite, this also means that the beliefs and values of this elite  become more important.  Of course, if their moral ideals are in stark contrast with others, e.g. economic interests, we should expect they will just throw most of these ideals over board or engage in elaborate rationalizations to present they are still holding them up high. But if the conflict of interst remains relatively weak, I do think this migth be a factor that palys a role. 
7
nora
3y
What plausible outside views are there? How much to rely on which? Here is another possible outside view one could take. Under this view, the question of how societies govern themselves is subject to evolutionary dynamics. (You allude to this a bit in one of your footnotes, when talking about economic determinism.) Different societies adopt different approaches, and societies with better approaches are more successful and become more dominant. Less successful societies either cease to exist or adopt the better approaches by imitation. Based on this view, we can identify "evolutionary pressures" and know some things about where these pressures are likely to steer us in the future. (Obviously we still don't know exactly where this development leads us, but the space of possible developments is in fact constrained by these co-evolutionary dynamics.)  What specifically might “fitness” look like here? Taking a perspective as roughly outlined in this paper, we could posit that in order for a species to grow ever larger in scale, it requires (what in the paper is called) information processing capacity. Democracy (or government/the policy making apparatus at large) can be viewed as essentially such an information processing technology, and thus adaptive/fitness enhancing. Given the size and complexity of present day societies, it does look like the largely top-down information processing technology of an authoritative regime would less adaptive.  One can argue that democracy is a “successful adaptation” and thus is likely stick around. Maybe this is true, but I think this argument is way harder to make than what I’ve offered above, and I’m not actually sure it stands. Reasons why this isn’t straightforward include that the evolutionary dynamics described above are not very pure (compared to “proper” Darwinian natural selection), and that the environmental conditions within which the process unfolds are changing drastically, which could for example mean that adaptations
5
nora
3y
If democracy retreats, what will it be replaced by? A lot of the time, people assume a natural dichotomy between democracy and authoritative regimes. While this is certainly a useful shorthand when looking at history, I think it is likely to be misleading when thinking about the future.  This "false dichotomy" between democracy and authoritative regimes often contrasts "my values and needs are adequately taken into account" (<> democracy) with "my values and needs basically don't matter" (<>authoritative regimes). By putting these things into the same bucket, we might overlook ways in which these connections might come apart.  For example, I might not inherently care about whether I will be able to directly or indirectly choose my political leader, but I definitely care about how well my values and needs will be taken into account in this process that steers my society into alternative futures.  Relatedly, discussions about democracy are often just as much about “democratic values'' (e.g. liberty, equality, justice) as they are about “the process of choosing our own leaders”.  I’d be curious whether your prediction about whether democracy will still be around in one thousand years largely overlaps with your prediction about, say, “will an average person in a thousand years from now feel like their values and needs are adequately taken into account by whoever or whatever is making decisions about how their society is being governed?”. (Of course, other operationalizations might be interesting, too).  The latter is much harder to predict, and democracy as you defined it might be the correct way of approaching the latter question. That said, understanding more about how lieky they are to come apart, and if so how seems potentially interesting. 
nora
3y20
0
0

Patient Longtermism as a benchmark

Meta: I haven’t seen this framing spelt out in these terms and think it’s a useful way of integrating considerations raised by patient longtermism into one overall EA worldview.

The considerations elucidated by patient longtermism, namely that our resources can “go further” in the future, are important. There is an analogous here to Singer’s drowning child argument, which says that, all else equal, you shouldn’t have a preference over helping someone who is spatially close to you compared to someone who is spatially far awa... (read more)