This link works for me:
https://openai.com/form/preparedness-challenge
(Just without period at the end)
Hey! I applied end of april and haven't received any notification like this nor a rejection and I'm not sure what this means about the status of my application. I emailed twice over the past 4 months, but haven't received a reply :/
Most of the researchers at GPI are pretty sceptical of AI x-risk.
Not really responding to the comment (sorry), just noting that I'd really like to understand why these researchers at GPI and careful-thinking AI alignment people - like Paul Christiano - have such different risk estimates! Can someone facilitate and record a conversation?
David Thorstadt, who worked at GPI, Blogs about reasons for his Ai skepticism (and other EA critiques) here https://ineffectivealtruismblog.com/
The object-level reasons are probably the most interesting and fruitful, but for a complete understanding of how the differences might arise, it's probably also valuable to consider:
An interesting exercise could be to go through the categories and elucidate 1-3 reasons in each category for why AI alignment people might believe X and cause prio people might believe not X.
I find it remarkable how little is being said about concrete mechanisms for how advanced AI would destroy the world by the people who most express worries about this. Am I right in thinking that? And if so, is this mostly because they are worried about infohazards and therefore don't share the concrete mechanisms they are worried about?
I personally find it pretty hard to imagine ways that AI would e.g. cause human extinction that feel remotely plausible (allthough I can well imagine that there are plausible pathways I haven't thought of!)
Relatedly, I wonde...
It seems a lot of people are interested in this one! For my part, the answer is "Infohazards kinda, but mostly it's just that I haven't gotten around to it yet." I was going to do it two years ago but never finished the story.
If there's enough interest, perhaps we should just have a group video call sometime and talk it over? That would be easier for me than writing up a post, and plus, I have no idea what kinds of things you find plausible and implausible, so it'll be valuable data for me to hear these things from you.
This 80k article is pretty good, as is this Cold Takes post. Here are some ways an AI system could gain power over humans:
I agree, and I actually have the same question about the benefits of AI. It all seems a bit hand-wavy, like 'stuff will be better and we'll definitely solve climate change'. More specifics in both directions would be helpful.
I wonder to what extent people take the alignment problem to be the problem of (i) creating an AI system that reliaby does or tries to do what its operators want it do as opposed to (ii) the problem of creating an AI system that does or tries to do what is best "aligned with human values" (whatever this precisely means).
I see both definitions being used and they feel importantly different to me: if we solve the problem of aligning an AI with some operator, then this seems far away from safe AI. In fact, when I try to imagine how an AI might cause a catastr...
I'm pretty late to the party (perhaps even so late that people forgot that there was a party), but just in case someone is still reading this, I'll leave my 2 cents on this post.
[Context: A few days ago, I released a post that distils a paper by Kenny Easwaran and others, in which they propose a rule for updating on the credences of others. In a (tiny) nutshell, this rule, "Upco", asks you to update on someones credence in proposition A by multiplying your odds with their odds.]
1. Using Upco suggests some version of strong epistemic modesty: wh...
Thanks, this seems useful! :) One suggestion: if there are similar estimates available for other causes, could you add at least one to the post as a comparison? I think this would make your numbers more easily interpretable.
Hey Daniel,
thanks for engaging with this! :)
You might be right that the geometric mean of odds performs better than Ucpo as an updating rule although I'm still unsure exactly how you would implement it. If you used the geometric mean of odds as an updating rule for a first person and you learn the credence of another person, would you then change the weight (in the exponent) you gave the first peer to 1/2 and sort of update as though you had just learnt the first and second persons' credence? That seems pretty cumbersome as you'd have to keep track o...
Thanks a lot for the update! I feel excited about this project and grateful that it exists!
As someone who stayed at CEEALAR for ~6 months over the last year, I though I'd share some reflections that might help people decide whether going to the EA Hotel is a good decision for them. I'm sure experiences vary a lot, so, general disclaimer, this is just my personal data point and not some broad impression of the typical experience.
Some of the best things that happened as a result of my stay:
Why is there so much more talk about the existential risk from AI as opposed to the amount by which individuals (e.g. researchers) should expect to reduce these risks through their work?
The second number seems much more decision-guiding for individuals than the first. Is the main reason that its much harder to estimate? If so, why?
(Hastily written, sry)
I would love to see more of the theories of change that researchers in EA have for their own career! I'm particularly interested to see them in Global Priorities Research as its done at GPI (because I find that both extremely interesting and I'm very uncertain how useful it is apart from field-building).
Two main reasons:
How do EA grantmakers take expert or peer opinions on decision-relevant claims into account? More precisely, if there's some claim X that's crucial to an EA grantmakers' decision and probabilistic judgements from others are available on X (e.g. from experts) -- how do EA grantmakers tend to update on those judgements?
Motivation: I suspect that in these situations it's common to just take some weighted average of the various credences and use that as one's new probability estimate. I have some strong reasons to think that this is incompatible with bayesian updating (post coming soon).
I wonder if it would be good to create another survey to get some data not only on who people update on but also on how they update on others (regarding AGI timelines or something else). I was thinking of running a survey where I ask EAs about their prior on different claims (perhaps related to AGI development), present them with someone's probability judgements and then ask them about their posterior. That someone could be a domain expert, non-domain expert (e.g., professor in a different field) or layperson (inside or outside EA).
At least if ...
Cool idea to run this survey and I agree with many of your points on the dangers of faulty deference.
A few thoughts:
(Edit: I think my characterisation of what deference means in formal epistemology is wrong. After a few minutes of checking this, I think what I described is a somewhat common way of modelling how we ought to respond to experts)
The use of the concept of deference within the EA community is unclear to me. When I encountered the concept in formal epistemology I remember "deference to someone on claim X" literally meaning (a) that you adopt t
However, even if we'd show that the repugnance of the repugnant conclusion is influenced in these ways or even rendered unreliable, I doubt the same would be true for the "very repugnant conclusion":
for any world A with billions of happy people living wonderful lives, there is a world Z+ containing both a vast amount of mildly-satisfied lizards and billions of suffering people, such that Z+ is better than A.
(Credit to joe carlsmith who mentioned this on some podcast)
Thanks for the post!
I'm particularly interested in the third objection you present - that the value of "lives barely worth living" may be underrated.
I wonder to what extent the intuition that world Z is bad compared to A is influenced by framing effects. For instance, if I think of "lives net positive but not by much", or something similar, this seems much more valueable than "lives barely worth living", allthough it means the same in population ethics (as I understand it).
I'm also sympathetic to the claim that ones response to world Z may be affected by o...
I agree that it seems like a good idea to get somewhat familiar with that literature if we want to translate "longtermism" well.
I think I wouldn't use "Langzeitethik" as this suggests, as you say, that longtermism is a field of research. In my mind, "longtermism" typically refers to a set of ethical views or a group of people/institutions. Probably people sometimes use the term to refer to a research field, but my impression is that this rather rare. Is that correct? :)
Also, I think that a new term - like "Befürworter der Langzeitverantwortung" - which is ...
This seems a bit inaccurate to me in a few ways, but I'm unsure how accurate we want to be here.
First, when the entry talks about "consequentialism" it seems to identify it with a decision procedure: "Consequentialists are supposed to estimate all of the effects of their actions, and then add them up appropriately". In the literature, there is usually a distinction made between consequentialism as a criterion of rightness and a decision procedure, and it seems to me like many endorse the latter and not the former.
Secondly, it seems to identify ...
Red team: is it actually rational to have imprecise credences in possible longrun/indirect effects of our actions rather than precise ones?
Why: my understanding from Greaves (2016) and Mogensen (2020) is that this has been necessary to argue for the cluelessness worry.
Thanks! :) And great to hear that you are working on a documentary film for EA, excited to see that!
Re: EA-aligned Movies and Documentaries
I happen to know a well-established documentary filmmaker, whos areas of interest overlap with EA topics. I want to pitch him to work on a movie about x-risks. Do you have any further info about the kinds of documentaries you'd like to fund? Anything that's not obvious from the website.
Hey! I wonder how flexible the starting date is. My semester ends mid-July, so I couldn't start before. This is probably the case for most students from Germany. Is that too late?
Thanks for the post!
Does this apply at all to undergrads or graduate students who haven't published any research yet?
Hey Pablo,
Thanks a lot for the answer, I appreciate you taking the time! I think I now have a much better idea of how these calculations work (and much more skeptical tbh because there are so many effects which are not captured in the expected value calculations that might make a big difference).
Also thanks for the link to Holdens post!
Hi Johannes!
I appreciate you taking the time.
"Linch's comment on FP funding is roughly right, for FP it is more that a lot of FP members do not have liquidity yet"
I see, my mistake! But is my estimate sufficiently off to overturn my conclusion?
" There were also lots of other external experts consulted."
Great! Do you agree that it would be useful to make this public?
"There isn't, as of now, an agreed-to-methodology on how to evaluate advocacy charities, you can't hire an expert for this."
And the same ist true for evaluating cost-effectiven...
..."The way I did my reviewing was to check the major assumptions and calculations and see if those made sense. But where a report, say, took information from academic studies, I wouldn't necessarily delve into those or see if they had been interpreted correctly. "
>> Thanks for clarifying! I wonder if it would be even better if the review was done by people outside the EA community. Maybe the sympathy of belonging to the same social group and shared, distinctive assumptions (assuming they exist), make people less likely to spot errors? This is pret
Hi Michael!
"You only mention Founders Pledge, which, to me, implies you think Founders Pledge don't get external reviews but other EA orgs do."
> No, I don't think this, but I should have made this clearer. I focused on FP, because I happened to know that they didn't have an external, expert review on one of their main climate-charity recommendations, CATF and because I couldn't find any report on their website about an external, expert review.
I think my argument here holds for any other similar organisation.
"This doesn't seem right, because ...
I'm not sure, but according to Wikipedia, in total ~3 billion dollars have been pledged via Founders Pledge. Even if that doesn't increase and only 5% of that money is donated according to their recommendations, we are still in the ballpark of around a hundred million USD right?
On the last question I can only guess as well. So far around 500 million USD have been donated via FoundersPledge. Founders Pledge exists for around 6 years, so on average around 85 million $ per year since it started. It seems likely to me that at least 5% have been allocated...
I actually think there is more needed.
If “its a mistake not to do X” means “its in alignment with the persons goal to do X”, then I think there are a few ways in which the claim could be false.
I see two cases where you want to maximize your contribution to the common good, but it would still be a mistake (in the above sense) to pursue EA:
I'd say that pursuing the project of effective altruism is worthwhile, only if the opportunity cost of searching C is justified by the amount of additional good you do as a result of searching for better ways to do good, rather then go by common sense A. It seems to me that if C>= A, then pursuing the project of EA wouldn't be worth it. If, however, C< A, then pursuing the project of EA would be worth it, right?
To be more concrete let us say that the difference in value between the commonsense distribution of resources to do good and th...
Do you still recommend these approaches or has your thinking shifted on any? Personally, I'd be especially interested if you still recommend to "Produce a shallow review of a career path few people are informed about, using the 80,000 Hours framework. ".
Hey, thank you very much for the summary!
I have two questions:
(1) how should one select which moral theories to use in ones evaluation of the expected choice worthiness of a given action?
"All" seems impossible, supposing the set of moral theories is indeed infinite; "whatever you like" seems to justify basically any act by just selecting or inventing the right subset of moral theories; "take the popular ones" seems very limited (admittedly, I dont have an argument against that option, but is there a positive one for it?)
(2)...
Very similar for me!