All of Rubi J. Hudson's Comments + Replies

Does the LTFF ever counter-offer with an amount that would move the grant past the funding bar for cost-effectiveness? I would guess that some of these hypothetical applicants would accept a salary at 80% of what they applied for, and if the grants are already marginal then a 25% increase in cost-effectiveness could push them over the bar.

Was Ben Pace shown these screenshots before he published his post?

With regards to #2, I shared your concern, and I thought Habryka's response didn't justify that the cost of a brief delay was sufficient if there was a realistic chance of evidence being provided to contradict the main point of this post.

However, upon reflection, I am skeptical that such evidence will be provided. Why did Nonlinear not provide at least some of the proof they claim to have, in order to justify time for a more comprehensive rebuttal? Or at least describe the form the proof will take? That should be possible, if they have specific evidence in... (read more)

Thanks for writing this up, I find quadratic funding falls into a class of mechanisms that are too clever by half in a way that makes them very fragile to the modelling assumptions. 

Also, I love how it's often posts with innocuous titles like 'Some thoughts on quadratic funding" that completely demolish an idea.

"People saying things that are mildly offensive but not worth risking an argument by calling out, and get tiring after repeated exposure" is just obviously a type of comment that exists, and is what most people mean when they say microaggression. Your paper debunking it alternates between much stricter definitions and claiming an absence of evidence for something that very clearly is going to be extremely hard to measure rigorously.

I'll edit to comment to note that you dispute it, but I stand by the comment. The AI system trained is only as safe as the mentor, so the system is only safe if the mentor knows what is safe. By "restrict", I meant for performance reasons, so that it's feasible to train and deploy in new environments.

Again, I like your work and would like to see more similar work from you and others. I am just disputing the way you summarized it in this post, because I think that portrayal makes its lack of splash in the alignment community a much stronger point against the community's epistemics than it deserves.

How does Rationalist Community Attention/Consensus compare? I'd like to mention a paper of mine published at the top AI theory conference which proves that when a certain parameter of a certain agent is set sufficiently high, the agent will not aim to kill everyone, while still achieving at least human-level intelligence. This follows from Corollary 14 and Corollary 6. I am quite sure most AI safety researchers would have confidently predicted no such theorems ever appearing in the academic literature. And yet there are no traces of any minds bei

An EA steelman example of similar points of thinking are EAs who are incredibly anti-working for OpenAI or Deepmind at all because it safety washes and pushes capabilities anyways. The criticism here is the way EA views problems means EA will only go towards solution that are piecemeal rather than transformative. A lot of Marxists felt similarly to welfare reform in that it quelled the political will for "transformative" change to capitalism. 

For instance they would say a lot of companies are pursuing RLHF in AI Safety not because it's the correct way

Thanks for the communication, and especially giving percentages. Would you be able to either break it down by grants for individuals vs. grants to organizations, or note if the two groups were affected equally? While I appreciate knowing how high the bar has risen in general, I would be particularly interested in  how high it has risen for the kinds of applications I might submit in the future.

What are your intuitions regarding length? What's the minimum time needed for a fellowship to be impactful, and at what length does it hit diminishing returns?

Seems worth asking in interviews "I'm concerned about advancing capabilities and shortening timelines, what actions is your organization taking to prevent that", with the caveat that you will be BSed.

Bonus: You can turn down roles explicitly because they're doing capabilities work, which if it becomes a pattern may incentivize them to change their plan.

This comment is object-level, perhaps nitpicky, and I quite like your post on a high level.

Saving a life via, say, malaria nets gets you two benefits:

1. The person saved doesn't die, meeting their preference for continuing to exist

2. The externalities of that person continuing to live, such as foregone grief by their family and community.

I don't think it's too controversial to say that the majority of the benefit from saving a life goes to the person whose life is saved, rather than the people who would be sad that they died. But the IDinsights survey only... (read more)

There are lots of advantages to being based in the Bay Area. It seems both easier and higher upside to solve the Berkeley real estate issue that to coordinate a move away from the Bay Area.

I love the idea of a Library of EA! It would be helpful to eventually augment it with auxiliary and meta-information, probably through crowdsourcing among EAs. Each book could also be associated with short and medium summaries of the key arguments and takeaways, and warnings about which sections were later disproven or controversial (or a warning that the whole thing is a partial story/misleading). There's also a lot of overlap and superseding within the books (especially within the rationality and epistemology section), so it would be good to say "If you'... (read more)

In addition to EAG SF, there are some other major events and a general concentration of EAs happening in this 2-week time span in the Bay Area, so it might be generally good to come to the Bay around this time. 

Which other events are happening around that time? 

their approaches are correlated with each other. They all relate to things like corrigibility, the current ML paradigm, IDA, and other approaches that e.g. Paul Christiano would be interested in.

You need to explain better how these approaches are correlated, and what an uncorrelated approach might look like. It seems to me that, for example, MIRI's agent foundations and Anthropic's prosaic interpretability approaches are wildly different!

By the time you get good enough to get a grant, you have to have spent a lot of time studying this stuff. Unpaid, mind y

I think your "digital people lead to AI" argument is spot on, and basically invalidates the entire approach. I think getting whole brain emulation working before AGI is such a longshot that the main effect of investing in it is advancing AI capabilities faster.

Hopefully one day they grow big enough to hire an executive assistant.

While I'm familiar with literature on hiring, particularly unstructured interviews, I think EA organizations should give serious consideration to the possibility that they can do better than average. In particular, the literature is  correlational, not causal, with major selection biases, and is certainly not as broadly applicable as authors claim.

From Cowen and Gross's book Talent, which I think captures the point I'm trying to make well:
> Most importantly, many of the research studies pessimistic about interviewing focus on unstructured interview... (read more)

One of the keys things you hit on is  "Treating expenditure with the moral seriousness it deserves. Even offhand or joking comments that take a flippant attitude to spending will often be seen as in bad taste, and apt to turn people off."

However, I wouldn't characterize this as an easy win, even if it would be an unqualified positive. Calling out such comments when they appear is straightforward enough, but that's a slow process that could result in only minor reductions. I'd be interested in hearing ideas for how to change attitudes more thoroughly and quickly, because I'm drawing a blank.

Cool, thanks!

I like many books on the list, but I think you're doing a disservice by trying to recommend too  many books at once. If you can cut it down to 2-3 in each category, that gives people a better starting point.

It's a bit surprising, but not THAT surprising. 50 more technical AI safety researchers would represent somewhere from a 50-100% increase  in the total number, which could be a justifiable use of 10% of OpenPhil's budget.

Thanks for the update!

Great writeup! 

Is there an OpenPhil source for "OpenPhil values a switch to an AI safety research career as +$20M in expected value"? It would help me a lot in addressing some concerns that have been brought up in local group discussions.

Even before a cost-benefit analysis, I'd like to see an ordinal ranking of priorities. For organizations like the CEA,  what would they do with a 20% budget increase? What would they cut if they had to reduce their budget by 20%? Same thing for specific events, like EAGs. For a student campus club, what would they do with $500 in funding? $2,000? $10,000? I think this type of analysis would be helpful for determining if some of the spending that appears more frivolous is actually the least important.

To clear up my identity, I am not Seán and do not know him. I go by Rubi in real life, although it is a nickname rather than my given name. I did not mean for my account to be an anonymous throwaway, and I intend to keep on using this account on the EA Forum. I can understand how that would not be obvious as this was my first post, but that is coincidental. The original post generated a lot of controversy, which is why I saw it and decided to comment.

You spoke to 20+ reviewers, half of which were sought out to disagree with you, and not a single one could

Hi Carla,

Thanks for taking the time to engage with my reply. I'd like to engage with a few of the points you made.

First of all, my point prefaced with 'speaking abstractly' was genuinely that. I thought your paper was poorly argued, but certainly within acceptable limits that it should not result in withdrawn funding. On a sufficient timeframe, everybody will put out some duds, and your organizations certainly have a track record of producing excellent work. My point was about avoiding an overcorrection, where consistently low quality work is guaranteed so... (read more)

>it's important that we don't condition funding on agreement with the funders' views.

Surely we can condition funding on the quality of the researcher's past work though? Freedom of speech and freedom of research are both important, but taking a heterodox approach shouldn't guarantee a sinecure either. 

If you completely disagree that people consistently producing bad work should not be allocated scare funds, I'm not sure we can have a productive conversation.

That puts a huge and dictatorial responsibility on funders in ways that are exactly what the paper argued are inappropriate.

If not the funders,  do you believe anyone should be responsible for ensuring harmful and wrong ideas are not widely circulated? I can certainly see the case that even wrong, harmful ideas should only be addressed by counterargument. However, I'm not saying that resources should be spent censoring wrong ideas harmful to EA, just that resources should not be spent actively promoting them. Funding is a privilege, consistently makin... (read more)

I thought the paper itself was poorly argued, largely as a function of biting off too much at once. Several times the case against the TUA was not actually argued, merely asserted to exist along with one or two citations for which it is hard to evaluate if they represent a consensus. Then, while I thought the original description of TUA was accurate, the TUA response to criticisms was entirely ignored. Statements like "it is unclear why a precise slowing and speeding up of different technologies...across the world is more feasible or effective than the sim... (read more)

Priors should matter! For example, early rationalists were (rightfully) criticized for being too open to arguments from white nationalists,  believing they should only look at the argument itself rather than the source. It isn't good epistemics to ignore the source of an argument and their potential biases (though it isn't good epistemics to dismiss them out of hand either based on that, of course).