All of jacobjacob's Comments + Replies

Apply to the ML for Alignment Bootcamp (MLAB) in Berkeley [Jan 3 - Jan 22]

Not addressing video recordings specifically; but we might run future iterations of this bootcamp if there's enough interest, it goes well and it continues to seem valuable. So feel free to submit the application form while noting you're only interested in future cohorts. 

2018-2019 Long-Term Future Fund Grantees: How did they do?

I was a grantee in 2019 and consent to having my evaluation shared publicly, if Nuno wants to post it. 

4Linch1y
I assumed that you'd be one of the people in the bucket, fwiw.
2018-2019 Long-Term Future Fund Grantees: How did they do?

To really make this update, I'd want some more bins than the ones Nuno provide. That is, there could be an "extremely more successful than expected" bin; and all that matters is whether you manage to get any grant in that bin. 

(For example, I think Roam got a grant in 2018-2019, and they might fall in that bin, though I haven't thought a lot about it.) 

4NunoSempere1y
I would also want more bins than the ones I provide, i.e., not considering the total value is probably one of the parts I like less about this post.
4Max_Daniel1y
Yeah I agree that info on how much absolute impact each grant seems to have had would be more relevant for making such updates. (Though of course absolute impact is very hard to estimate.) Strictly speaking the info in the OP is consistent with "99% of all impact came from one grant", and it could even be one of the "Not as successful as hoped for". (Though taking into account all context/info I would guess that the highest-impact grants would be in the bucket "More successful than expected".) And if that was the case one shouldn't make any updates that would be motivated by "this looks less heavy-tailed than I expected".
How Dependent is the Effective Altruism Movement on Dustin Moskovitz and Cari Tuna?

Counterpoint: yes, Facebook has lots of public image issues. As a result, we have good evidence that they're an org that's unusually resistant to such problems!

They've been having scandals since they were founded. And in spite of all the things you mention, their market cap has almost doubled since the bottom of the Cambridge Analytica fall-out.

They're also one of the world's most valuable companies, and operate in a sector (software) that on an inside view seems well poised to do well in future (unlike, say, Berkshire Hathaway, ... (read more)

Delegate a forecast

Conditional on OpenAI API generating at least $100M in total revenue for OpenAI, by what year will that happen?

(You might also want to combine this with an estimate of the binary variable of whether it will generate $100M in revenue at all.)

This was really interesting to forecast! Here's my prediction, and my thought process is below. I decomposed this into several questions:

  • Will OpenAI commercialize the API?
    • 94% – this was the intention behind releasing the API, although the potential backlash adds some uncertainty [1]
  • When will OpenAI commercialize the API? (~August/September)
    • They released the API in June and indicated a 2 month beta, so it would begin generating revenue in August/September [2]
  • Will the API reach $100M revenue? (90%)
... (read more)
[Job ad] Lead an ambitious COVID-19 forecasting project [Deadline: June 1st]

I'm also posting a bounty for suggesting good candidates: $1000 for successful leads on a new project manager; $100 for leads on a top 5 candidate

DETAILS

I will pay you $1000 if you:

  • Send us the name of a person…
  • …who we did not already have on our list…
  • …who we contacted because of your recommendation...
  • ...who ends up taking on the role

I will pay you $100 if the person ends up among the top 5 candidates (by our evaluation), but does not take the role (given the other above constraints).

There’s no requirement for you to ... (read more)

Seeking a CEO for new x-risk funding charity in the UK

Ought (~$5000) and Rethink Priorities (~$500) have both done it, with bounties roughly what I indicated (though I'm a bit uncertain). Don't think either has completed the relevant hiring rounds yet.

Concerning the Recent 2019-Novel Coronavirus Outbreak

In addition, I'll mention:

Attempted summary of the 2019-nCoV situation — 80,000 Hours

Here's a list of public forecasting platforms where participants are tracking the situation:

Foretold is tracking ~20 questions and is open to anyone adding their own, but doesn't have very many predictions.

Metaculus is tracking a handful questions and has a substantial number of predictions.

The John Hopkins disease prediction project lists 3 questions. You have to sign up to view them. (I also think you can't see the crowd average before you've made your prediction.)

2JimmyJ3y
Could you please provide the JHU questions and predictions for those of us who don't want to sign up?
[Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form.

This set-up does seem like it could be exploitable in an adversarial manner... but my impression from reading the poll results, is that this is weak evidence against that actually being a failure mode -- since it doesn't seem to have happened.

I didn't notice any attempts to frame a particular person multiple times. The cases where there were repeated criticism of some orgs seemed to plausibly come from different accounts, since they often offered different reasons for the criticism or seemed stylistically different.

Moreover, if asked beforehand ... (read more)

[Link] What opinions do you hold that you would be reluctant to express in front of a group of effective altruists? Anonymous form.
If there were a way to do this with those opinions laundered out, then I wouldn't have a problem.

I interpret [1] you here as saying "if you press the button of 'make people search for all their offensive and socially disapproved beliefs, and collect the responses in a single place' you will inevitably have a bad time. There are complex reasons lots of beliefs have evolved to be socially punished, and tearing down those fences might be really terrible. Even worse, there are externalities such that one person saying something crazy is goi... (read more)

[Part 1] Amplifying generalist research via forecasting – models of impact and challenges
Why do you think this is better than encouraging people to join foretold.io as individuals? Do you think that we are lacking an institution or platform which helps individuals to get up to speed and interested in forecasting (so that they are good enough that foretold.io provides a positive experience)?

I'm not sure if the group should fully run the tournaments, as opposed to just training a local team, or having the group leader stay in some contact with tournament organisers.

Though I have an intuition that some support from a local group might make ... (read more)

Why do social movements fail: Two concrete examples.

A while back me and habryka put up a bounty for people to compile a systematic list of social movements and their fates, with some interesting results. You can find it here.

6NunoSempere3y
Curiously enough, 'twas me who fulfilled the bounty.
Could we solve this email mess if we all moved to paid emails?
How would this be an "internal practice"? The only way this would work would be to have people publically post their earn addresses.

"Internal" in the sense of being primarily intended to solve internal coordination purposes and primarily used in messaging within the community.

I think you underrate the cost of weirdness.

You gave a particular example of a causal pathway by which weirdness leads to bad stuff, but it doesn't really cause me to change my mind because I was already aware of it as a failure mode. What makes you think I u... (read more)

Could we solve this email mess if we all moved to paid emails?
It's not clear to me that we are in a mess.

Well, that's why I'm posting this -- to get some data and find out :)

(I guess the title seemed to have turned a few people off, though)

In hindsight, I should have made the intended use-cases clearer in the post. I optimised for shipping it fast rather than not at all, but that had its costs.

The reason I wrote this was basically entirely motivated by problems I've encountered myself.

For example, I’ve spent this year trying to build an AI forecasting community, and faced the awkward problem ... (read more)

6richard_ngo3y
This all makes sense, and it does seem that people who are launching big projects might benefit from paid emails as a norm. On the other hand, you seem unusually worried about "spamming" people by sending them things it's pretty plausible they'd be interested in. It would be fairly easy to put at the top of your email something like "If you're interested in doing AI forecasting, read on; otherwise feel free to ignore this email" which means the cost is something like ~10 seconds per uninterested recipient, which seems reasonable. On a meta note, I think I felt less positively towards this post than I otherwise would have, because it felt like a call to action (which I hold to high standards) rather than an exploratory poll - e.g. I read the first few bullet points as rhetorical questions. Seems like it was just a phrasing issue; and as an exploratory poll, I think it's interesting and I'm glad to have had the issue brought to mind :)
Could we solve this email mess if we all moved to paid emails?

This is super helpful, thanks (and that's a really awesome list of email hygiene tips, I've saved it).

I wonder whether educating and encouraging good email hygiene could be an easier solution (at least initially).

I think it would improve things on the margin, and also has a much smaller risk of landing us in a worse equilibrium, so it seems robustly good for people to do.

Still, I'm not super excited because if you believe that the initial mess is a coordination problem, the solution is not for individuals to put in lots of effort to be h... (read more)

It's not clear to me that we are in a mess. The only actual example you gave was a spammy corporate newsletter, which seems irrelevant.

This might look as follows: Lots of people write to senior researchers asking for feedback on papers or ideas, yet they’re mostly crackpots or uninteresting, so most stuff is not worth reading. A promising young researcher without many connections would want their feedback (and the senior researcher would want to give it!), but it simply takes too much effort to figure out that the paper is promising, so it never gets
... (read more)
Could we solve this email mess if we all moved to paid emails?

On the topic of weirdness: I expect that if what I'm pointing to is a real problem, and paid emails would help the situation, then the benefits from becoming more effective at coordinating internally would massively outweigh reputational risks from increased weirdness.

I find it somewhat hard to elucidate the reasons I believe this (though could try if you'd want me to), but some hand-wavy examples are Paul Graham's thoughts that it's almost always a mistake for startups to worry about competitors as opposed to focusing on building a go... (read more)

3ChristianKleineidam3y
How would this be an "internal practice"? The only way this would work would be to have people publically post their earn addresses. I think you underrate the cost of weirdness. Let's say there's a journalist who wants to write a story where he might ask a high-status EA to comment because it falls into their domain expertise. Then the journalist searches for ways to contact the EA and finds that the EA prefers to get cold approaches via this system. The journalist might think: "This is bad, I don't have a budget for this, paying sources is what evil people do". Even when the journalist then finds that there's a free way to contact the EA, they have their first contact with negative emotional attachment. While the kind of his status EA that might be contacted this way might get more emails then they prefer, it's important for them to be easily contacted by outsiders because that allows for valuable interactions to happen.
Could we solve this email mess if we all moved to paid emails?

I think the way to answer the question is: "given the distribution of equilibria we expect following this change, what are the expected costs and benefits, and how does that compare with the costs and benefits under the current equilibrium?" (as well as considering strategic heuristics like avoiding irreversible actions and unilateralist action.)

I don't update much on your comment since it feels like it's just pointing out a bunch of costs under a particular new equilibrium, without engaging enough with how likely this is or what the be... (read more)

I guess I don't experience major problems with email (sending or receiving), so I don't see very significant benefits. I just read your post as very costly ways to achieve marginal gains.

Could we solve this email mess if we all moved to paid emails?

This was crossposted to LessWrong, replacing all the mentions of "EA" with "rationality", mutatis mutandis.

Could we solve this email mess if we all moved to paid emails?

I'm posting this as a first step towards collecting data. Poll is a good idea, thanks!

Conversation on forecasting with Vaniver and Ozzie Gooen

I'm unfortunately only publishing the transcript at this time. The audio contains some sections that were edited out for privacy reasons.

Building a successful economy for collaborative cognitive work with high externalities

Thanks, that's great to hear.

The prize has been going on for a while, which seems important, and I think the transparency of the Prize post is really important for making common knowledge of what kind of work there is demand for. So overall it's pretty great.

The structure of feedback looks to me like: "here's the object-level content of the post, and here are 2-3 reasons we liked it". I think you could be more clear about what you want to incentivise. More precisely, the current structure doesn't answer:

  • How strong were the rea
... (read more)
2Aaron Gertler3y
Adding a bit of critique where appropriate seems good, since I'm the person who writes the summaries and can ask myself to put in more time. Your other questions all come back to the black-box voting criterion, and I don't push judges to share their individual reasoning. That's largely because doing so makes being a judge more difficult; right now, voting is relatively little work, and given the small amounts of money at stake, I'm reluctant to ask for many more EA hours going toward the project). We may consider splitting up the prize funding differently in the future; saying "piece X deserves four times as much money as piece Y" isn't a good way to think about how the Prize works, but there's still an argument for trying to more closely match grant funding to judges' enthusiasm levels.
Inviting EA forecasters to AI forecasting platform

[Note: I double-checked with the moderators before posting this to ensure it was not too "marketingy".]

  1. When I and Tom came up with that, I don't think we meant "belief" to be imbued with the usual philosophical connotations. Rather, we intended it to mean something like "action-guiding, introspectively accessible representation of a state of affairs existing independently of whether it is queried".

When people ask me what I think about the world, I can often come up with lots of intelligent sounding answers -- but it is unfortunately more rare that my actual actions, plans and normative evaluations are somehow suitably hooked up to, and crucially depend upon, those answers.