All of amandango's Comments + Replies

Good note, agreed that it's better to centralize forecasts on the LW thread!

Oh this looks really interesting, I'll check it out, thanks for linking!

The probability of an interval is the area under the graph! Currently, it's set to 0% that any of the disaster scenarios kill < 1 people. I agree this is probably incorrect, but I didn't want to make any other assumptions about points they didn't specify. Here's a version that explicitly states that.

2
MichaelA
4y
It's not obvious to me how to interpret this without specifying the units on the y axis (percentage points?), and when the x axis is logarithmic and in units of numbers of deaths. E.g., for the probability of superintelligent AI killing between 1 and 10 people, should I multiply ~2.5 (height along x axis) by ~10 (length along y axis) and get 25%? But then I'll often be multiplying the height along the x axis by more than 100 and getting insane probabilities? So at the moment I can make sense of which events are seen as more likely than other ones, but not the absolute likelihood they're assigned. I may be making some basic mistake. Also feel free to point me to a pre-written guide to interpreting Elicit graphs.

Thank you for putting this spreadsheet database together! This seemed like a non-trivial amount of work, and it's pretty useful to have it all in one place. Similar to other comments, seeing this spreadsheet really made me want more consistent questions and forecasting formats such that all these people can make comparable predictions, and also to see the breakdowns of how people are thinking about these forecasts (I'm very excited about people structuring their forecasts more into specific assumptions and evidence).

I thought the 2008 GCR questio... (read more)

3
MichaelA
4y
Also, your comment suggests you already saw my comment quoting Beard et al. on breaking down the thought process behind forecasts. You might also be interested in Baum's great paper replying to and building on Beard et al.'s paper. I also commented on Baum's paper here, and Beard et al. replied to it here.
2
MichaelA
4y
Thanks for your comment! Yes, I share that desire for more comparability in predictions and more breakdowns of what's informing one's predictions. Though I'd also highlight that the predictions are often not even very clear in what they're about, let alone very comparable or clear in what's informing them. So clarity in what's being predicted might be the issue I'd target first. (Or one could target multiple issues at once.) And your comments on and graph of the 2008 GCR conference results are interesting. Are the units on the y axis percentage points? E.g., is that indicating something like a 2.5% chance superintelligent AI kills 1 or fewer people? I feel like I wouldn't want to extrapolate that from the predictions the GCR researchers made (which didn't include predictions for 1 or fewer deaths); I'd guess they'd put more probability mass on 0.

I just eyeballed the worst to best case for each revenue source (and based on general intuitions about e.g. how hard it is to start a podcast). Yeah, this makes a lot of sense – we've thought about showing expected value in the past so this is a nice +1 to that.

Gwern's comment was really helpful to see the different paradigms, thanks for sharing! This reasoning makes sense to me in terms of increasing compute - I could see this pushing me slightly more towards shorter timelines, although I'd want to spend a lot longer researching this.

Yeah, I mostly focused on the Q1 question so didn't have time to do a proper growth analysis across 2021 – I just did 10% growth each quarter and summed that for 2021, and it looked reasonable given the EA TAM. This was a bit of a 'number out of the air,' and in reality I wouldn't expect it to be the same growth rate across all quarters. Definitely makes sense that you're not just focusing on the EA market – the market for general productivity services in the US is quite large! I looked briefly at the subscr... (read more)

1
Halffull
4y
Yeah, I was talking about the Q1 model when I was trying to puzzle out what your growth model was. A lot of the value of potential growth vectors of a business come in the tails. For this particular forecast it doesn't really matter because it's roughly bell-curved shape, but if I was using this as for instance decisionmaking tool to decide what actions to take, I'd really want to look at which ideas had a small chance of being very runaway successes, and how valuable that makes them compared to other options which are surefire, but don't have that chance of tail success. Choosing those ideas isn't likely to pay off on any single idea, but is likely to pay off over the course of a business's lifetime.

Here’s my Q1 2021 prediction, with more detailed notes in a spreadsheet here. I started out estimating the size of the market, to get reference points. Based on very rough estimates of CEA subscriptions, # of people Effective Altruism Coaching has worked with, and # of people who have gone through a CFAR workshop, I estimated the number of EAs who are interested enough in productivity to pay for a service to be ~8000. The low number of people who have done Effective Altruism Coaching (I estimated 100, but this is an important assumption that could b... (read more)

4
Halffull
4y
Thanks, this was great! The estimates seem fair, Honestly, much better than I would expect given the limited info you had, and the assumptions you made (the biggest one that's off is that I don't have any plans to only market to EAs). Since I know our market is much larger, I use a different forecasting methodology internally which looks at potential marketing channels and growth rates. I didn't really understand how you were working in growth rate into your calculations in the spreadsheet, maybe just eyeballing what made sense based on the current numbers and the total addressable market? One other question I have about your platform is that I don't see any way to get the expected value of the density function, which is honestly the number I care most about. Am I missing something obvious?

You should be able to access the doc from the link in my comment now! That's useful feedback re: selecting a range and seeing the probability. You can currently see the probability of an interval by defining the interval, leaving the prob blank, and hovering over the bin, but I like the solution you described.

Yeah I could definitely see it being sooner, but didn't find any sources that convinced me it would be more likely in the next 10 years than later – what's driving your shorter timelines?

2
kokotajlod
4y
I have a spreadsheet of different models and what timelines they imply, and how much weight I put on each model. The result is 18% by end of 2026. Then I consider various sources of evidence and update upwards to 38% by end of 2026. I think if it doesn't happen by 2026 or so it'll probably take a while longer, so my median is on 2040 or so. The most highly weighted model in my spreadsheet takes compute to be the main driver of progress and uses a flat distribution over orders of magnitude of compute. Since it's implausible that the flat distribution should extend more than 18 or so OOMs from where we are now, and since we are going to get 3-5 more OOM in the next five years, that yields 20%. The biggest upward update from the bits of evidence comes from the trends embodied in transformers (e.g. GPT-3) and also to some extent in alphago, alphazero, muzero: Strip out all that human knowledge and specialized architecture, just make a fairly simple neural net and make it huge, and it does better and better the bigger you make it. Another big update upward is... well, just read this comment. To me, this comment did not give me a new picture of what was going on but rather confirmed the picture I already had. The fact that it is so highly upvoted and so little objected to suggests that the same goes for lots of people in the community. Now there's common knowledge.

Here’s my prediction for this. It’s pretty uncertain, and I expect others have perspectives which could narrow the range on this forecast. Some thoughts:

  • Although the same algorithms can be generalized, we’re still at the stage where agents have to be trained on individual games [1] [2] [3] [4]
  • It’s really hard to quickly get a sense of how this will progress and what the challenges are without knowing more about the technical research
  • Given that, my prediction is very uncertain over the range, but bounded by AGI timelines

Does this... (read more)

1
kokotajlod
4y
Thanks! It's about what I expected, I guess, but different from my own view (I've got more weight on much shorter timelines). It's encouraging to hear though!

This was really interesting to forecast! Here's my prediction, and my thought process is below. I decomposed this into several questions:

  • Will OpenAI commercialize the API?
    • 94% – this was the intention behind releasing the API, although the potential backlash adds some uncertainty [1]
  • When will OpenAI commercialize the API? (~August/September)
    • They released the API in June and indicated a 2 month beta, so it would begin generating revenue in August/September [2]
  • Will the API reach $100M revenue? (90%)
... (read more)

This was pretty difficult to forecast in a limited amount of time, so you should take my prediction with a large grain of salt. Broadly, I thought about this as:

  • How likely is the 1000th baby to involve iterated embryo selection?
    • There’s a lot of controversy around genetic manipulation for ability, and it’s possible that stem cell gamete reproduction is regulated such that you can only use it as an alternative fertility treatment
      • E.G. controversy around the ethics of genetic relationship of parents to children (see this series of papers for an o
... (read more)
3
Pablo
4y
Thank you, that was informative. I don't think you missed anything, though I haven't myself thought about this question much—that is in part why I was curious to see someone else try to answer it. I think genetic selection and/or editing has the potential to be transformative, and perhaps even to result in greater-than-human intelligence. Despite this, it's comparatively neglected, both within EA and society at large. So having more explicit forecasts in this area seems pretty valuable.

Here's my prediction! My median is October 3, 2020. If you want to keep checking in on this, the Bureau of Consular Affairs is helpfully tracking their passport backlog and how many they're processing each week here.

Was this in line with what you were expecting?

2
kokotajlod
4y
Thanks! Yes it is. All I had been doing was looking at that passport backlog, but I hadn't made a model based on it. It's discouraging to see so much probability mass on December, but not too surprising...

Here’s my prediction. Based on this timeline, I started out thinking it would be quite a while (10+ years) before all 50 states legalized recreational marijuana. This paper caused a pretty significant update towards thinking that federal legalization was more likely sooner than I had previously thought. I also found this map useful for getting a quick sense of the current status.

Curious what you think - here’s a blank distribution if you want to make your own.

2
Eli_Nathan
4y
Yeah — this seems pretty reasonable to me. I'd not thought about this explicitly before, but the rough numbers/boundaries you provide seem quite plausible!

Here’s my prediction for this! Awesome proposal, I enjoyed reading it. I wrote up more of my thought process here, but a brief overview:

  • It would help a lot to know the base rate of EA initiatives succeeding past the first year. I couldn’t find any information on this, but it possibly does exist
  • It wasn’t entirely clear to me what the impact you expect from this project is, which made it hard to estimate cost effectiveness.
    • I suspect a lot of the indirect impact (building EA connections, converting researchers to EA philosophies) will tak
... (read more)
3
EdoArad
4y
Fantastic, thanks!  I've requested access to the doc :)  (Regarding the platform, I think it would help a bit to clarify things if I could do something like selecting a range with the mouse and have the probability mass of that interval displayed) 

You didn't misunderstand! The intention was that you ask any question that's interesting to you, including personal questions. I'm assuming you're more interested in the first question you asked, so I'll answer that unless you feel otherwise :)

1
alex lawsen (previously alexrjl)
4y
Ok awesome, thanks!