All of jungofthewon's Comments + Replies

Users can give feedback (thumbs up / down) when they see a wrong answer (image below). We also run evaluation with contractors and test Elicit for our own research.  

 

Yea once we're done here we might go back over there and write some comments :P I agree that it's an interesting perspective. I also liked the comments! 

Thanks for your question!

I found your factored cognition project really interesting, is anyone still researching this? (besides the implementation in Elicit)

Outside of Elicit, not sure. johnwentworth implied there are new researchers interested in this space.

Are you currently collaborating with other EA orgs doing research?

Nothing formal at the moment but we study a lot of independent EA researchers closely. Researchers at GiveWell and Happier Lives Institute have been particularly helpful recently. In the past, we’ve also worked closely with organizations... (read more)

2
Lorenzo Buonanno
2y
Thanks so much and kudos for sharing the LessWrong post, even if it's unjustifiably uncharitable it's an interesting perspective.

Ought is a team of people, each of whom have their own worldviews, values, and communities. For some Oughters, being an EA is an important part of their identity. Others have only heard about it recently! 

I think everyone at Ought cares about being effective and altruistic though :)

This is a live product - not just a demo! You can use it at elicit.org. 

More than 45K users have tried it and ~ 10K use it each month. Users say that Elicit saves them ~ 1-2 hours / week. They proactively share positive feedback on places like Twitter and with their colleagues or friends: Elicit’s growth is entirely by word of mouth.

I agree that having people pay for it is one of the greatest indicators of value. We’ll have to balance financial sustainability with the desire to make high-quality accessible.

At some point, we probably will do a more for... (read more)

We’re experimenting with collecting donations from the individual researchers who use it. We might launch spin-off products in the future that are more commercial or enable Elicit overall to be financially sustainable e.g. an API that lets research orgs run Elicit on their own documents. 

Thanks for thinking through this. Did you give this feedback directly to the people and teams you interacted with? If so and if possible to share without identifying too much - how did that go?

1
AnonymousThrowAway
2y
In deliberately vague summary, where I could and to greater or lesser degrees depending on how much circumstances permitted it.

I have been ruminating about this issue within EA for ages

 

You spent too much time in Phase 1 :P 

+1, generally excited about recent EA momentum towards doing the thing.

Unsurprisingly, I see Elicit as Phase 2 work and am excited for more Phase 2 work. 

Update after reading the rest of it: I actually agree with this so much that I think this can replace large parts of the "user guide" that I aspired to write one day. Thanks for saving me so much time!

2
Peter Wildeford
2y
Oh awesome! That's amazing to hear!

This was great, thanks for taking the time to write this up.

4
jungofthewon
2y
Update after reading the rest of it: I actually agree with this so much that I think this can replace large parts of the "user guide" that I aspired to write one day. Thanks for saving me so much time!

https://elicit.ought.org/builder/RT9kxWoF9 My distribution! Good question Linch; it had a fun mix of investigative LinkedIn sleuthing + decomposition + reasoning about Linch + thoughts that I could sense others might disagree with.

I'll make a distribution. Do you want to make a distribution too and then we can compare?

6
jungofthewon
4y
https://elicit.ought.org/builder/RT9kxWoF9 My distribution! Good question Linch; it had a fun mix of investigative LinkedIn sleuthing + decomposition + reasoning about Linch + thoughts that I could sense others might disagree with.

Non-forecasting question: have you ever felt like an outsider in any of the communities you consider yourself to be a part of?

Yes, I think part of feeling like you don't belong is just pretty normal to being human! So on the outside, this should very much be expected.

But specifically:

  • I think of myself as very culturally Americanized (or perhaps more accurately EA + general "Western internet culture"), so I don't really feel like I belong among Chinese people anymore. However, I also have a heavy (Chinese) accent, so I think I'm not usually seen as "one of them" among Americans, or perhaps Westerners in general.
    • I mitigate this a lot by hanging ou
... (read more)

Which types of forecasting questions do you like / dislike more?

I will forecast a personal question for you e.g. "How many new friends will I make this year?" What do you want to ask me?

2
Linch
4y
In 2021, what percentage of my working hours will I spend on things that I would consider to be forecasting or forecasting-adjacent?

Thanks so much for doing this work! I'm pretty excited about making progress on this. If others want to collaborate, please ping jungwon@ought.org.

Also like this idea! Confused about Moody's revenue - it's $1T? Is it not ~ $4.2BN with 12,000 employees (as opposed to 1,300)?

4
Hauke Hillebrandt
4y
Nice, you found another blunder in the literature! "First, and classically, rating agencies’ fees tend to be high. The revenues of rating agencies come from new ratings and from the reexamination of former ones, as it is very difficult for a company, once it has been rated, to withdraw its rating from the market. It means the operational risk of rating agencies is quite low, just as the volatility of their revenues. We don’t know much about the prices of ratings and the profits of agencies. Nevertheless, in 2011, the operational profit of Standard and Poor’s and Moody’s was about 40 %; and Fitch’s was 31 %. For the first nine months of 2011, the revenue of Standard and Poor’s reached US$ 1.3 trillion for about 1,400 analysts. The figures for Moody’s were US$ 1.2 trillion for 1,300 analysts. These figures make for an annual revenue per analyst higher than US$ 1 million, which is quite high." from this paper on reforming rating agencies: https://sci-hub.tw/https://link.springer.com/chapter/10.1007/978-3-319-44287-7_12 So this should be billions, not trillions. I had actually interpreted the figure differently and thought that rating agencies analysts rate trillions in value or something. Have deleted these from the dataset.

Thanks for this post, Paul.

NOTE: Response to this post has been even greater than we expected. We received more applications for experiment participant than we currently have the capacity to manage so we are temporarily taking the posting down. If you've applied and don't hear from us for a while, please excuse the delay! Thanks everyone who has expressed interest - we're hoping to get back to you and work with you soon.