Users can give feedback (thumbs up / down) when they see a wrong answer (image below). We also run evaluation with contractors and test Elicit for our own research.
Yea once we're done here we might go back over there and write some comments :P I agree that it's an interesting perspective. I also liked the comments!
Thanks for your question!
I found your factored cognition project really interesting, is anyone still researching this? (besides the implementation in Elicit)
Outside of Elicit, not sure. johnwentworth implied there are new researchers interested in this space.
Are you currently collaborating with other EA orgs doing research?
Nothing formal at the moment but we study a lot of independent EA researchers closely. Researchers at GiveWell and Happier Lives Institute have been particularly helpful recently. In the past, we’ve also worked closely with organizations...
Ought is a team of people, each of whom have their own worldviews, values, and communities. For some Oughters, being an EA is an important part of their identity. Others have only heard about it recently!
I think everyone at Ought cares about being effective and altruistic though :)
This is a live product - not just a demo! You can use it at elicit.org.
More than 45K users have tried it and ~ 10K use it each month. Users say that Elicit saves them ~ 1-2 hours / week. They proactively share positive feedback on places like Twitter and with their colleagues or friends: Elicit’s growth is entirely by word of mouth.
I agree that having people pay for it is one of the greatest indicators of value. We’ll have to balance financial sustainability with the desire to make high-quality accessible.
At some point, we probably will do a more for...
We’re experimenting with collecting donations from the individual researchers who use it. We might launch spin-off products in the future that are more commercial or enable Elicit overall to be financially sustainable e.g. an API that lets research orgs run Elicit on their own documents.
Thanks for thinking through this. Did you give this feedback directly to the people and teams you interacted with? If so and if possible to share without identifying too much - how did that go?
Update after reading the rest of it: I actually agree with this so much that I think this can replace large parts of the "user guide" that I aspired to write one day. Thanks for saving me so much time!
https://elicit.ought.org/builder/RT9kxWoF9 My distribution! Good question Linch; it had a fun mix of investigative LinkedIn sleuthing + decomposition + reasoning about Linch + thoughts that I could sense others might disagree with.
Non-forecasting question: have you ever felt like an outsider in any of the communities you consider yourself to be a part of?
Yes, I think part of feeling like you don't belong is just pretty normal to being human! So on the outside, this should very much be expected.
But specifically:
I will forecast a personal question for you e.g. "How many new friends will I make this year?" What do you want to ask me?
Thanks so much for doing this work! I'm pretty excited about making progress on this. If others want to collaborate, please ping jungwon@ought.org.
Also like this idea! Confused about Moody's revenue - it's $1T? Is it not ~ $4.2BN with 12,000 employees (as opposed to 1,300)?
Thanks for this post, Paul.
NOTE: Response to this post has been even greater than we expected. We received more applications for experiment participant than we currently have the capacity to manage so we are temporarily taking the posting down. If you've applied and don't hear from us for a while, please excuse the delay! Thanks everyone who has expressed interest - we're hoping to get back to you and work with you soon.
Highly recommend Insights as well! Pricey though.