All of Jaime Sevilla's Comments + Replies
Good points, I agree that the articles I linked dont directly imply a less than 50% chance of 2ºC warming.
And FWIW Metaculus disagrees with me here, the community prediction is 85% probability of >2ºC warming.
I still hold my position, where my model is that:
- Predictions today are much more optimistic than predictions 10 years ago
- I expect that trend to continue, because we keep understimating social and tech progreess
- I think that the academic process is biased towards being more pessimistic on climate change than the evidence warrants, because of policy c
I appreciate this report and the effort that went into it. That being said, I think it's overly pessimistic considering the evidence we currently have. [1] [2]
I'd bet $1000 at 1:1 odds that we won't see warming over 2°C by 2100.
I'd be happy to take a betting approach that allows for an earlier resolution - I think Tamay Besiroglu described one such procedure somewhere. I can dig it up if anyone wants to take my bet.
Alternatively I would be happy to bet $100 that IPCC projections by 2032 will imply less than 50% probability of higher than 2°C warming by 210... (read more)
Do you see a particular vector or case where harassment might be a risk?
There is precedence for episodes of harassment in the community [1]. One motivated and misguided individual could use this list to conduct more harassment in the future.
There is also precedence for scams directed at academics - I remember distinctly one such scam where one of my colleague's account was spoofed and they tried to scam money out of me.
Overall I agree that this is less risky than a list of people who share a particular belief, and as risky as other public ... (read more)
I'll share with you one such list privately.
One note of caution: there are some important data privacy concerns here. A public list like this could be used to spam or harass researchers. Asking for researcher permission to include their name and information and a mechanism for people to opt out later seems important.
Since I first read your piece on future-proof ethics my views have evolved from "Not knowing about HAT" to "HAT is probably wrong/confused, even if I can't find any dubious assumptions" and finally "HAT is probably correct, even though I do not quite understand all the consequences".
I would probably not have engaged with HAT if not for this post, and now I consider it close in importance to VNM's and Cox' theorems in terms of informing my worldview.
I particularly found the veil of ignorance framing very useful to help me understand and accept HAT.
I'l... (read more)
Yes, thanks! Somehow the copy paste didnt carry over the links
Here is a long answer I wrote a while ago. Not sure how action guiding they were but I am glad the work I mentioned was done.
The relevant part:
Do you have any ideas about how to make progress on [studying the cultural legacy of intentional movements]?
There is a large corpus of historical analysis studying social movements like the suffragettes or the slavery abolitionists. My bet is that there would be large value in summari... (read more)
Fair points. In particular, I think my response should have focused more on the role of academia + industry.
a disproportionate amount of progress on [mechanistic interpretability] has been made outside of academia, by Chris Olah & collaborators at OpenAI & Anthropic
Not entirely fair: if you open the field just a bit to "interpretability" in general you will see that most important advances in the field (eg SHAP and LIME) were done inside academia.
I would also not be too surprised to find people within academia who are doing great mechanistic interp... (read more)
Here my personal interpretation of the post:
> the EA/LW community has a comparative advantage at stating the right problem to solve and grantmaking, the academic community has a comparative advantage at solving sufficiently defined problems
I think this is fairly uncontroversial, and roughly right - I will probably be thinking in these terms more often in the future.
Implications are that the most important output the community can hope to produce is research agendas, benchmarks, idealized solutions and problem statements, and leave ML research, pra... (read more)
Is there any work on historical studies of leaks in the ML field?
Would you like such a project to exist? What sources of information are there?
Any hot takes on the recent NVIDIA hack? Was it preventable? Was it expected? Any AI Safety implications?
Why is Anthropic working on computer security? What are the key computer security problems she thinks is prioritary to solve?
What are the key companies Nova would like to help strenghten their computer security?
How worried is she about dual use of https://hofvarpnir.ai/ for capability development?
What AI Safety research lines are most bottlenecked on compute?
This is comedy gold 🤣
Personally, I have been using ML to make art, help me write short stories and suggest alternative ways of framing the abstracts in my papers.
I expect these applications will become way better soon, and become a staple as they are integrated in text editors like Word.
I think its likely (60%) that a major text editor will by 2026 have a language model making online suggestions, and plausibly an option to generate an appropriate image to be inserted in the middle of the document.
The AI Tracker team has been tracking some potential (mis)uses of cutting-edge AI.
these include phising, social media manipulation, disinformation campaigns, harassment and blackmail, surveillance, cyberattacks, evidence fabrication, bioweapon development and denial-of-service attacks.
"I read this in one sitting" is the highest praise a new author can receive <3
Thank you for giving it a go!
You seem frustrated that some EAs are working on leading AI labs, because you see that as accelerating AI timelines when we are not ready for advanced AI.
Here are some cruxes that might explain why working at leading AI labs might be a good thing:
We are uncertain of the outcomes of advanced AI
AI can be used to solve many problems, including eg poverty and health. It is plausible that we would be harming people who would benefit from this technology by delaying it.
Also, accelerating progress of space colonization can ultimately give you access to a va... (read more)
If you click on the link icon next to the votes you will be redirected to the comment's URL.
For exampe, here is a link to your comment above.
I could not figure out the interface in five minutes and gave up.
My suggestions would be existential windfall and existential bonanza.
Both of them are real words that I expect many people will know.
There is precedent for using the term windfall in AI governance to denote a large increase in revenue, which might make it slighly more confusing. But in any case they seem good words.
FWIW I also like eucatastrophe.
HTML injections?
I wanted to write a post with color highlighting. This would have been easy to do if I could inject some HTML code into my posts. I imagine there are other use cases where people want to do something special that the code base does not support yet.
Being able to embed OWiD interactive graphs and other visualization would be a great plus too!
I think it's bunk that we get to control the number of splits, unless your value function is really weird and considers branches which are too similar to not count as different worlds.
Come on people, the whole point of MWI is that we want to get rid of the privileged role of observers!
though my real expectation is that we probably could just be honest and straightforward, and this wouldn't actually hurt candidates
Endorsed.
Lately I've had two minor unrelated experiences where I have been recommended to not say what I believe straight up out of fear of being misunderstood by people outside the community.
I think on the margin the community is too concerned with reacting to "what people might think" instead of their actual reactions.
I think on the margin the community is too concerned with reacting to "what people might think" instead of their actual reactions.
I see where you're coming from with this general heuristic, but I'm less sure how applicable the heuristic is to this context. In most cases, it seems right to ask, "How will a random person react if they hear X, if they randomly stumble across it?" But given the adversarial nature of politics, the more relevant question here might be, "How will a random person react if they hear X, if it's presented however an adversary want... (read more)
Footnotes are great!
One feature that would make then even greater is if I could copy paste text from a Google Doc that includes footnotes, and have them be formatted correctly.
Thank you Ben!
The ability to add links in bios would be great!
If we could make it so I can edit my bio like I would edit a post it would be even better.
EDIT: ohh the bio uses markdown, noted.
I run a digital art studio, and some of my work is inspired by Effective Altruism themes and ideas.
Particularly, Shared Identity, Shared Values and Science and Identity borrow heavily from the community.
Why both #announcements and #general? What is the use case for each?
Yes, that is right.
I don't have any recent examples in the EA Forum, but here is an article I wrote in LessWrong where the equations where very annoying to edit.
I expect I occassionally would use larger equations, better formated (with underbraces and such) if it was easier to edit in the WYSIWYG editor.
Thanks to you!
In hindsight, the footnotes was the thing I really wanted so I am a very happy user indeed!
Would be good to be able to switch between editors to do things like eg editing complicated LaTeX (right now its complicated to edit it in the WYIWYG editor). But maybe the more reasonable ask is to make the WYSIWYG equation editor span multiple lines for large equations.
TL;DR: I'd like to have a single board where to see a summary of the analytics for all my posts.
I've been really enjoying the analytics feature!
I used it for example to notice that my post on persistence had become very popular, which led me to write a more accessible summary.
One thing I've noticed is that it is very time consuming to track the analytics of each post. That requires me to go to each post, click on analytics and have them load.
I think Medium has a much nicer interface. They have a main user board for stats, from which I can see overall engag... (read more)
I am sure that if you join the AI Alignment slack [1], Rob Miles discord server [2] or ask questions on LW you will find people willing to answer.
Finding a dedicated tutor might be harder, but if you can compensate them for their time. The bountied rationality Facebook group [3] might be a good place to ask.
[1] https://eahub.org/profile/jj-hepburn/ [2] https://www.patreon.com/posts/patreon-discord-41901653 [3] https://m.facebook.com/groups/bountiedrationality/about/
I am so excited for this feature! Finally I will be able to update my posts with real footnotes instead of awkwardly adding them at the end of my posts ^^
Thanks for chipping in Alex!
It's the other way around for me. Historical baseline may be somewhat arbitrary and unreliable, but so is 1:1 odds.
Agreed! To give some nuance to my recommendation, the reason I am hesitant is mainly because of lack of academic precedent (as far as I know).
If the motivation for extremizing is that different forecasters have access to independent sources of information to move them away from a common prior, but that common prior is far from 1:1 odds, then extremizing away from 1:1 odds shouldn't work very well.
Note that the data ... (read more)
UPDATE: Eric Neyman recently wrote about an extra assumption that I believe cleanly cuts into why this example fails.
The assumption is called the weak substitutes condition. Essentially, it means that there are diminishing marginal returns to each forecast.
The Jack, Queen and King example does not satisfy the weak substitutes condition, and forecast aggregation methods do not work well in it.
But I think that when the condition is met we can get often get good results with forecast aggregation. Furthermore I think it is a very reasonable condition to ... (read more)
Future Perfect from Vox is an EA aligned outlet.
Author Kelsey Piper in particular ran the Stanford EA group and frequently covers issues from an EA perspective.
Thank you for writing and sharing. I think I agree with most of the core claims in the paper, even if I disagree with the framing and some minor details.
One thing must be said: I am sorry you seem to have had a bad experience while writing criticism of the field. I agree that this is worrisome, and makes me more skeptical of the apparent matters of consensus in the community. I do think in this community we can and must do better to vet our own views.
Some highlights:
I am big on the proposal to have more scientific inquiry . Most of the work today on existe... (read more)
Some basic functionality I would benefit a lot from:
- Add functionality for footnotes in the WYSIWYG editor
- Make both editors interoperable
- Have a way to toggle between the markdown and the WYSIWYG editor on the fly
Footnotes are a thing that I would use more often if it was easy to do so.
I love editing using the WYSIWYG editor, which does not support them. So when I want to add footnotes I would need to: 1) copy paste my article into a google doc, 2) run a plugin to turn the text to markdown, 3) change my editor settings to Markdown, 4) create a new artic... (read more)
Thank you Rose! You make interesting points, let me try to reason through them:
These papers look at measurable and relatively narrow features of the past, and how far they explain features of the present which are again measurable and relatively narrow.
This is a point worth grappling with. And let's be fair - there are many obvious ways in which cultural transmission clearly has had an effect on modern society.
Case in point: Xmas is approaching! And the fact that we have this arbitrary ritual of meeting once a year to celebrate is a very clear and me... (read more)
I think this is a good point and worth emphasizing.
The studies are focused on studying variation across populations - if everyone in the studied population is equally affected by the cultural forces in question, then this will not show up in the results.
This still means that in practice deliberate cultural interventions are less appealing. In this interpretation, you cannot work towards improving the values of a subpopulation and hope that they will persist through the time - the forces of dispersal and diffussion, as you say, will slowly wilt away t... (read more)
Yes please. This is a great idea and I would want us to move towards a culture where this is more common. Even better if we can use logarithmic odds instead, but I understand that is a harder sell.
Talking about probabilities makes sense for repeated events where we care about the proportion of outcomes. This is not the case for existential risk.
Also I am going to be pedantic and point out that Tao's example about the election is misleading. The percentage is not the chances of winning the election! Instead is the pollling results. The implicit probab... (read more)
Effective Thesis might be able to help if you reach out to them!
In How to generate research proposals I sought to help early career researchers in the daunting task of writing their first research proposal.
Two years after the fact, I think the core of the advice stands very well. The most important points in the post are:
- Develop a pipeline to collect ideas as they come to you.
- Think about scope (is your question concrete?) and methodology (is your question tractable?).
- Devote some time to figuring out what good research looks like.
None of this is particularly original. The value I added is collecting all the advice in a ... (read more)
Somewhat related : a while ago I wrote a benefit-risk analysis of research in improving device detection. I concluded the risks likely outweight the benefits.
(disclaimer: this is my opinion)
In short: Spanish civil protection would not as of today consider making plans to address specific GCRs
There is this weird tension where they believe that resilience is very important, and that planning in advance is nearly useless for non-recurring risks.
The civil protection system is very geared towards response. Foresight, mitigation and prevention seldom happens.This means they are quite keen on improving their general response capacity but they have no patience for hypotheticals. So they would not consider specifi... (read more)
Sounds reasonable enough to me.
The bet will resolve in your favor if the median temperature increase in the stated policies scenario of the 2032 IEA report is above 2°C.
If the IEA report does not exist or does not report an equivalent of their stated policies scenario the bet resolves ambiguously.
Very curious to see what will actually happen!