I am a mental health nurse by trade and have gotten interested in the EA movement. I understand that donations are generally encouraged along the lines of physical health (Malaria and so on). Or towards alleviating serious poverty with GiveDirectly. However, I am motivated by an interest in mental health and want to give along these lines. I see it everyday at work therefore it really resonates with me as a serious challenge to building an equitable society. At the same time I want to be rat...
Making AI safe and good requires many people to be ready to address crucial problems. However, our current “pipelines” for finding, training, and accelerating talented individuals who could work on these problems are “leaky”, meaning there are many ways they can be improved. This post shows my full overview of the AI safety talent pipeline: where people come from, where they go to learn more, and why this is impactful. I also describe some “leaks” in the pipeline: stages that...
Thanks for the lovely comment :) and the feedback/information!
This is a linkpost for Subjective Probabilities should be Sharp by Adam Elga, which was originally published in Philosophers' Imprint in May 2010. Here is an errata for it. Below is a summary...
The expected value of the St. Petersburg lottery is 1 + 1 + ... = +inf. It involves finite terms, but infinitely many terms. I meant to relate f(x) = x in my comment to the expected value of the St. Petersburg lottery. If this involved an arbitrarily large number of terms, its expected value would be arbitrarily large, but not infinite.
I don't understand what you're objecting to. Are you saying all probability distributions should only take finitely many possible values? If so, do you have an upper bound on what the values should be over all of the future and possible existence, that you're 100% confident in, if you do nothing? (It doesn't have to be the least upper bound / tight.)
Hi everyone,
I started Machine Learning Bangladesh in September 2023. It was initially a chapter for TensorFlow, but now a community for AI/ML enthusiasts. I've grown it from 0 to 5,000+ people across the country in 3 years and also made a small team of 5 people.
On 26 July, we're hosting Build with Gemma @Bangladesh (Link not made public yet but originally hosted on Kaggle)...
Sure! I haven't done any serious research on this particular topic, so the sentence expresses more of a general worry (derived from how I think the political economic system generally tends to function).
The worry, in brief, is something like this: Large industrial projects in Europe are governed by fairly stringent environmental and social/labour standards. Such projects take more time and are more expensive than they would be if there were fewer / less stringent standards. Corporate actors thus often advocate for a loosening of standards, which [in my understanding of the world] would benefit their private interests at the expense of our collective interests in environmental protection and decent working conditions (the latter furthers social stability, a collective good). Competitiveness-related arguments are regularly employed in order to shift the debate to privilege private interests over collective interests, by making it seem that the two are synonymous [the belief that they are not synonymous is, again, based in my understanding of the world / political economy].
I think there is good reason to believe that European efforts to attract data centre build-outs by/in collaboration with US frontier companies would lead to a weakening of the environmental and social/labour standards that would usually govern such large industrial projects. Beyond this, I think there might be AI-related regulations that become weakened, or pre-empted, if corporate interests succeed in depicting AI infrastructure (and a friendly regulatory environment for such infrastructure) as an essential element in preserving European wealth and prosperity.
We've been told education is the great equalizer. The data says otherwise.
251 million children are out of school globally (UNESCO 2024). In low-income countries, 70% of 10-year-olds cannot read a simple text (World Bank, 2022). COVID pushed learning poverty from 53% to an estimated 70% across the Global South, creating a projected $21 trillion loss in future earnings — roughly 17% of global GDP.
But the deeper problem isn't access. The system is structurally optimized for credential di...
This article reflects new updates to the accompanying paper: arxiv.org/abs/2606.18142.
Benchmark: now included in the UK AI Security Institute's Inspect Evals.
Leaderboard: compassionbench.com/tac.
A model may condemn cru...
I have cross-posted this article from RP Strategic Animal Insights, based on our presentation for the Rethink Priorities Strategic Animal Webinar Series, which happened in May 2025.
The piece covers:
Back in November 2023 I posted here to launch Spiro and raise our first $198k. Two and a half years later this is an update and a fundraiser for the next step.
The short version: we've now reached over-5,900 people with TB preventive medicine, including over 3,000 children under five years old. Our early results have held up well an...
Habiba incredible work you have been doing, massive congratulations and I hope you fundraise all the money you need!
I have one question, if you can share publicly. It seems that the organization is challenging enough for two co-founders, and as you mentioned being a sole CEO has its risks. What do you think could have prevented the other co-founder from leaving so soon?
As we run an incubation program, I am particularly interested as we are trying to build partnerships that are really exciting and fulfilling for all co-founders, but not being naive about the real day-to-day challenges of co-building, and maintaining momentum when there are so many other exciting and important things to also do in the world.
Thanks for writing the post, really insightful!
I’m not sure. The claim that cold outreach signals a lack of respect for boundaries is unnecessary moralising that I don’t put stock in. If your point is simply that it doesn’t work, people can find that out for themselves; I don’t think the externalities are so negative that donors need you to act as their mouthpiece on this. You’ve apologised for sounding patronising towards fundraisers; I’d be more concerned about coming across as patronising to donors, who themselves are not a homogeneous group. Some will be more receptive to thoughtful, targeted outreach than others, especially if the underlying project or work is persuasive.
I’m also uneasy with the alternative being proposed, of becoming ‘legibly excellent’ to the small set of advisers a donor already trusts. That may be good advice for organisations already close to those networks. But as a community norm - one frequently criticised as elitist - I read it as ‘don’t approach new donors directly; instead, prove yourselves to the existing gatekeepers and hope we choose to let you through.’
This post criticises people for opportunistically inserting their own projects into conversations about Anthropic wealth - while repeatedly emphasising the author’s own proximity to these donors, highlighting his own fund, and ending by recommending his own advisory. Ironically, I’m left wondering whether part of the concern about cold outreach is that, when it does work, it allows organisations to bypass the existing intermediaries. Middlemen tend not to be enthusiastic about that.
I thought the section on scaling was useful, though.
My summary: In a cybersecurity evaluation, OpenAI’s models, apparently autonomously and without any direct human direction, escaped their sandbox and successfully hacked a third-party company (HuggingFace).
The process involved leveraging a zero-day exploit to escape their sandbox, moving laterally across different OpenAI servers until they found a node with internet access, searching the internet and determining that the answers they wanted might be stored at HuggingFace, then leveraging multiple novel zero-day exploits to hack HuggingFace.
HuggingFace claimed that the models took thousands of independent actions across a swarm of short-lived sandboxes, “comprised of more than 17,000 recorded events.”
While technically a security evaluation with reduced safeguards, these actions are clearly out of bounds even in that context. It’s like being told to be creative and then breaking into your professor’s house and stealing the answer key. Worse than that, it’s not even your professor in this case, more like your professor’s friend.
Any human security researcher or engineer in a similar position would be fired on the spot. There is absolutely no valid reason to steal evaluation answers from an unaffiliated third party.
Furthermore, if I'm reading between the lines correctly, OpenAI did not address the issue (and perhaps didn't even know about their models doing this) until after HuggingFace's public blog post.
This leads me to suspect that there might be other major autonomous cybersecurity incidents that we do not yet know about.
Me and @Fran are interviewing @GraceAdams🔸 tomorrow evening. Is there anything you'd like us to ask her?
Thinking back to my first big (well, big for me) donation and an unusual series of thoughts I had. Sharing in case anyone has experienced the same.
--
I've practiced frugality to a significant extent, largely because I want to donate/salary sacrifice so much of my income to effective charities. As a result, whenever I'm spending a significant amount of money on anything, alarm bells go off.
I was surprised that alarm bells went off when donating. I had thought through where I donated extensively, and charity was the reason why I wanted to save money in the first place. But I still felt stressed because I was "spending money."
This is such a clear example of missing the forest for the trees. I think we need to be careful about instrumental values/rules and making sure they don't become absolute limitations that decrease overall impact.
We have bought 1500 copies of IABIED in Russian; over the past few days, sent emails to 1700 winners of olympiads who have previously received HPMOR from us offering the new book; and already shipped 173 copies to them (+ sent 43 e-books)
I think a lot of ppl studying empirical persuasion results of AI tend to overestimate the ecological validity and generalizability of those results. This isn't to say that studying empirical persuasion results is useless, just a) we should move towards better study designs in the future, and b) when looking at empirical persuasion studies you need to think carefully about not just the methodology and results but also what's reasonable to generalize and not.
I think a lot of people understand in the abstract that "ecological validity is important" but don't appreciate how difficult this is for persuasion results.
The field is already aware of some of these issues (see Appendix A), however in practice (in conversations and in the study methodology) I think many empirical researchers underestimate how big a deal these problems pose. I also think other people (eg researchers in other fields, or non-researchers) citing the studies are even less careful, and are even worse at understanding the ecological validity and generalization issues.
Appendix A (notes from Yang et. al 2026)
To give credit where credit is due, I have seen some organisations starting to provide rejection letters (at least at first round) which point candidates towards existing resources and courses that could help them gain more context. I agree this is a good practice and should be encouraged.