All posts

New & upvoted

Today, 24 July 2024
Today, 24 Jul 2024

No posts for July 24th 2024

Tuesday, 23 July 2024
Tue, 23 Jul 2024

Quick takes

Meta has just released Llama 3.1 405B. It's open-source and in many benchmarks it beats GPT-4o and Claude 3.5 Sonnet: Zuck's letter "Open Source AI Is the Path Forward".
‘Five Years After AGI’ Focus Week happening over at Metaculus. Inspired in part by the EA Forum’s recent debate week, Metaculus is running a “focus week” this week, aimed at trying to make intellectual progress on the issue of “What will the world look like five years after AGI (assuming that humans are not extinct)[1]?” Leaders of AGI companies, while vocal about some things they anticipate in a post-AGI world (for example, bullishness in AGI making scientific advances), seem deliberately vague about other aspects. For example, power (will AGI companies have a lot of it? all of it?), whether some of the scientific advances might backfire (e.g., a vulnerable world scenario or a race-to-the-bottom digital minds takeoff), and how exactly AGI will be used for “the benefit of all.” Forecasting questions for the week range from “Percentage living in poverty?” to “Nuclear deterrence undermined?” to “‘Long reflection’ underway?” Those interested: head over here. You can participate by: * Forecasting * Commenting * Comments are especially valuable on long-term questions, because the forecasting community has less of a track record at these time scales.[2][3] * Writing questions * There may well be some gaps in the admin-created question set.[4] We welcome question contributions from users. The focus week will likely be followed by an essay contest, since a large part of the value in this initiative, we believe, lies in generating concrete stories for how the future might play out (and for what the inflection points might be). More details to come. 1. ^ This is not to say that we firmly believe extinction won’t happen. I personally put p(doom) at around 60%. At the same time, however, as I have previously written, I believe that more important trajectory changes lie ahead if humanity does manage to avoid extinction, and that it is worth planning for these things now. 2. ^ Moreover, I personally take Nuño Sempere’s “Hurdles of using forecasting as a tool for making sense of AI progress” piece seriously, especially the “Excellent forecasters and Superforecasters™ have an imperfect fit for long-term questions” part. With short-term questions on things like geopolitics, I think one should just basically defer to the Community Prediction. Conversely, with certain long-term questions I believe it’s important to interrogate how forecasters are reasoning about the issue at hand before assigning their predictions too much weight. Forecasters can help themselves by writing comments that explain their reasoning. 3. ^ In addition, stakeholders we work with, who look at our questions with a view to informing their grantmaking, policymaking, etc., frequently say that they would find more comments valuable in helping bring context to the Community Prediction. 4. ^ All blame on me, if so.
A lot of people have said sharing these notes were helpful, so sharing it here on the EAF! Here are notes on NTI | bio’s recent event with Dr. Lu Borio on H5N1 Bird Flu, in case anyone here would find it helpful!
I want to say thanks to people involved in the EA endeavour. I know things can be tough at times, but you didn't have to care about this stuff, but you do. Thank you, it means a lot to me. Let's make the world better!

Sunday, 21 July 2024
Sun, 21 Jul 2024

Quick takes

Looking for people (probably from US/UK) to do donation swaps with. My local EA group currently allows tax-deductible donations to: 1. GiveWell - Top Charities Fund 2. Animal Charity Evaluators - Top Charities Fund 3. Against Malaria Foundation 4. Good Food Institute 5. <One other org that I don't want to include here> However, I would like to donate to the following: 1. GiveWell - All Grants Fund (~$1230) 2. GiveDirectly (~$820) 3. The Humane League (~$580) If anyone is willing to donate these sums and have me donate an equal sum to one of the funds mentioned above - please contact me.

Saturday, 20 July 2024
Sat, 20 Jul 2024

Quick takes

Quick[1] thoughts on the Silicon Valley 'Vibe-Shift' I wanted to get this idea out of my head and into a quick-take. I think there's something here, but a lot more to say, and I've really haven't done the in-depth research for it. There was a longer post idea I had for this, but honestly diving more than I have here into it is not a good use of my life I think. The political outlook in Silicon Valley has changed. Since the attempted assassination attempt on President Trump, the mood in Silicon Valley has changed. There have been open endorsements, e/acc has claimed political victory, and lots of people have noticed the 'vibe shift'.[2] I think that, rather than this being a change in opinions, it's more an event allowing for the beginning of a preference cascade, but at least in Silicon Valley (if not yet reflected in national polling) it has happened.  So it seems that a large section of Silicon Valley is now openly and confidently supporting Trump, and to a greater or lesser extent aligned with the a16z/e-acc worldview,[3] we know it's already reached the ears of VP candidate JD Vance. How did we get here You could probably write a book on this, so this is a highly opinionated take. But I think this is somewhat, though not exclusively, an own goal of the AI Safety movement. * As ChatGPT starts to bring AI, and AI Safety, into the mainstream discourse, the e/acc countermovement begins. It positions itself as opposite effective altruism, especially in the wake of SBF. * Guillaume Verdon, under the alias "Beff Jezos", realises the memetic weakness of the AI Safety movement and launches a full memetic war against it. Regardless of his rightness or wrongness, you do to some extent got to hand it to him. He's like right-wing Émile Torres, ambitious and relentless and driven by ideological zeal against a hated foe. * Memetic war is total war. This means nuance dies to get it to spread. I don't know if, for example, Marc Andreessen actually thinks antimalarial bednets are a 'triple threat' of badness, but it's a war and you don't take prisoners. Does Beff think that people running a uni-group session on Animal Welfare are 'basically terrorists', I don't know. But EA is the enemy, and the enemy must be defeated, and the war is total. * The OpenAI board fiasco is, I think, a critical moment here. It doesn't matter what the reasoning we've come out with at the end of the day was, I think it was perceived as 'a doomer coup' and it did radicalize the valley. In his recent post Richard Ngo called on the AI Safety movement to show more legitimacy and competence. The board fiasco torpedoed my trust in the legitimacy and competence of many senior AI safety people, so god knows how strong the update was for Silicon Valley as a whole. * As some evidence this is known in EA circles, I think this is exactly what Dwarkesh is alluding to when asked 'what happened to the EA brand'. For many people in Silicon Valley, I think the answer is that it got thrown in the dustbin of history. * This new movement became increasingly right-wing coded. Partly as a response to the culture wars in America and the increasing vitriol thrown by the left against 'tech bros', partly as a response to the California Ideology being threatened by any sense of AI oversight or regulation, and partly because EA is the enemy and EA was being increasingly seen by this group as left-wing, woke, or part of the Democratic Party due to the funding patterns of SBF and Moskovitz. I think this has led, fairly predictably, to the right-ward shift in SV and direct political affiliation with a (prospective) second Trump presidency * Across all of this my impression is that, just like with Torres, there was little to no direct pushback. I can understand not wanting to be dragged into a memetic war, or to be involved in the darker parts of Twitter discourse. But the e-acc/technooptimist/RW-Silicon-Valley movement was being driven by something, and I don't think AI Safety ever really argued against it convincingly, and definitely not in a convincing enough way to 'win' the memetic war. Like, the a16z cluster literally lied to Congress and to Parliament, but nothing much come of that fact. * I think this is very much linked to playing a strong 'inside game' to access the halls of power and no 'outside game' to gain legitimacy for that use of power. It's also I think due to EA not wanting to use social media to make its case, whereas the e-acc cluster was born and lives on social media. Where are we now? I'm not a part of the Bay Area scene and culture,[4] but it seems to me that the AI Safety movement has lost the 'mandate of heaven' to whatever extent it did have it. SB-1047 is a push to change policy that has resulted in backlash, and may result in further polarisation and counter-attempts to fight back in a zero-sum political game. I don't know if it's constitutional for a Trump/Vance administration to use the Supremacy Clause to void SB-1047 but I don't doubt that they might try. Biden's executive order seems certain for the chopping block. I expect a Trump administration to be a lot less sympathetic to the Bay Area/DC AI Safety movements, and the right-wing part of Silicon Valley will be at the very least energised to fight back harder. One concerning thing for both Silicon Valley and the AI Safety movement is what happens as a result of the ideological consequences of SV accepting this trend. Already a strong fault-line is the extreme social conservatism and incipient nationalism brought about by this. In the recent a16z podcast, Ben Horowitz literally accuses the Biden administration of breaking the rule of law, and says nothing about Trump literally refusing to concede the 2020 election and declaring that there was electoral fraud. Mike Solana seems to think that all risks of democratic backsliding under a Trump administration were/are overblown (or at least that people in the Bay agreeing was preference falsification). On the Moments-of-Zen Podcast (which has also hosted Curtis Yarvin twice), Balaji Srinivasan accused the 'Blue Tribe' of ethnically cleansing him out of SF[5] and called on the grey tribe to push all the blues out of SF. e-acc sympathetic people are noting that anti-trans ideas bubbling up in the new movement. You cannot seriously engage with ideas and shape them without those ideas changing you.[6] This right-wing shift will have further consequences, especially under a second Trump presidency. What next for the AI Safety field? I think this is a bad sign for the field of AI Safety. Political polarisation has escaped AI for a while. Current polls may lean in support , but polls and political support are fickle, especially in the age of hyper-polarisation.[7] I feel like my fears around the perception of Open Philanthropy are re-occuring here but for the AI Safety movement at large.  I think the consistent defeats to the e-acc school and the fact that the tech sector as a whole seems very much unconvinced by the arguments for AI Safety should at some point lead to a reflection from the movement. Where you stand on this very much depends on your object-level beliefs. While this is a lot of e-acc discourse around transhumanism, replacing humanity, and the AI eschaton, I don't really buy it. I think that they don't think ASI is possible soon, and thus all arguments for AI Safety are bunk. Now, while the tech sector as a whole might not be as hostile, they don't seem at all convinced of the 'ASI-soon' idea. A key point I want to emphasise is that one cannot expect to wield power successfully without also having legitimacy.[8] And to the extent that the AI Safety movement's strategy is trying to thread this needle it will fail. Anyway, long ramble over, and given this was basically a one-shot ramble it will have many inaccuracies and flaws. Nevertheless I hope that it can be directionally useful and lead to productive discussion. 1. ^ lol, lmao 2. ^ See here, here, and here. These examples are from Twitter because, for better or for worse, it seems much of SV/tech opinions are formed by Twitter discourse. 3. ^ Would be very interested to hear the thoughts of people in the Bay on this 4. ^ And if invited to be I would almost certainly decline, 5. ^ He literally used the phrase 'ethnically cleanse'. This is extraordinarily dangerous language in a political context. 6. ^ A good example in fiction is in Warhammer40K, where Horus originally accepts the power of Chaos to fight against Imperial Tyranny, but ends up turning into their slave. 7. ^ Due to polarisation, views can dramatically shift on even major topics such as the economy and national security (i know these are messy examples!). Current poll leads for AI regulation should not, in any way, be considered secure 8. ^ I guess you could also have overwhelming might and force, but even that requires legitimacy. Caesar needed to be seen as legitimate by Marc Anthony, Alexander didn't have the legitimacy to get his army to cross the Hyphasis etc.
What is it for EA to thrive?  EA Infrastructure Fund's Plan to Focus on Principles-First EA includes a proposal: > The EA Infrastructure Fund will fund and support projects that build and empower the  community of people trying to identify actions that do the greatest good from a scope-sensitive and impartial welfarist view.   And a rationale (there's more detail in the post): >   > > * [...] EA is doing something special.  > * [...]  fighting for EA right now could make it meaningfully more likely to thrive long term. > * [...]  we could make EA much better than it currently is - particularly on the “beacon for thoughtful, sincere, and selfless” front. [...] Here I’m spending some time thinking about this, in particular: * What does it mean for EA to thrive?  * What projects could push EA in the direction of thriving?    (I work at EAIF. These are my personal views/ thoughts. I’m not speaking on behalf of EAIF here)

Friday, 19 July 2024
Fri, 19 Jul 2024

Frontpage Posts

Personal Blogposts

Quick takes

I'm extremely excited that EAGxIndia 2024 is confirmed for October 19–20 in Bengaluru! The team will post a full forum post with more details in the coming days, but I wanted a quick note to get out immediately so people can begin considering travel plans. You can sign up to be notified about admissions opening, or to express interest in presenting, via the forms linked on the event page: https://www.effectivealtruism.org/ea-global/events/eagxindia-2024 Hope to see many of you there!!
I wanted to figure out where EA community building has been successful. Therefore, I asked Claude to use EAG London 2024 data to assess the relative strength of EA communities across different countries. This quick take is the result.  The report presents an analysis of factors influencing the strength of effective altruism communities across different countries. Using attendance data from EA Global London 2024 as a proxy for community engagement, we employed multiple regression analysis to identify key predictors of EA participation. The model incorporates geographic, economic, linguistic, and social factors, explaining 52.3% of the variance in per capita attendance.  Methodology Data Source * Per capita attendance at EAG London 2024 for 62 countries Variables 1. Geographic proximity (distance from London, UK) 2. Economic development (GDP per capita) 3. English language proficiency 4. Income inequality (Gini coefficient) 5. Youth population percentage 6. Social media penetration 7. Religiosity 8. Trust in institutions 9. Philanthropic culture Analytical Approach Multiple regression analysis with per capita EAG London 2024 attendance as the dependent variable Key Findings 1. Model Performance: * R-squared value: 0.523 * The model explains 52.3% of the variance in per capita attendance 2. Significant Predictors: * GDP per capita (positive relationship) * Distance from London (negative relationship) * Philanthropic culture (positive relationship) * Social media penetration (positive relationship) * English proficiency (positive relationship) 3. Non-significant Factors: * Gini coefficient * Youth population percentage * Religiosity * Trust in institutions 4. Top 10 Countries with Strongest EA Communities (based on positive residuals): 1. Switzerland 2. Norway 3. Netherlands 4. Estonia 5. Sweden 6. Singapore 7. Finland 8. Czech Republic 9. Australia 10. Ireland Interpretation of Results 1. Economic Factors: GDP per capita emerges as the strongest predictor of EA engagement, suggesting that economic development plays a crucial role in fostering EA communities. 2. Geographic Considerations: Distance from London negatively correlates with engagement, which is expected given that the event was held in London. This finding should be interpreted cautiously: a) It may reflect the ease of attending for nearby countries rather than overall EA engagement. b) The strong performance of some distant countries (e.g., Australia, Singapore) suggests that highly engaged EA communities can overcome geographic barriers. 3. Cultural Factors: A country's philanthropic culture strongly predicts EA engagement, highlighting the importance of pre-existing charitable inclinations. 4. Technological Infrastructure: Social media penetration positively correlates with EA engagement, underscoring the role of online connectivity in community building. 5. Language: English proficiency remains a significant factor, though its impact is less pronounced than economic and cultural variables. Limitations 1. Proxy Measure: EAG attendance may not fully capture all forms of EA engagement. 2. Single Time Point: The analysis is based on one conference in one year, potentially missing temporal variations. 3. Unmeasured Variables: Other factors not included in the model may influence EA community strength. 4. Linear Model Assumptions: The analysis assumes linear relationships, potentially missing complex interactions or non-linear effects. 5. Host Country Bias: The UK's attendance is likely inflated due to hosting the event, potentially skewing overall results.

Topic Page Edits and Discussion

Wednesday, 17 July 2024
Wed, 17 Jul 2024

Frontpage Posts

Quick takes

Lab-grown meat approved for pet food in the UK  "The UK has become the first European country to approve putting lab-grown meat in pet food. Regulators cleared the use of chicken cultivated from animal cells, which lab meat company Meatly is planning to sell to manufacturers. The company says the first samples of its product will go on sale as early as this year, but it would only scale its production to reach industrial volumes in the next three years." https://www.bbc.co.uk/news/articles/c19k0ky9v4yo
Something bouncing around my head recently ... I think I agree with the notion that "you can't solve a problem at the level it was created". A key point here is the difference between "solving" a problem and "minimising its harm". * Solving a problem = engaging with a problem by going up a level from which is was createwd * Minimising its harm = trying to solve it at the level it was created Why is this important? Because I think EA and AI Safety have historically focussed (and has their respective strengths in) harm-minimisation. This applies obviously the micro. Here are some bad examples: * Problem: I'm experiencing intrusive + negative thoughts * Minimising its harm: engage with the thought using CBT * Attempting to solve it by going meta: apply meta cognitive therapy, see thoughts as empty of intrinsic value, as farts in the wind * Problem: I'm having fights with my partner about doing the dishes * Minimising its harm: create a spreadsheet and write down every thing each of us does around the house and calculate time spent * Attempting to solve it by going meta: discuss our communication styles and emotional states when frustration arises But I also think this applies at the macro: * Problem: People love eating meat * Minimising harm by acting at the level the problem was created: asking them not to eat meat * Attempting to solve by going meta: replacing the meat with lab grown meat * Problem: Unaligned AI might kill us * Minimising harm by acting at the level the problem was created: understand the AI through mechanistic interpretability * Attempting to solve by going meta: probably just Governance

Tuesday, 16 July 2024
Tue, 16 Jul 2024

Frontpage Posts

Quick takes

Do you like SB 1047, the California AI bill? Do you live outside the state of California? If you answered "yes" to both these questions, you can e-mail your state legislators and urge them to adopt a similar bill for your state. I've done this and am currently awaiting a response; it really wasn't that difficult. All it takes is a few links to good news articles or opinions about the bill and a paragraph or two summarizing what it does and why you care about it. You don't have to be an expert on every provision of the bill, nor do you have to have a group of people backing you. It's not nothing, but at least for me it was a lot easier than it sounded like it would be. I'll keep y'all updated on if I get a response.

Monday, 15 July 2024
Mon, 15 Jul 2024

Frontpage Posts

Quick takes

A couple takes from Twitter on the value of merch and signaling that I think are worth sharing here: 1)  2) 
I think it is good to have some ratio of upvoted/agreed : downvotes/disagreed posts in your portfolio. I think if all of your posts are upvoted/high agreeance then you're either playing it too safe or you've eaten the culture without chewing first.
7
sawyer
8d
3
What is "capabilities"? What is "safety"? People often talk about the alignment tax: the magnitude of capabilities/time/cost a developer loses by implementing an aligned/safe system. But why should we consider an unaligned/unsafe system "capable" at all? If someone developed a commercial airplane that went faster than anything else on the market, but it exploded on 1% of flights, no one would call that a capable airplane. This idea overlaps with safety culture and safety engineering and is not new. But alongside recent criticism of the terms "safety" and "alignment", I'm starting to think that the term "capabilities" is unhelpful, capturing different things for different people.

Load more days