Hide table of contents

Hey everyone, I’m working on a data science project where I’ll be analyzing and visualizing correlations on any datasets I want to with some ML algorithms. I’ve found Our World in Data but I’m wondering if anyone knows other places to find EA related datasets?

Thanks for your time and help :)


(If you have any specific ideas about what datasets to look into, I’d be really interested in hearing your ideas! Right now I’m thinking about looking at correlations between the aspects of life we associate with well-being across the world with measured well-being. I’d also be especially interested in working on a dataset related to AI safety or animal welfare.)


 

10

0
0

Reactions

0
0
New Answer
New Comment


7 Answers sorted by

Hi! I work at OWID – here are a few things that could help you. Their relevance will depend a lot on:

  • the volume of data that you need
  • what you mean by EA-related (for example there are many datasets available on global health and wellbeing, but very few on longtermism and x-risks)

IHME's Global Burden of Disease, used for most disease burden estimates. This is an extremely dense dataset with many dimensions, so if you find the subject relevant, you probably won't run out of ideas with this one.

World Happiness Report. Not a ton of data here, but some tables that could be interesting to explore.

Crop and livestock data from the FAO

Livestock counts - HYDE & FAO (2017)

Data on CO2 and Greenhouse Gas Emissions by Our World in Data

I'm aggregating and visualising EA datasets on https://www.effectivealtruismdata.com/. 

I haven't yet implemented data download links, but they should be done within a week.

The first thing to do with safety that jumps at me is the correlation of gradients between different uses of scaling laws

Anyone make progress on this? I might start an Airtable with 'lists of (lists of) EA relevant datasets' ... including the ones below and more

Thanks for your question, Harry! As it happens, I am also looking for a potentially EA-related dataset for a data science project, so I figured I'd bandwagon on.

Thank you for suggesting OWID as a source of datasets. I'll check that out. I'll also post back if I find anything responsive to your interests.

Follow-up question: Do you care about how large the data set is? Like are we talking 100's of KB, many GB, or some other order of magnitude?

That's great! I have to decide by Thursday, so I'll let you know what we're working on :). 

Definitely nothing larger than a few gigabytes I would say. I'm pretty new to data science and we're using pretty simple methods in this project, so I'm guessing we'll also want to do a relatively simple regression or classification analysis on a relatively simple (and maybe small) dataset.

I recently found a huge public database of databases, and I figured I'd share it here.

A bit stale now, but a good source I found for datasets relevant to EA-adjacent topics (particularly climate change and global health) is https://www.drivendata.org/. They have competitions similar to Kaggle, which I think makes it a pretty ideal source of datasets for people who are new to data science and machine learning. Somebody else posted in a different thread on this forum about DrivenData IIRC.

Curated and popular this week
 ·  · 20m read
 · 
Advanced AI could unlock an era of enlightened and competent government action. But without smart, active investment, we’ll squander that opportunity and barrel blindly into danger. Executive summary See also a summary on Twitter / X. The US federal government is falling behind the private sector on AI adoption. As AI improves, a growing gap would leave the government unable to effectively respond to AI-driven existential challenges and threaten the legitimacy of its democratic institutions. A dual imperative → Government adoption of AI can’t wait. Making steady progress is critical to: * Boost the government’s capacity to effectively respond to AI-driven existential challenges * Help democratic oversight keep up with the technological power of other groups * Defuse the risk of rushed AI adoption in a crisis → But hasty AI adoption could backfire. Without care, integration of AI could: * Be exploited, subverting independent government action * Lead to unsafe deployment of AI systems * Accelerate arms races or compress safety research timelines Summary of the recommendations 1. Work with the US federal government to help it effectively adopt AI Simplistic “pro-security” or “pro-speed” attitudes miss the point. Both are important — and many interventions would help with both. We should: * Invest in win-win measures that both facilitate adoption and reduce the risks involved, e.g.: * Build technical expertise within government (invest in AI and technical talent, ensure NIST is well resourced) * Streamline procurement processes for AI products and related tech (like cloud services) * Modernize the government’s digital infrastructure and data management practices * Prioritize high-leverage interventions that have strong adoption-boosting benefits with minor security costs or vice versa, e.g.: * On the security side: investing in cyber security, pre-deployment testing of AI in high-stakes areas, and advancing research on mitigating the ris
saulius
 ·  · 22m read
 · 
Summary In this article, I estimate the cost-effectiveness of five Anima International programs in Poland: improving cage-free and broiler welfare, blocking new factory farms, banning fur farming, and encouraging retailers to sell more plant-based protein. I estimate that together, these programs help roughly 136 animals—or 32 years of farmed animal life—per dollar spent. Animal years affected per dollar spent was within an order of magnitude for all five evaluated interventions. I also tried to estimate how much suffering each program alleviates. Using SADs (Suffering-Adjusted Days)—a metric developed by Ambitious Impact (AIM) that accounts for species differences and pain intensity—Anima’s programs appear highly cost-effective, even compared to charities recommended by Animal Charity Evaluators. However, I also ran a small informal survey to understand how people intuitively weigh different categories of pain defined by the Welfare Footprint Institute. The results suggested that SADs may heavily underweight brief but intense suffering. Based on those findings, I created my own metric DCDE (Disabling Chicken Day Equivalent) with different weightings. Under this approach, interventions focused on humane slaughter look more promising, while cage-free campaigns appear less impactful. These results are highly uncertain but show how sensitive conclusions are to how we value different kinds of suffering. My estimates are highly speculative, often relying on subjective judgments from Anima International staff regarding factors such as the likelihood of success for various interventions. This introduces potential bias. Another major source of uncertainty is how long the effects of reforms will last if achieved. To address this, I developed a methodology to estimate impact duration for chicken welfare campaigns. However, I’m essentially guessing when it comes to how long the impact of farm-blocking or fur bans might last—there’s just too much uncertainty. Background In
 ·  · 2m read
 · 
In my opinion, we have known that the risk of AI catastrophe is too high and too close for at least two years. At that point, it’s time to work on solutions (in my case, advocating an indefinite pause on frontier model development until it’s safe to proceed through protests and lobbying as leader of PauseAI US).  Not every policy proposal is as robust to timeline length as PauseAI. It can be totally worth it to make a quality timeline estimate, both to inform your own work and as a tool for outreach (like ai-2027.com). But most of these timeline updates simply are not decision-relevant if you have a strong intervention. If your intervention is so fragile and contingent that every little update to timeline forecasts matters, it’s probably too finicky to be working on in the first place.  I think people are psychologically drawn to discussing timelines all the time so that they can have the “right” answer and because it feels like a game, not because it really matters the day and the hour of… what are these timelines even leading up to anymore? They used to be to “AGI”, but (in my opinion) we’re basically already there. Point of no return? Some level of superintelligence? It’s telling that they are almost never measured in terms of actions we can take or opportunities for intervention. Indeed, it’s not really the purpose of timelines to help us to act. I see people make bad updates on them all the time. I see people give up projects that have a chance of working but might not reach their peak returns until 2029 to spend a few precious months looking for a faster project that is, not surprisingly, also worse (or else why weren’t they doing it already?) and probably even lower EV over the same time period! For some reason, people tend to think they have to have their work completed by the “end” of the (median) timeline or else it won’t count, rather than seeing their impact as the integral over the entire project that does fall within the median timeline estimate or