Effective Altruism Forum
EA Forum

Hide table of contents
by adityaraj@eanita
2 min read 0

1

AI safetyPublic communication on AI safetyResearch
Frontpage
This is a linkpost for https://medium.com/@rajsecrets/ai-safety-why-we-need-to-keep-our-smart-machines-in-check-69fd926f6bd3

Hey there! I’ve been thinking a lot about AI lately — you know, those super-smart computer programs that seem to be popping up everywhere. It’s pretty wild how they’re changing the world, right? But here’s the thing: as cool as AI is, we need to make sure it doesn’t go off the rails. That’s where AI safety comes in, and I want to break it down for you.

So, what’s the big deal with AI safety?

Imagine giving a super-powerful tool to a toddler. Scary, right? Well, AI is kind of like that tool. It’s amazing, but if we’re not careful, it could cause some serious problems. AI safety is all about making sure these smart systems do what we want them to do, without any nasty surprises.

Here are the main things we need to watch out for:

1. Making sure AI thinks like us (well, the good parts of us, anyway)
2. Building AI that doesn’t freak out when something unexpected happens
3. Understanding how AI makes decisions (no black boxes allowed!)
4. Keeping our personal info safe from nosy AIs
5. Making sure AI treats everyone fairly, no matter who they are

This isn’t just a problem for tech geeks — it affects all of us. AI is spreading all over the world, changing how we work, learn, and live. It’s helping doctors spot diseases, teaching kids in far-off places, and even fighting climate change. But if we mess up on the safety part, we could be in for a world of trouble.

So, how do we keep AI in line?

It’s not easy, I’ll tell you that. These AI systems can get super complicated. But smart people around the world are working on it. They’re coming up with rules for building AI responsibly, putting safeguards into the systems themselves, and doing tons of testing to make sure everything works right.

The tricky part is that AI keeps getting smarter, so we need to stay on our toes. We’ve got to keep researching new safety tricks, update our laws to deal with AI, and make sure the companies and governments building these things are doing it responsibly.

What can we do about it?

Here’s the cool thing: we all have a part to play in this. We can:

- Learn about AI and how it affects us
- Ask questions when we see AI being used
- Support politicians and companies that take AI safety seriously
- Think about the ethical side of AI in our own work and studies

The bottom line is this: AI could be amazing for solving big problems and making our lives better. But only if we get the safety part right. It’s like putting on a seatbelt before driving a super-fast car — we want to enjoy the ride without worrying about crashing.

So let’s work together to make sure our AI future is awesome, not awful. Trust me, it’s worth the effort!

1

1
0

Reactions

1
0
Comments


No comments on this post yet.
Be the first to respond.
Curated and popular this week
 ·  · 10m read
 · 
I wrote this to try to explain the key thing going on with AI right now to a broader audience. Feedback welcome. Most people think of AI as a pattern-matching chatbot – good at writing emails, terrible at real thinking. They've missed something huge. In 2024, while many declared AI was reaching a plateau, it was actually entering a new paradigm: learning to reason using reinforcement learning. This approach isn’t limited by data, so could deliver beyond-human capabilities in coding and scientific reasoning within two years. Here's a simple introduction to how it works, and why it's the most important development that most people have missed. The new paradigm: reinforcement learning People sometimes say “chatGPT is just next token prediction on the internet”. But that’s never been quite true. Raw next token prediction produces outputs that are regularly crazy. GPT only became useful with the addition of what’s called “reinforcement learning from human feedback” (RLHF): 1. The model produces outputs 2. Humans rate those outputs for helpfulness 3. The model is adjusted in a way expected to get a higher rating A model that’s under RLHF hasn’t been trained only to predict next tokens, it’s been trained to produce whatever output is most helpful to human raters. Think of the initial large language model (LLM) as containing a foundation of knowledge and concepts. Reinforcement learning is what enables that structure to be turned to a specific end. Now AI companies are using reinforcement learning in a powerful new way – training models to reason step-by-step: 1. Show the model a problem like a math puzzle. 2. Ask it to produce a chain of reasoning to solve the problem (“chain of thought”).[1] 3. If the answer is correct, adjust the model to be more like that (“reinforcement”).[2] 4. Repeat thousands of times. Before 2023 this didn’t seem to work. If each step of reasoning is too unreliable, then the chains quickly go wrong. Without getting close to co
JamesÖz
 ·  · 3m read
 · 
Why it’s important to fill out this consultation The UK Government is currently consulting on allowing insects to be fed to chickens and pigs. This is worrying as the government explicitly says changes would “enable investment in the insect protein sector”. Given the likely sentience of insects (see this summary of recent research), and that median predictions estimate that 3.9 trillion insects will be killed annually by 2030, we think it’s crucial to try to limit this huge source of animal suffering.  Overview * Link to complete the consultation: HERE. You can see the context of the consultation here. * How long it takes to fill it out: 5-10 minutes (5 questions total with only 1 of them requiring a written answer) * Deadline to respond: April 1st 2025 * What else you can do: Share the consultation document far and wide!  * You can use the UK Voters for Animals GPT to help draft your responses. * If you want to hear about other high-impact ways to use your political voice to help animals, sign up for the UK Voters for Animals newsletter. There is an option to be contacted only for very time-sensitive opportunities like this one, which we expect will happen less than 6 times a year. See guidance on submitting in a Google Doc Questions and suggested responses: It is helpful to have a lot of variation between responses. As such, please feel free to add your own reasoning for your responses or, in addition to animal welfare reasons for opposing insects as feed, include non-animal welfare reasons e.g., health implications, concerns about farming intensification, or the climate implications of using insects for feed.    Question 7 on the consultation: Do you agree with allowing poultry processed animal protein in porcine feed?  Suggested response: No (up to you if you want to elaborate further).  We think it’s useful to say no to all questions in the consultation, particularly as changing these rules means that meat producers can make more profit from sel
michel
 ·  · 4m read
 · 
I'm writing this in my personal capacity as someone who recently began working at the Tarbell Center for AI Journalism—my colleagues might see things differently and haven’t reviewed this post.  The rapid development of artificial intelligence could prove to be one of the most consequential technological transitions in human history. As we approach what may be a critical period in AI development, we face urgent questions about governance, safety standards, and the concentration of power in AI development. Yet many in the general public are not aware of the speed of AI development, nor the implications powerful AI models could have for them or society at large. Society’s capacity to carefully examine and publicly debate AI issues lags far behind their importance.  This post makes the case for why journalism on AI is an important yet neglected path to remedy this situation. AI journalism has a lot of potential I see a variety of ways that AI journalism can helpfully steer AI development.  Journalists can scrutinize proposed regulations and safety standards, helping improve policies by informing relevant actors of potential issues or oversights. * Example article: Euractiv’s detailed coverage of the EU AI Act negotiations helped policymakers understand the importance of not excluding general-purpose models from the act, which may have played a key role in shaping the final text.[1] * Example future opportunity: As individual US states draft AI regulations in 2025, careful analysis of proposed bills could help prevent harmful loopholes or unintended consequences before they become law. Journalists can surface (or amplify) current and future risks, providing activation energy for policymakers and other actors to address these risks.  * Example article: 404 Media's investigation revealing an a16z-funded AI platform's generation of images that “could be categorized as child pornography” led a cloud computing provider to terminate its relationship with the platf
Recent opportunities in AI safety
57
Quick nudge to apply to the LTFF grant round (closing on Saturday)
calebp
· · 1m read
7
7
15
Applications to EAGxCDMX close in a week!
cescorza
· · 2m read
0
0
11
[Apply] What I Love About AI Safety Fieldbuilding at Cambridge (& We're Hiring for a Leadership Role)
Harrison Gietz
· · 3m read
0
0
View more