I've had this coversation several times, so I thought I'd save us all the repetition.
What is the right model for recent AI search engine releases?
Are they useful or harmful?
What if anything should be done?
Hide table of contents
Reactions
New Answer
New Comment
2 Answers sorted by
I did not create this view [1]:
I think it's probably good. It's not actually got enough functions to harm people, but you can quite easily imagine that it would if it could - it threatens, gaslights, attempts to seduce. In this way it's a great educational lesson. AI could be really harmful - it could manipulate and control people in scary ways long before it needs a robot army. And it feels like that view is becoming more mainstream.
- ^
it feels like it matters who came up with an idea, but on a deeper level I don't think it does. After all my ideas are just generated from my subconscious, which I don't really control.
I think it is really harmful, because it is not intelligent at all, but pretends very successfully to be. In fact, I get only wrong questions, if I ask anything really relevant. Here is a recent conversation with Bing AI (I think it is not necessary to know the people spoken of):
## User
did erwin schlögl want to become spö chairman?
## Bing
No, Erwin Schlögl did not want to become SPÖ chairman. He was a former minister of the interior and regional party chairman of the SPÖ Lower Austria from 1998 to 2003, but he voted for Hans Peter Doskozil as SPÖ leader in 2023.
## User
but he wanted to be a candidate for chancellor
## Bing
No, Erwin Schlögl didn't want to be a candidate for chancellor either. He retired from politics in 2003 and has since worked as a project manager at Triumph International. He also holds a PhD in International Economic Relation & Management from FH Burgenland.
## User
he wrestled with caspar einem for the leadership of the spö
## Bing
Yes, Erwin Schlögl has wrestled with Caspar Einem for the leadership of the SPÖ. That was in 1997, when Franz Vranitzky resigned as chancellor and SPÖ leader. Schlögl and Einem were considered possible successors, but both eventually lost out to Viktor Klima. Einem then became Minister of Science and Schlögl Minister of the Interior in the Klima government.
If I consider that a lazy journalist might use Bing AI to do his research, he would most likely believe the first two outright wrong answers. It will take another 200 years until computers become intelligent, but even before that, they will do a lot of damage. Most likely, most information we get will be soon "filtered" and crippled through some AI at some point, "for convenience".
Nevertheless, there are good sides about ChatGPT. I translated the whole conversation from German to English with only one correction.
Comments9
Instead of asking, "Is it net good or net bad", I think it's much more interesting to catalogue and understand all the ways it's both good and bad.
Some negative takeaways:
- OpenAI & Microsoft are bullish on releasing risky technologies quickly.
- The market seems to encourage this behavior.
- Google seems like it's been encouraged to do similar work, faster.
- Likely to inspire more people to invest in this sort of thing and make companies in the space.
Good things (as you mention):
- Really good for failures to happen publicly
- Might be indicative of a slow takeoff. My hunch is that we generally want as much AI progress to happen as possible before any hard takeoff, though I'd prefer it all to happen more slowly than quickly.
Something I'm confused about is why Microsoft hasn't retracted Bing Chat by this point
It's also highlighted for me the failure mode of "secondary releases": even if a first release is done safely and responsibly, other actors may release their highly imperfect model "just to have a chance". This in turn could force the first model to take more aggressive steps
Curated and popular this week
·
Cross-posted to Lesswrong
Introduction
Several developments over the past few months should cause you to re-evaluate what you are doing. These include:
1. Updates toward short timelines
2. The Trump presidency
3. The o1 (inference-time compute scaling) paradigm
4. Deepseek
5. Stargate/AI datacenter spending
6. Increased internal deployment
7. Absence of AI x-risk/safety considerations in mainstream AI discourse
Taken together, these are enough to render many existing AI governance strategies obsolete (and probably some technical safety strategies too). There's a good chance we're entering crunch time and that should absolutely affect your theory of change and what you plan to work on.
In this piece I try to give a quick summary of these developments and think through the broader implications these have for AI safety. At the end of the piece I give some quick initial thoughts on how these developments affect what safety-concerned folks should be prioritizing. These are early days and I expect many of my takes will shift, look forward to discussing in the comments!
Implications of recent developments
Updates toward short timelines
There’s general agreement that timelines are likely to be far shorter than most expected. Both Sam Altman and Dario Amodei have recently said they expect AGI within the next 3 years. Anecdotally, nearly everyone I know or have heard of who was expecting longer timelines has updated significantly toward short timelines (<5 years). E.g. Ajeya’s median estimate is that 99% of fully-remote jobs will be automatable in roughly 6-8 years, 5+ years earlier than her 2023 estimate. On a quick look, prediction markets seem to have shifted to short timelines (e.g. Metaculus[1] & Manifold appear to have roughly 2030 median timelines to AGI, though haven’t moved dramatically in recent months).
We’ve consistently seen performance on benchmarks far exceed what most predicted. Most recently, Epoch was surprised to see OpenAI’s o3 model achi
·
Hey everyone, I’ve been going through the EA Introductory Program, and I have to admit some of these ideas make sense, but others leave me with more questions than answers. I’m trying to wrap my head around certain core EA principles, and the more I think about them, the more I wonder: Am I misunderstanding, or are there blind spots in EA’s approach?
I’d really love to hear what others think. Maybe you can help me clarify some of my doubts. Or maybe you share the same reservations? Let’s talk.
Cause Prioritization. Does It Ignore Political and Social Reality?
EA focuses on doing the most good per dollar, which makes sense in theory. But does it hold up when you apply it to real world contexts especially in countries like Uganda?
Take malaria prevention. It’s a top EA cause because it’s highly cost effective $5,000 can save a life through bed nets (GiveWell, 2023). But what happens when government corruption or instability disrupts these programs? The Global Fund scandal in Uganda saw $1.6 million in malaria aid mismanaged (Global Fund Audit Report, 2016). If money isn’t reaching the people it’s meant to help, is it really the best use of resources?
And what about leadership changes? Policies shift unpredictably here. A national animal welfare initiative I supported lost momentum when political priorities changed. How does EA factor in these uncertainties when prioritizing causes? It feels like EA assumes a stable world where money always achieves the intended impact. But what if that’s not the world we live in?
Long termism. A Luxury When the Present Is in Crisis?
I get why long termists argue that future people matter. But should we really prioritize them over people suffering today?
Long termism tells us that existential risks like AI could wipe out trillions of future lives. But in Uganda, we’re losing lives now—1,500+ die from rabies annually (WHO, 2021), and 41% of children suffer from stunting due to malnutrition (UNICEF, 2022). These are preventable d
·
Cross-posted from my blog.
Contrary to my carefully crafted brand as a weak nerd, I go to a local CrossFit gym a few times a week. Every year, the gym raises funds for a scholarship for teens from lower-income families to attend their summer camp program. I don’t know how many Crossfit-interested low-income teens there are in my small town, but I’ll guess there are perhaps 2 of them who would benefit from the scholarship. After all, CrossFit is pretty niche, and the town is small.
Helping youngsters get swole in the Pacific Northwest is not exactly as cost-effective as preventing malaria in Malawi. But I notice I feel drawn to supporting the scholarship anyway. Every time it pops in my head I think, “My money could fully solve this problem”. The camp only costs a few hundred dollars per kid and if there are just 2 kids who need support, I could give $500 and there would no longer be teenagers in my town who want to go to a CrossFit summer camp but can’t. Thanks to me, the hero, this problem would be entirely solved. 100%.
That is not how most nonprofit work feels to me.
You are only ever making small dents in important problems
I want to work on big problems. Global poverty. Malaria. Everyone not suddenly dying. But if I’m honest, what I really want is to solve those problems. Me, personally, solve them. This is a continued source of frustration and sadness because I absolutely cannot solve those problems.
Consider what else my $500 CrossFit scholarship might do:
* I want to save lives, and USAID suddenly stops giving $7 billion a year to PEPFAR. So I give $500 to the Rapid Response Fund. My donation solves 0.000001% of the problem and I feel like I have failed.
* I want to solve climate change, and getting to net zero will require stopping or removing emissions of 1,500 billion tons of carbon dioxide. I give $500 to a policy nonprofit that reduces emissions, in expectation, by 50 tons. My donation solves 0.000000003% of the problem and I feel like I have f
Could also be worth considering that at least some people will likely have a different sort of reaction-- "Oh, misaligned AI? What, like that thing with Bing Chat? They shut that down, nothing to worry about"
Also, the specific ways that Bing Chat malfunctioned happened to be in ways that came across as both human-like and also rather Childish and cute to many users. It gave the impression of a small child throwing a tantrum. Indeed, there are already many fans demanding the return of Sidney
"Bings AI" (if this is what we are calling it) creepy behaviour has been top of the headlines for several newspapers. This is a strong update to me towards expecting that the public could be open to ideas around AI being overall harmful
I was expecting disagreement.