Effective Altruism Forum
EA Forum

Login
Sign Up
by sergia1 min read2 Mar 2023 71

14

AI safetyAnthropicAI race
Frontpage
This is a linkpost for https://forum.effectivealtruism.org/posts/FnszH6ZGBi9hd8rtv/google-invests-usd300mn-in-artificial-intelligence-start-up

I'm surprised that once again it starts as "let's work on safety together! Let's share ideas and work on a good thing", then some entity grows bigger and bigger, starts taking unilateral decisions that are controversial, still using the name and support of community

I feel the same scheme as with SBF: first, the community is used to build The Thing. Then, The Thing forgets about anything ethical or safe and just turns into an "effective profit/PR maxmizer". I feel kinda conned and used. Even when I'm mostly talking to AI ethicists now, I still regarded Anthropic as something not evil. Even I was shocked.

I ask people to demand answers from them. I feel there's a no-confidence case for us trusting Anthropic to do what they are doing well.

I was more about "let's have safety and ethics people together" (which ethicists didn't like), less and less in time. Now I don't know anymore. I want answers.

I feel traumatized in general by the safety community and EA. I was doing research internships at Google and CHAI Berkeley. I was doing later an ethics nonprofit. All of those were somewhat EA-aligned (not 100% outside). I don't know how can I trust people who say "safety" anymore.

What is going on?

See my thread for more questions. I feel traumatized by EA, by this duplicity (that I have seen "rising up" before this, see my other threads). I'm searching for a job and I'm scared of people. Because this is not the first time, not at all. Somehow tech people are "number one" at this. And EA/tech people seem to be "number 0", even better at Machiavellianism and duplicity than Peter Thiel or Musk. At least, Musk openly says he's "red-pilled" and talks to Putin. What EA/safety is doing is kinda similar but hidden under the veil of "safety".

Not all people are like this. Let's not be like this.

I expect downvotes - I don't care. I want answers.

https://twitter.com/sergia_ch/status/1631338866840948737?s=20

14

More posts like this

Comments71
Sorted by
new & upvoted
Click to highlight new comments since: Today at 10:57 PM
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
Ofer3mo4742

I think for starters Anthropic should publish the list of its board members, its bylaws and cap table. I'm not aware of any of that currently being public info. (Since Anthropic is a public-benefit corporation, presumably the CEO can choose to publish that info even if doing so is not aligned with the financial interests of the shareholders.)

Reply
-13
sergia
3mo
Linch3mo3435

I don't know if demanding answers makes sense, but I do think it's a pretty hard call whether Anthropic is net positive or net negative for AI safety; I'm surprised at the degree to which some people seem to think this question is obvious; I'm annoyed at the EA memeplex for not making this uncertainty more transparent to newcomers/outsiders; I hope not too many people join Anthropic for bad reasons.

Reply
0
sergia
3mo
I'm looking at this discourse since 2018, including when I was in EA and doing AI safety. At no point I saw a discussion whether a big EA-adjacent org is net-positive or net-negative. It's some sort of a "blind spot": we evaluate other people's charities. But ours are, of course, pretty good. I feel it's time to have a discussion about this, that would be awesome.
Linch3mo108

I mean in at least in global health and animal welfare, most of the time we don't evaluate charities for being net-negative, we only look at "other people's charities" that are already above a certain bar. I would be opposed to spending considerable resources looking at net negative charities in normal domains, most of your time is much better spent trying to triage resources to send to great projects and away from mediocre ones.

In longtermism or x-risk or meta, everything is really confusing so looking at net-positive vs net-negative becomes more compelling.

For what it's worth, it's very common at LTFF and other grantmakers to consider whether grants are net negative.

Also to be clear, you don't consider OpenAI to be EA-adjacent right? Because I feel like there are many discussions about OpenAI's sign over the years.

Reply
-17
sergia
3mo
freedomandutility3mo1912

My crux here is whether or not I think Anthropic has joined the arms race.

Why do you believe that it has?

Reply
5
Habryka
3mo
See for example this summary of someone who spent quite a lot of time trying to understand and pass the ITT of Anthropic's strategy: https://www.lesswrong.com/posts/MNpBCtmZmqD7yk4q8/my-understanding-of-anthropic-strategy [https://www.lesswrong.com/posts/MNpBCtmZmqD7yk4q8/my-understanding-of-anthropic-strategy] 
2
sergia
3mo
Do you believe in it? Just seems weird if someone said "to be safe from a deadly disease, what we really need is to develop it as soon as we can" I get that the metaphore has holes, just, seems a bit "out there". I'd say that "to have safe agi, we need to do agi engineering the fastest way possible" is a very extraordinary claim. It requires very extraordinary evidence to support it. My thing which is "can we ask them to explain it" seems like a very ordinary claim to me. So it doesn't require much evidence at all.
-3
sergia
3mo
Here we have Microsoft's CEO saying they're "gonna make Google dance" with my comments about how Microsoft's CEO sounds like a comic book villain https://twitter.com/sergia_ch/status/1624438579412799488?s=20 [https://twitter.com/sergia_ch/status/1624438579412799488?s=20] If being serious, I don't feel it when thinking of the phrase "Google just invested into Anthropic to advance AI safety". Just don't feel it. Don't know why, maybe because of how Google handled it ethics team? Or when they said "were not gonna be doing weapons" and then like, started doing it? Just seems like something rather likely if we consider their character inferred from their previous actions that they want their own chat bot, to show everyone how smart they are (regardless of the consequences) Once a prof told me how he sees the ML field: people there don't do it for "humanity" or "knowledge", he told me it's because they want to show how their stuff is superior to someone else's and show off. Not everyone's like this, of course, but ML/tech has this vibe - people from the first row of seats from school who don't know anything about the real world and instead try to impress the teacher, living of petty drama between same people on the front row. A lot of people like this in ML Saying this as ex one of those people. To sum up, here's my personal story as one who was in the field, and as in another reply, I invite you to form your own understanding based on whatever you like. I can't convince you, I only have a personal story as an AIS beginning researcher, I don't have statistics and expected value calculations people here seem to want. Thank you
2
freedomandutility
3mo
So rather than a specific claim about specific activities being done by Anthropic, would you say that: 1. from your experiences, it’s very common for people to join the arms race under the guise of safety 2. you think by default, we should assume that new AI Safety companies are actually joining the arms race, until proven otherwise 3. the burden of proof should essentially rest on Anthropic to show that they are really doing AI Safety stuff? Given the huge potential profits from advancing AI capabilities faster than other companies and my priors on how irrational money makes people, I’d support that view.
sapphire3mo7-2

Effective altruism's meta-strategy is about friendliness to (tech) power. All our funding comes from tech billionaires. We recruit at elite colleges. We strongly prioritize good relations with AI labs and the associated big tech companies. EA just isn't going to be genuinely critical or antagonistic toward the powerful groups we depend on for support and status. Not how EA works.

Reply
5
Sabs
3mo
This doesn't seem like a bad meta-strategy, fwiw. Surely otherwise EA just gets largely ignored.
-3
sergia
3mo
See my other comment in the same thread
-8
sergia
3mo
-4
sergia
3mo
I totally feel this isn't the only choice to do things. There are massive crowdfunding campaigns that work. I think that an entity that is not opposing itself to the power in any way has its own limitations, serious limitations. Here's an example from Russia where some charities collect money, but HAVE to say they're pro-government. In many cases those were criticised, and I think justly, that they created more troubles than the effects of their charity. For example, some used TV ads to gather money for cancer treatment for children. However, the real problem is: Putin used all the taxes and gas profits on his wars and "internet research" operations, as well as personal luxury items. So these charities, some argue, were used as a "front" by the government to convince people that "medicine is OK, no need to worry" Those charities only helped like, the few, and some argue, if they didn't exist, at all, people wouldn't have a false belief that "healthcare works fine in Russia", and would protest and maybe we could get it. All because of charity's inability to protest against existing power structures. I think it applies to alignment too, it's hard to do alignment when one gets funding from a corp that has a financial interest in "profit first safety second"
Sabs3mo53

I still don't really understand how you can do safety & alignment research on something that doesn't exist and maybe never will but I guess maybe I'm just too low-IQ to understand this Big Brain logic. Also I don't understand how everyone is freaked out about a chatbot that can't even reliably tell you basic biographical information about famous figures, for all that it can draft a very nice email and apparently write good code? idk

Reply
-3
sergia
3mo
I don't think it's close to agi either or that it's good tech. It does harm people today though. And it is an alignment problem EAs talk about: a thing that is not doing what it's supposed to is put in a position where it has to make a decision. Just not about superintelligence. See my motivation here: https://www.lesswrong.com/posts/E6jHtLoLirckT7Ct4/how-truthful-can-llms-be-a-theoretical-perspective-with-a [https://www.lesswrong.com/posts/E6jHtLoLirckT7Ct4/how-truthful-can-llms-be-a-theoretical-perspective-with-a]
3
Sabs
3mo
how does it harm people? I mean I guess there's a problem of people taking these LLM outputs as oracular truths because they don't realize how frequently they hallucinate, but isn't this just a self-correcting problem eventually as people figure it out? We don't instantly shut down access to all new tech just because people struggle to use it correctly at first.
-6
sergia
3mo
Tobias Häberli3mo40

The most recent Scott Alexander Post seems potentially relevant to this discussion.

The following long section is about what OpenAI could be thinking – and might also translate to Anthropic. (The rest of the post is also worth checking out.)

Why OpenAI Thinks Their Research Is Good Now, But Might Be Bad Later

OpenAI understands the argument against burning timeline. But they counterargue that having the AIs speeds up alignment research and all other forms of social adjustment to AI. If we want to prepare for superintelligence - whether solving the technical c

... (read more)
Reply
-2
sergia
3mo
This analysis seems to be considering only the future value, ignoring current value. How does it address current issues, like ones here? https://forum.effectivealtruism.org/posts/bmfR73qjHQnACQaFC/call-to-demand-answers-from-anthropic-about-joining-the-ai?commentId=ZxxC8GDgxvkPBv8mK [https://forum.effectivealtruism.org/posts/bmfR73qjHQnACQaFC/call-to-demand-answers-from-anthropic-about-joining-the-ai?commentId=ZxxC8GDgxvkPBv8mK] Why does a small secretive group of ppl who plan to do some sort of a "world AI revolution" that brings "UBI" (without much plan on how exactly) is by-default considering itself "good" I'm one of those who was into this secretive group of people before, only to see how much there is on the outside. Not everyone think what currently is is "good by-default" Goodness comes from participation, listening, talking to each other. Not necessarily from some moral theory. I call to discuss this plan with larger public. I think it will go well and I have evidence for this if you're interested. Thank you.
sergia3mo-7-7