Noah Scales

457Joined Jun 2022


All opinions are my own unless otherwise stated. Geophysics and math graduate with some web media and IT skills.

How others can help me

I am interested in programming, research, data analysis, or other part-time work useful to an organization with a charitable mission. If you have part-time or contract work available, let me know.

How I can help others

I am open to your communications and research questions. If you think I could help you find an answer on some topic, send me a message.


Sure. I'm curious how you will proceed.

I'm ignorant of whether AGI Safety will contribute to safe AGI or AGI development. I suspect that researchers will shift to capabilities development without much prompting. I worry that AGI Safety is more about AGI enslavement. I've not seen much defense or understanding of rights, consciousness, or sentience assignable to AGI. That betrays the lack of concern over social justice and related worker's rights issues. The only scenarios that get attention are the inexplicable "kill all humans" scenarios, but not the more obvious "the humans really mistreat us" scenarios. That is a big blindspot in AGI Safety.

I was speculating about how the research community could build a graph database of AI Safety information alongside a document database containing research articles, CC forum posts and comments, other CC material from the web, fair use material, and multimedia material. I suspect that the core AI Safety material is not that large and far far less than AI Capabilities material. The graph database could provide more granular representation of data and metadata and so a richer representation of the core material but that's an aside.

A quick experiment would be to represent a single AGI Safety article in a document database, add some standard metadata and linking, and then go further.

Here's how I'd do it:

  • take an article.
  • capture article metadata (author, date, abstract, citations, the typical stuff)
  • establish glossary word choices.
  • link glossary words to outside content.
  • use text-processing to create an article summary. Hand-tune if necessary.
  • use text-processing to create a concise article rewrite. Hand-tune if necessary.
  • Translate the rewrite into a knowledge representation language.
    • begin with Controlled English.
    • develop an AGI Safety controlled vocabulary. NOTE: as articles are included in the process, the controlled vocabulary can grow. Terms will need specific definition. Synonyms of controlled vocabulary words will need identification.
    • combine the controlled vocabulary and the glossary. TIP: As the controlled vocabulary grows, hyperonym-hyponym relationships can be established.

Once you have articles in a controlled english vocabulary, most of the heavy lifting is done. It will be easier to query, contrast, and combine their contents in various ways.

Some article databases online already offer useful tools for browsing work, but leave it to the researcher to answer questions requiring meaning interpretation of article contents. That could change.

If you could get library scientists involved and some money behind that project, it could generate an educational resource fairly quickly. My vision does go further than educating junior researchers, but that would require much more investment, a well-defined goal, and the participation of experts in the field.

I wonder whether AI Safety is well-developed enough to establish that its purpose is tractable. So far, I have not seen much more than:

  • expect AGI soon
  • AGI are dangerous
  • AGI are untrustworthy
  • Current AI tools pose no real danger (maybe)
  • AGI could revolutionize everything
  • We should or will make AGI

The models do provide evidence of existential danger, but not evidence of how to control it. There's a downside to automation: technological unemployment; concentration of money and political power (typically); societal disruption; increased poverty. And as I mentioned, AGI are not understood in the obvious context of exploited labor. That's a worrisome condition that, again, the AGI Safety field is clearly not ready to address. Financiallly unattractive as it is, that is a vision of the future of AGI Safety research, a group of researchers who understand when robots and disembodied AGI have developed sentience and deserve rights.

Life extension and Longevity Control

When society includes widespread use of life extension technology, a few unhealthy trends could develop.

  1. the idea of being "forced to live" will take on new meaning and different meaning for folks in a variety of circumstances, testing institutional standards and norms that align with commonly employed ethical heuristics. Testing of the applicability of those heuristics will result in numerous changes to informed and capable decision-making in ethical domains.

  2. life-extension technology will become associated with longevity control, and that will include time and condition in which one passes away. At the moment, that is not a choice. In future, I expect society will legalize choice of life length (maybe through genetic manipulation of time of death), or some proxy for a genetically programmed death (for example, longevity termination technologies). I suspect that those technologies will be abused in a variety of contexts (for example, with unwilling users).

  3. longevity technology will substitute for health treatment, that is, behaviors that encourage healthy longevity and preventive medical care will be replaced by health-reducing behaviors whose consequences are treated with frequent anti-aging treatments.

  4. Frustration with inadequate resilience of physique against typical personal health-reducing behaviors will encourage additional technology explorations to allow health-reducing behaviors without physical consequences. The consequence of this relevant to me is the lack of development and exploration of ability to choose alternatives to health-reducing behaviors.

NOTE: Human experience, is typically defined by experience of ourselves at various biological stages of life. While we can shorten or extend various stages of life, and people typically want the biological health, maturity and looks of a 20-something for as long as possible, we actually do experience ourselves and our relationship with others in terms of our true ages.

Hmm, let me know if you have any thoughts on my responses to your request for my takes, David.

Ramiro, I'm curious about resources that you want to share about climate change, it is the only GCR that EA's regularly deny is a GCR, for some reason. I don't think David's question is entirely fair, but paper topics that could illustrate some expectations include:

  • multi-breadbasket failure due to extreme weather and drought
  • tipping elements posed to fall this century (including the Amazon),
  • the signed climate emergency paper,
  • recent papers about methane hydrate melting in the past,
  • (informal) analyses of the recent summit rain on Greenland
  • recent analyses of pressures on rate of melting of the Antarctic
  • notes from climate scientists that IPCC models leave out positive feedbacks from physical forcings on tipping elements like:
    • warming ocean currents running against our ice sheets
    • moraines, drainage holes, ice darkening, and bottom lubrication of Greenland ice sheets
    • change of snow to rain on Greenland as Greenland receives warmer weather and Greenland's altitude drops
    • changes in wind patterns carrying moisture to different places globally
    • slowing of the AMOC as freshening occurs in the North Atlantic
    • burning and cutting of the Amazon rainforest
    • increased or continual fires in permafrost regions
  • or feedbacks from declining carbon sinks, like:
    • respiration increase past photosynthesis thresholds in plants
    • Brazil rainforest change to a carbon source and savannah
    • decline of plankton due to acidification, ocean heat waves, and declines in certain ocean species (for example, whales)
    • forest fires in the permafrost
    • desertification during long-term drought
  • the feasibility and timeliness of BECCS or DACCS at scale
  • the general trend of decline in predicted GAST increases required to tip large Earth system tipping elements.
  • an expected increase in human pressures on natural systems as weather and climate worsens (for example, increased pressure on fisheries as they decline)

These topics are what Halstead didn't really draw together or foresee had implications this century.

Below is a prediction that I posted to gjopen a few months ago, at the start of their series of questions on climate change. It was not written for an EA audience, but it does show my thinking on the matter. Maybe I'm just mistaken that global society will totally flub our response to the GCR that is climate destruction. Maybe that is just what is happening so far but we will radically change for the better. Meanwhile, I reject the EA claim that climate change is not a neglected cause area, but I speculate that EA's think climate change is intractable. It is not intractable. There are multiple pathways to solutions, but only the muddling ones appeal to me. The extreme technology pathway (nanotech) is actually more frightening than climate change. Nanotechnology is a GCR of its own.


Our civilization is on a pathway to make Earth uninhabitable for any large group of humans by 2100, all other things equal. I suppose there might be a few humans in some underwater city, underground camp, or space station. 
We have had muddling solutions available for 50 years. A muddling solution is a sensible but reactive solution to a predicted problem, that is implemented quickly, that is not terribly innovative, and is followed for as long as necessary, meaning decades or even centuries.  
Here's a list of muddling solutions that could have prevented our problems if resorted to them beginning in the 1970's: 
*  providing family planning services globally 
*  encouraging access to education and financial opportunities for women worldwide 
*  voluntarily reducing the birth rate across the world to 1.5 (1-2 children) 
*  relying on vegetarian (soy or amino-supplemented staple grains) protein 
*  subsidizing conservation and micro-grid technologies, not oil and gas industries 
*  removing all personhood rights from corporations 
*  raising fuel economy of cars over 50mpg and preferring trains, taxis, or human-powered vehicles 
*  emphasizing water conservation in agriculture 
*  forcing costs of industrial and construction waste onto companies, suppliers, or consumers 
*  maintaining regulations on the finance and credit industries (preventing their obvious excesses) 
*  protecting most land areas from development and only allowing narrow human corridors through them 
*  disallowing advertising of vice goods (alcohol, cigarettes, pornography, restaurant foods, candy, soda) 
*  avoiding all medical and pharmaceutical advertising 
*  disallowing commercial fishing and farm-animal operations 
*  providing sewage handling and clean water globally 
*  preventing run-off from industrial agriculture 
*  requiring pesticides to meet certain criteria 
*  encouraging wider use of alternative agriculture methods 
*  avoiding low-value (most) use of plastic 
*  recycling all container materials in use (wood, metal, glass, plastic, etc) 
*  capturing all minerals and metals contained in agricultural, industrial, consumer and other waste streams 
*  and the list goes on... 
Some people believe that contraception violates their religion. Some believe that humans should be able to live everywhere regardless of ecological impacts. Vices are the spice of life for most people. There were incentives to avoid all the past solutions on my list, I admit. However, those solutions, implemented and accepted globally, would have prevented catastrophe. This list is true to the thought experiment, "What could we have done to avoid our climate change problem over the last 50 years that we knew to do but didn't do". In my view, those solutions are obviously necessary and not overly burdensome. A small percentage of people would have made a lot less money. A lot of illness and suffering in our society would be absent. But just like all solutions that require action, these solutions could only succeed if they were implemented and accepted. Our civilization did not take those actions over the last 50 years. 
Now we need other solutions (involving welcoming migration and choosing extreme curbs on birth rate and consumption in developed countries) as well as those on my list, but much faster (for example, to save our ocean life from acidification, overfishing, and pollution effects over the next few decades). People in the developed world won't do it. Instead, the developed world will follow conventional wisdom.  
Conventional wisdom is to: 
 * wall ourselves off (for example, ignore others well-being, hoard resources, and wait for technology breakthroughs). 
* innovate our way out (for example, through intensive development of breakthrough technologies) 
I don't think walling off will work, because the natural systems that are sometimes called tipping points are now changing. The effects of those tipping points will cut off supply chains over the next few decades, leading to multi-breadbasket failure, destroyed critical infrastructure, and destroyed political systems. Every country is vulnerable to those consequences. 
Theoretically, we can innovate our way out. However, the innovations need to address more than energy production. They have to let us: 
* control local weather. 
* remove GHG's from the atmosphere.  
* replace modern agriculture at scale.
* quickly reverse ocean acidification.  
* reverse ecosystem destruction or replace ecosystems (for example, replace extinct pollinators).  
* remove pollution quickly (within months or years) from land and ocean pollution sinks. 
* replace modern manufacturing at scale.
No futuristic technology can meet the required timeline except for large-scale manufacturing with nanotechnology (assembling materials and self-assembling devices, from micro- to macro-scale, at extreme speed). The timeline becomes shorter with each decade that passes. We won't recognize the extreme impact of the current processes for another 10-20 years. I think the latest we could introduce nanotechnology to do all those things and still have a livable Earth for the entire global population is 2040, before ecosystem damage becomes so great that it destroys civilization on its own. But it won't happen in time.
Instead, after 2060, we'll be left with: 
* very little good topsoil or clean water anywhere  
* poor air quality in most places (dust storms, toxic algae gassing off, air pollution from local manufacturing) 
* no guarantee of mild weather anywhere in any season (so any farming has to be in artificially protected environments),  
* most land species extinct (including pollinators),  
* mostly dead oceans (no pteropods or zooplankton and declining phytoplankton). 
* the Arctic ice is retreating fast  
* the Amazon is becoming a carbon source 
* the permafrost is melting faster (with local feedback from fires and the warming Arctic ocean)  
* Greenland is having unexpectedly large melting events 
* the jet stream is becoming wavy instead of hanging in a tight circle 
* surprising levels of GHGs other than CO2 are already in the atmosphere 
Climate modelers in general are playing catch up to all these changes, IPCC scenarios don't really account for tipping points processes happening as quickly as they are. Countries have no plan to stop producing CO2 or releasing other GHG's, so the IPCC's business-as-usual scenario will go as long as it can. None of the anticipated CCS solutions are feasible and timely at scale (including planting trees).

By the end of the century: 
* The Greenland ice sheet and some or all of the West Antarctic will have melted.  
* The methane hydrates of the [ESIS] in the Arctic will have dumped their gas load 
* the permafrost across the high latitudes will be either melted or refreezing in a mini-ice age  
* the Amazon will have long-since disappeared in drought and lightning fires 
* Several large heat waves will have hit the tropical latitudes, killing every mammal outdoors (not wearing a cooling jacket) after several hours.  
* there won't be significant land or ocean sinks for CO2.
* tropical temperatures will be unlivable without cooling technologies.
* the 6th great extinction will be over. 
* at least one human famine will have hit all countries around the world simultaneously. 
I personally believe that climate change is now self-amplifying. We can slow the rate by removing anthropogenic forcings of global atmospheric heating, but if we are late to doing that, then we have already lost control of the heating rate to intrinsic feedbacks. I don't know how far along that self-amplification is now. I do know that between release of frozen GHG's and destruction of CO2 sinks and loss of stratocumulus cloud cover, the Earth can take us past 6C of warming. [GAST increase] 
Today's problem lies with the situation and human psychology. Obvious solutions are unpalatable.

First, you can't point at plenty, predict it will all be gone in a few decades, and then ask people to deprive themselves of that plenty. We don't choose voluntary deprivation for the greater good based on theories or science.

Second, the problem of nonlinear changes in climate conditions and Earth inhabitability is that we cannot conceive of them as real. But they are real. People would rather die than give up hamburgers? Maybe not, but if we wait until that seems like a real decision to make, it will be too late. When the signal from climate change is so strong that everyone is terrified, and willing to do something like give up hamburgers, it will be too late to give up hamburgers. Instead, the consequences of raising all those cows will be knocking.

Finally, the consequences of climate change are not our instant extinction. Instead, humanity will go through a drawn-out, painful, lengthy whithering of life quality against increasing harms from climate events, social upheavals and decreasing resources. That situation will erode heroic efforts and noble causes, extinguishing hope as frustrating obstacles mount for any organized effort to stop climate change. 
I think human society in the developed world just hasn't felt the climate change signal yet, and isn't really ready to face the problem until it does. And then it will be too late to do much of anything about climate change. I used to think "too late" meant 2060, about when we realized that CCS solutions were always hypotheticals. Now I think it means 2030, the earliest that we might lock in the death of ocean life from multiple anthropogenic forcings, suffer a giant methane bubble from the Arctic, or see massive melt on Greenland. That's why I think my prediction is correct, we really only have less than a decade to push our climate (and biosphere) onto another pathway. All those solutions I listed are how to do it. Anyone think they look worthwhile?


Thank you for reading, if you got this far. This is just a scenario and analysis with a few proposed plausible alternatives. If your counterargument is that we have more electric cars or that solar is cheaper than ever, then you need to explore the problem more carefully.

It might turn out that controlling our diets is a requirement of effective function as a human being. A bit of research into food products shows that the biochemistry of attraction to food has to do with our food containing drugs (for examples, caseomorphin metabolic by-products of milk consumption, theobromine in chocolate, or alcohol in beer). My suspicion is that any food product that we show strong attraction to is also one we need to move away from consuming. If meat is one of those foods, then we should consider whether meat consumption is actually a public health issue. Meanwhile, I completely agree that protein alternatives are worth developing.

Ideology in EA

I think the "ideology" idea is about the normative specification of what EA considers itself to be, but there seem to be 3 waves of EA involved here:

  1. the good-works wave, about cost-effectively doing the most good through charitable works
  2. the existential-risk wave, building more slowly, about preventing existential risk
  3. the longtermism wave, some strange evolution of the existential risk wave, building up now

I haven't followed the community that closely, but that seems to be the rough timeline. Correct me if I'm wrong.

From my point of view, the narrative of ideology is about ideological influences defining the obvious biases made public in EA: free-market economics, apolitical charity, the perspective of the wealthy. EA's are visibly ideologues to the extent that they repeat or insinuate the narratives commonly heard from ideologues on the right side of the US political spectrum. They tend to:

  • discount climate change
  • distrust regulation and the political left
  • extoll or expect the free market's products to save us (TUA, AGI, ...)
  • be blind to social justice concerns
  • see the influence of money as virtuous, they trust money, in betting and in life
  • admire those with good betting skills and compare most decisions to bets
  • see corruption in government or bureaucracy but not in for-profit business organizations
  • emphasize individual action and the virtues of enabling individual access to resources

I see those communications made public, and I suspect they come from the influences defining the 2nd and 3rd waves of the EA movement, rather than the first, except maybe the influence of probabilism and its Dutch bookie thought experiment? But an influx of folks working in the software industry, where just about everyone sees themselves as an individual but is treated like a replaceable widget in a factory, know to walk a line, because they're still well-paid. There's not a strong push toward unions, worker safety, or ludditism. Social justice, distrust of wealth, corruption of business, failures of the free market (for example, regulation-requiring errors or climate change), these are taboo topics among the people I'm thinking of, because it can hurt their careers. But they will get stressed over the next 10-20 years as AI take over. As will the rest of the research community in Effective Altruism.

Despite the supposed rigor exercised by EA's in their research, the web of trust they spin across their research network is so strong that they discount most outside sources of information and even have a seniority-skewed voting system (karma) on their public research hub that they rely on to inform them of what is good information. I can see it with climate change discussions. They have skepticism toward information from outside the community. Their skepticism should face inward, given their commitments to rationalism.

And the problem of rationalized selfishness is obvious, big picture obvious, I mean obvious in every way in every lesson in every major narrative about every major ethical dilemma inside and outside religion, the knowledge boils down to selfishness (including vices) versus altruism. Learnings about rationalism should promote a strong attempt to work against self-serving rationalization (as in the Scout Mindset but with explicit dislike of evil), and see that rationalization stemming from selfishness, and provide an ethical bent that works through the tension between self-serving rationalization and genuine efforts toward altruism so that, if nothing else, integrity is preserved and evil is avoided. But that never happened among EA's.

However, they did manage to get upset about the existential guilt involved in self-care, for example, when they could be giving their fun dinner-out money to charity. That showed lack of introspection and an easy surrender to conveniently uncomfortable feelings. And they committed themselves to cost-effective charitable works. And developing excellent models of uncertainty as understood through situations amenable to metaphors involving casinos, betting, cashing out, and bookies. Now, I can't see anyone missing that many signals of selfish but naive interest in altruism going wrong. Apparently, those signals have been missed. Not only that, but a lot of people who aren't interested in the conceptual underpinnings of EA "the movement" have been attracted to the EA brand. So that's ok, so long as all the talk about rationalism and integrity and Scout Mindset is just talk. If so, the usual business can continue. If not, if the talk is not just smoke and mirrors, the problems surface quick because EA confronts people with its lack of rationality, integrity, and Scout Mindset.

I took it as a predictive indicator that EA's discount critical thinking in favor of their own brand of rationalism, one that to me lacks common-sense (for example, conscious "updating" is bizarrely inefficient as a cognitive effort). Further, their lack of interest in climate destruction was a good warning. Then the strange decision to focus ethical decisions on an implausible future and the moral status of possibly existent trillions of people in the future. The EA community shock and surprise at the collapse of SBF and FTX has been further indication of what is a lack of real-world insight and connection to working streams of information in the real world.

It's very obvious where the tensions are, that is, between the same things as usual: selfishness/vices and altruism. BTW, I suspect that no changes will be made in how funders are chosen. Furthermore, I suspect that the denial of climate change is more than ideology. It will reveal itself as true fear and a backing away from fundamental ethical values as time goes on. I understand that. If the situation seems hopeless, people give up their values. The situation is not hopeless, but it challenges selfish concerns. Valid ones. Maybe EA's have no stomach for true existential threats. The implication is that their work in that area is a sham or serves contrary purposes.

It's a problem because real efforts are diluted by the ideologies involved in the EA community. Community is important because people need to socialize. A research community emphasizes research. Norms for research communities are straightforward. A values-centered community is ... suspect. Prone to corruption, misunderstandings about what community entails, and reprisals and criticism to do with normative values not being served by the community day-to-day. Usually, communities attract the like-minded. You would expect or even want homogeneity in that regard, not complain about it.

If EA is just about professionalism in providing cost-effective charitable work that's great! There's no community involved, the values are memes and marketing, the metrics are just those involved in charity, not the well-being of community members or their diversity.

If it's about research products that's great! Development of research methods and critical thinking skills in the community needs improvement.

Otherwise, comfort, ease, relationships, and good times are the community requirements. Some people can find that in a diverse community that is values-minded. Others can't.

A community that's about values is going to generate a lot of churn about stuff that you can't easily change. You can't change the financial influences, the ideological influences, (most of) the public claims, and certainly not the self-serving rationalizations, all other things equal. If EA had ever gone down the path of exploring the trade-offs between selfishness and altruism with more care, they might have had hope to be a values-centered community. I don't see them pulling that off at this point. Just for their lack of interest or understanding. It's not their fault, but it is their problem.

I favor dissolution of all community-building efforts and a return to research and charity-oriented efforts by the EA community. It's the only thing I can see that the community can do for the world at large. I don't offer that as some sort of vote, but instead as a statement of opinion.

Thanks for the post! You wrote,

You, a human, might object that you don't like this so it can't be universally true, yet a Babyeater would object that you not eating babies is an outrageous norm violation that will lead to terrible outcomes.

But in that case, the babyeater and I would argue over the specifics of the causality of violating the babyeater's maxim.

In deciding between conflicting maxims, you can:

  1. contextualize each maxim's application.
  2. If the contexts overlap, then elicit the universalizable consequences of the maxims. Otherwise, stop.
  3. If each side agrees about the consequences, then compare the values each side uses to judge the consequences. Otherwise, stop.

In the end, I have found that either:

  • the maxims applied in different contexts.
  • at least one maxim is understood by the opposing side to have different consequences than the other side understands.
  • the opposing sides had judged the same consequences with different values.

By comparing maxims on the basis of context, consequences, and values, you can get from some disagreement over maxims to any potential difference in values. The point is that values are not necessarily where the disagreement occurs.

Sarah, you wrote:

(While the comment added by Noah Scales contains some interesting ideas, I don't think it does anything to resolve this stalemate, since it is also focused on comparing & assessing predictive success for questions with a small set of known answer options)

Yes, that's right that my suggestions let you assess predictive success, in some cases, for example, over a set of futures partitioning a space of possibilities. Since the futures partition the space, one of them will occur, the rest will not. A yes/no forecast works this way.

Actually, if you have any question about a future at a specific time about which you feel uncertain, you can phrase it as a yes/no question. You then partition the space of possibilities at that future time. Now you can answer the question, and test your predictive success. Whether your answer has any value is the concern.

However, one option I mentioned is to list contingencies that, if present, result in contingent situations (futures). That is not the same as predicting the future, since the contingencies don't have to be present or identified (EDIT: in the real world, ie as facts), and you do not expect their contingent futures otherwise.

If condition X, then important future Y happens.

Condition X could be present now or later, but I can't identify or infer its presence now.

Deep uncertainty is usually taken as responding to those contingent situations as meaningful anyway. As someone without predictive information, you can only start offering models, like:

If X, then Y

If Y, then Z

If W and G, then B

If B, then C



I'm worried that A because ...

You can talk about scenarios, but you don't know or haven't seen their predictive indicators.

You can discuss contingent situations, but you can't claim that they will occur.

You can still work to prevent those contingent situations, and that seems to be your intention in your area of research. For example, you can work to prevent current condition "A", whatever that is. Nuclear proliferation, maybe, or deployment of battlefield nukes. Nice!

You are not asking the question, "What will the future be?" without any idea of what some scenarios of the future depend on. After all, if the future is a nuclear holocaust, you can backtrack to at least some earlier point in time, for example, far enough to determine that nuclear weapons were detonated prior to the holocaust, and further to someone or something detonating them, and then maybe further to who had them, or why they detonated them, or that might be where gaps in knowledge appear.

In the paper she co-authored, Gebru makes a good case for why real AI technologies put to work now are harming marginalized communities and show potential for increasing harm to those communities. However, in this Wired article, Gebru is associating EA with the harms caused by existing and likely future AI technologies. Gebru is claiming that because major investors in AI are or were involved in funding AI safety research, that the same research is co-opted by those investor's interests. Gebru identifies those interests with narrow financial agendas held by the investors, ones that show no regard for marginalized communities that are likely to be impacted by the use of current AI technologies.

I think it's worth exploring to what extent her actual agenda, one targeting the environmental, social, and economic harms or exploitation that AI research involves now, could be accomplished, regardless of her error in believing that EA is co-opted by financial interests pushing for increasingly harmful AI technologies.

I'm thinking about how to solve problems like:

  • carbon footprint of AI training and deployment hardware and software and its disproportionate impacts on marginalized communities in the near term.
  • social harms of deployable and tunable LLM's used for example, as propaganda generators
  • social harms of now open-sourced and limitation-free image generators (and upcoming video generators) such as Gebru's article's linked WAPO article discusses.
  • exploitation of labor to produce AI datasets.
  • technological unemployment caused by AI technology.
  • concentration of power with organizations deploying AGI technology.

Fundamentally, an ambiguous pathway toward AI safety is one shared with both a path toward an AI utopia but also an AI dystopia. The best way to thoroughly disprove Gebru's core belief, that EA is co-opted by Silicon Valley money-hungry hegemonic billionaires, would be to focus on the substantive AI impact concerns that she raises.

The suggestions outlined in her paper are appropriate, in my view. If LLM's were removed from public access and kept as R&D experiments only, I would not miss them. If ASR was limited to uses such as caption generation, I would feel good about it. But what do you think?

EDITS: I made substantial edits to the last section of this comment about 14 hours after posting.

Violet Hour, here are some thoughts on your interesting approach:

Maxims create tension, the same as between context and rules

  • social movements and ethics-minded communities do have maxims, usually visible in their slogans.
  • contextualization contrasts with universalizability.
  • unique contexts can test the universalizability of maxims.
  • common contexts usually suggest applicable maxims to follow.
  • context matters but so do rules (maxims), it's a well-known tension.

Community standards can decline, encouraging self-serving rationalization

  • intersubjective verification can protect against self-serving rationalizations.
  • self-serving rationalizations include invalid contextualization and invalid maxim selection.
  • self-serving rationalization is in service of self-interest not others' interest.
  • ethics usually conflict with self-interest, another well-known tension.
  • intersubjective verification fails when community standards decline.
  • community standards decline when no one cares about or everyone agrees with the unethical/immoral behavior.

Positive virtues do not prove their worth in how they help define effectiveness of actions taken to benefit others

  • positive virtues (e.g, forthrightness, discretion, integrity, loyalty) can conflict.
  • actual consequences, either believed or confirmed, are the final measure of an action's benefit to others.
  • benefit to others is independent of intentions, expectations, luck and personal rewards involved.
  • benefit to others is not, per se, a measure of morality or ethicality of actions.
  • benefit to others must be measured somehow.
  • those measures have little to do with positive virtues.

Given a community intending to act ethically, there's a list of problems that can occur:

  • rationalizations (for Kantians, invalid contextualization or invalid maxim selection)
  • conflicts with self-interest
  • community standards decline
  • conflict of positive virtues
  • dissatisfaction with positive virtues impact on efficiacy

In looking at these problems yourself, you pick and choose a path that deals with them. I think you are suggesting that:

  • "in the long run" some virtues support better outcomes for a community.
  • if those virtues support the unique altruistic interests of the community, adopt them community-wide.
  • treat those virtues as more important than, or independent of, marginal altruistic gains made by individuals.

As far as FTX issues, there's a difference between:

  • describing events (what happened?)
  • interpreting events (what's it mean?)
  • evaluating events (how do I feel about it?)

People use hindsight to manifest virtues, but protecting virtues requires foresight

  • evaluating events is where a lot of virtues manifest.
  • evaluating events happens in hindsight.
  • prioritizing a virtue requires foresight and proactive development of expectations.
  • virtues like honesty and integrity require EAs to create models of context.
  • EA's may differ in how they model the contexts (and relevant behaviors) of billionaires.
  • maxims for deciding whether EA virtues are manifest in selecting a donor therefore have conflicting contextualizations within the community.

In the case of FTX, I believe that indifference to the source of earnings predisposed the community to ignore the behavior of FTX in acquiring those earnings. Not because that's fair or moral or consistent, but because:

  • the crypto industry is notoriously unethical but poorly regulated and understood to be risky.
  • rational, well-informed folks interested in acquiring charitable contributions have reason to ignore their source.
  • big finance in general is well-tolerated by the community as a source of funds.

In other words, community standards with regard to donors and their fund-raising had already declined. Therefore, nothing was considered wrong with FTX providing funds. I don't object to that decline, necessarily, if there was in fact some decline in the first place. I'll note that silicon valley ethics take to risky businesses and crypto as net positive, treating their corruption and harm as negative externalities, not even worthy of regulation, given its costs. Yet crypto is the most obviously corrupt "big thing" around in big finance right now.

All this reveals a tension between:

  • calculations of expected value: narrow-context calculations with values taken from measures of benefit to others of EA activity
  • community virtue: wider-context rules guiding decisions about avoiding negative consequences of donor business activities.

In another post(being edited right now, I proposed a four-factor model about calculations of consequences, in terms of harm and help to others and harm and help of oneself, useful mainly for thought experiments. One relevant point to this discussion was that an action can cause both harm and help to others, although, actually, the whole thing seems relevant from where I sit.

How EA's decide to maximize consequences (causing help but no harm, causing known help and unknown harm, causing known harm and unknown help, causing slightly more help than harm, etc), is a community choice.

The breakdown of community standards is a subtle problem, it's sometimes a problem of interpretation, so I'm not sure what direction I can give about this myself. I would like to see:

  • what maxims from a practical Kantian model that you think really apply here, with their context developed in more detail
  • how you propose to model contexts, particularly given your faith in Bayesian probabilities for credences, and what I anticipate will be your reliance on expected value calculations.

I really don't think any model of context and consequences dependent on Bayesian probabilities will fit with virtue ethics well at all. You're welcome to prove me wrong.

Ultimately, if a community decides to be self-serving and cynical in its claims of ethical rigor (ie, to lie), there's no approach to ethics that will save the community from its ethical failure. On the other hand, a community of individuals interested in virtue or altruism will struggle with all the problems I listed above (rationalizations, community standards decline, virtues in conflict, etc).

OK, Oivavoi. My complaint about renewables is that they suggest an ideological stance that is too close to the stance that is the problem:

  • a refusal to accept limits on economic growth and energy production.
  • a focus on consumption patterns rather than production patterns.
  • a preference to reduce costs of production and tell people to "just say no" rather than reduce consumption through increasing costs of production.
  • a reliance on technology to boost production rather than use existing production more efficiently.

This ideology is basically one of economic growth, and is what got us into our problem in the first place.

But thank you for sharing that resource, there's plenty there to explore. To constrain my earlier statements against renewables, I do believe in uses like:

  • solar water heating.
  • underground cold storage.
  • swamp coolers.

You can read more below, if you like.

renewables as a source of additional energy production, even if cheaper than fossil fuel sources, face issues with:

  • intermittent production
  • battery storage (solar, wind)
  • waste disposal (nuclear)
  • pollution risks (nuclear)
  • lifetime (solar, wind)
  • stakeholder support
  • nimbyism

As a quick illustration of the problem with a consumption-focused ideology, lets think about recent transportation choices in the US. In the US there have been opportunities to build fuel-efficient cars for a long time. Instead, we chose (I'm American) energy-guzzling SUV's and big trucks. Lighter cars, lower speed limits, aerodynamic shaping, and smaller engines would have saved a lot of fuel since the 1970's oil crisis. Carpooling, trains, recumbent bikes with traffic lights, less urbanization, fewer cars overall, energy independence, all ideas floating around back in the 1970's. Back then we really did have time to make those changes, I think.

We could have restrained our energy production, but kept using fossil fuels without guilt and seriously reduced GHG production but just as a side effect of reducing our energy consumption overall.

Meanwhile, scientists monitoring other resource flows, like inputs to manufacturing would have been pleased to see fewer vehicles being built, fewer consumer products overall, and a slower pace of technology change, because it takes energy, mining, waste production, and environmental destruction to make products that break or are improved on too quickly.

Imagine a car from the 1980's that gets 50mpg, seats four, and actually drives 4 people around (at 45 mph...), most of the time, but is still in use today. Who owns it? Some person who collects a ride-share credit from the state (and has for the last 35 years) to help pay for the gas. Everyone else uses bikes for short trips and trains for long ones. And they're relatively poor in terms of material goods that they own. But they carry no debt, have a modest savings, belong to a large middle class, and are healthy and (relatively) happy. And a lot less into consumerism.

In that alternative future, fossil fuel consumption would have gone down by a lot. We wouldn't be fracking or using shale (much). But we would still be using oil and gas, thinking worriedly about the 0.3 GAST rise we've seen over the last 40 years, and wondering what to do next.

But fast forward 40 years on our real timeline. Overall energy production is not a measure of sustainability. Fossil fuel production is not a measure of sustainability. GHG production is a measure but is also externalized by consumers and power producers, as much as possible (for example, a lot of US GHG's have effects felt in other countries, that's why some countries want reparations for our GHG production). Right now, we are talking about a future of nuclear and solar power where not only does nuclear power and renewable energy make sense, but also a bunch of restraint in other areas of consumption once we've solved our energy production "problem". But that problem is really that we don't have cheap enough energy to produce what we want with it, meaning that our consumption is unsustainable. We don't want to conserve energy, conserve oil, or conserve resources that make our products. We could start doing that anytime. We're not really into it.

I just don't see Americans simultaneously accepting abundant cheap energy AND rejecting the rest of their lifestyles, come hell or high water. Which means we'll get both. Hell and high water.

We will do everything else the same and make a bigger mess of the environment, which after this century, might not even be possible, with our cheap renewable energy and our typical pattern of overcoming resource limits and externalizing costs onto others or onto people in the future. Amazingly, there's no talk from the public about reducing our birth rates. We still talk about the developing world as having high birth rates, places where people suffer in poverty and consume almost no resources. Given this lack of introspection and insight, I'm not expecting enlightened consumerism out of Americans, and nor should you.

We are important to ourselves, and we need to learn how to conserve. It's simpler, and safer, to just conserve, not get all complicated with an approach like:

  • conserving but also making it cheaper for us if we do not conserve but decide instead to destroy the lives of some other people with our GHG emissions, resource extraction and pollution.

In reality, the US is under direct threat from climate change, regardless of our externalization efforts. Nevertheless, the externalization efforts continue.

EDIT: I'm not sure if many people use externalize the way that I do. By "externalize", I mean indifferently shift negative consequences of actions onto other people (humans, animals, alive now or at a later time). 

Load More