The difference between human extinction and a wide range of positive outcomes is dependent upon how technology is developed and implemented. The EA community knows this better than anyone.

But a sustainable process to maximize the positive contributions of technology to human well-being while minimizing the probability of existential and global catastrophic risks requires participation by domain leaders in tech development, policy, academia, and business. This is currently not the case.

One of the ways to remedy this is by targeting future domain leaders. That’s why I’m excited to announce a new EA-aligned organization, Envision, with the goal of imbuing a forward-looking but safety-conscious mindset towards technology in future leaders in tech development, policy, academia, and business by intervening at the college level. The goal is two-fold: convince future domain leaders that technology will play a pivotal role in humanity’s future, and instill in them the careful consideration of technological safety as a core value.

This post will:

1. Outline Envision’s current status
2. Justify the claims made above
3. Outline Envision’s goals and strategy
4. Define metrics of success and measurement mechanisms
5. Review failure modes
6. Explain how EAs can potentially be involved

 

Current status

As a general outline, Envision will be a global network of student chapters with the goal of imbuing a forward-looking but safety-conscious mindset towards technology in future leaders in tech development, policy, academia and in relevant business leaders.

Envision is currently a 6-month old student group at Princeton, with 15 officers and 91 members, which will hopefully significantly increase with fall recruiting. Envision’s website, although still under construction, provides more information about past events.

Students at MIT, Stanford, Cambridge, and UPenn have tentatively expressed interest in opening a chapter.

The locus of Envision is our Conference, Dec 2-4, which will serve as a focal point for recruiting and kickstart many of the processes and strategies I am about to describe.

Our current official partners include MIRI, Future of Life Institute, Jaan Tallinn, and Sam Altman among others. Confirmed speakers at our conference include Andrew Critch from MIRI, Robin Hanson, Anders Sandberg from FHI, and Ruth Chadwick from the Centre for Economic and Social Aspects of Genomics. MIRI will most likely host a workshop during the conference.

 

Justification of claims

 

This section will expand upon the claims made in the introduction.

1.     The difference between human extinction and a wide range of positive outcomes is dependent upon how technology is developed and implemented.

   • Existential risk is inherent in artificial intelligence, synthetic biology, and nanotechnology, as argued by Nick Bostrom among others.
   • Even given that existential risk is avoided, these technologies, in addition to genetic engineering, space technology, fusion, virtual and augmented reality, and neuro-engineering, have a wide range of possible impacts, from the catastrophic to unprecedented advances in human well-being and capabilities.
   • Given the rapid pace of technological development,
     • Other technologies are likely to arise which will be crucially important to humanity's future, and
     • The impacts of the development of these technologies is likely to be shaped by actions in the short to medium-term and will be felt within a century.
2. But a sustainable process to minimize the probability of existential and global catastrophic risks while maximizing the positive contributions of technology to human well-being requires participation by domain leaders in tech development, policy, academia, and business.
   • The argument is often made that AI is more safely developed by a small group of people, without significant domain leadership involvement. However, the key word is sustainable: the future is unpredictable and new technologies and issues will crop up. It is not sustainable to have to convince the relevant players and assemble a team anew every time. As Nick Beckstead argues, broader and more general interventions, including in education, are important to shaping the far future.
   • Furthermore, ‘participation’ is different from ‘exposure’ and is assumed to be positive. A misinformed elite meddling in AI safety is not a desirable outcome; an elite participating in safe technological development is a positive outcome. Participation is far harder to achieve than exposure, but because of how Envision is set up, is achievable.
   • Technology does not exist in a vacuum. How it is implemented is as important as how it is developed. The actions taken by governments, academic institutions, and businesses both influence technological development and determine implementation.
3. This is currently not the case.
   • Although a small number of domain leaders in tech development and policy are now Effective Altruists and/or are engaged with AI and other technologies in a safety-conscious manner, this remains the exception.
4. The way to remedy this is by targeting future domain leaders during university.
   • College strikes a balance between flexibility and accessibility on the one hand, and intellectual maturity and indentifiability on the other.
     • Flexibility: Still sufficiently early that beliefs are in flux as students are exposed to new ways of viewing the world, and the most important career choices still lie in the future. Many smart, socially aware students default to tech, consulting, and finance for lack of prestigious and effective alternatives, which can be more easily avoided pre- than post-decision.
     • Intellectual maturity: However, it is late enough such that existential risk, exponential growth, the technical details of technology, and other important concepts can be grasped.
     • Accessibility: Future domain leaders are still easily accessible at low cost.
     • Identifiability: However, it is late enough that future domain leaders are considerably more readily identifiable and can be specifically reached out to.
   • One major disadvantage is the more extended time frame before interventions reap benefits, as it takes time before current students reach the top positions within their domain. This is valid, but there are reasons to think a college intervention now will almost certainly pay off.
     • Sub-elites exert significant influence on the culture of domain leaders, so effects will be realized considerably before the future domain leaders that are exposed to Envision assume full control. The impact of the intervention begins increasing from the moment students enter the workforce and is thus not as long-term as it appears.
     • The massive impact a successful university-level intervention would have in causing a significant shift in mindsets is likely to justify the longer time until full realization of impact.
   • Intervening with future domain leaders also reaps longer rewards: current domain leaders have considerably shorter time left in power.
   • Identifiability is an issue; current domain leaders can be far more easily identified than future elites. There will certainly be a lower success rate of identification, but what matters is the absolute number of future domain leaders affected, not the percentage of those affected who become domain leaders. It is thus theoretically possible for all future leaders to be influenced despite a low success rate of identification if enough students are reached.
5. There currently does not exist an organization that does what Envision does.
   • There is no mechanism by which college students are exposed in a single context to a balanced view of both the potential and the dangers of rapidly developing disruptive technologies.
   • X-Risk organizations and student groups do not appeal to most future domain leaders, who are more excited by the potential of technology to improve the world.
   • Tech-specific and issue-specific groups exist, but none are holistically oriented to take into account the bigger picture.
   • There is no mechanism for pulling in future policy makers and business leaders into the same space and mindset.
   • Although online resources exist, they are not easy to find, and the message is not brought to most future domain leaders. There is thus no easy way to access the information without an existing initial interest or recommendation.

 

Envision – The Strategy

 

As a summary, Envision will be a network of student chapters at top unis worldwide with the goal of imbuing a forward-looking but safety-conscious mindset towards technology in future leaders in tech development, policy, academia and in relevant business leaders.

This section provides an overview of Envision’s strategy for accomplishing this.

 

Goals can be broken down into four separate categories:

1. Attract and retain future leaders across the domains relevant to future technology development.
2. Convince future leaders that technology will play a pivotal role in humanity’s future and integrate technology safety into their mindset.
3. Provide the tools to contribute towards the positive development of technology.
4. Providing a pipeline into EA.

 

Strategies for accomplishing goals

 

For attracting and retaining future leaders

Leaders must first be identified. For this, we are developing a predictive model identifying key variables current domain leaders had in common during university.

 

Leaders must then be attracted. There seem to be three factors that have widespread appeal:

• Positive messaging emphasizing the potential of technology while integrating concern for safety. Envision’s official mission is inspiring students to pioneer a brighter future through the responsible and innovative use of rapidly developing disruptive technologies, and a lot of our events focus on the positive applications of technology.

• Domain leadership in the fields they’re interested in. To appeal to this, our strategies are:
• • Partnering with domain-leading organizations and individuals. Our current list includes MIRI, FLI, Sam Altman, and Jaan Tallinn, and we are in the process of expansion.
  • Interfacing with firms on the cutting edge of technology, through inviting them to demo their technologies and visiting their offices and labs.
  • Partnering at each university with established domain-leading student groups. At Princeton, our founding location, we are partnered with Entrepreneurship Club.
  • Inviting domain-leading speakers.
• Developing prestige to attract the most talented students. This includes:
  • emphasizing selectivity in officer and fellow selection, although membership will be open to all
  • partnering with prestigious organizations
  • including a competitive element
  • ensuring sleek design and streamlined operations
  • targeted reach-out.

 

 

Additionally, we will maintain independence from EA. Although there will be links with EA, partnerships with EA orgs, and many EA members, Envision will remain officially non-affiliated to maintain its appeal to those who would be off-put by EA and maintain the differentiation of its message.

 

The above strategy is highly subject to change as we learn more about what most attracts future leaders and we will continuously update our strategies in response to new information.

Convince future leaders that technology will play a pivotal role in humanity’s future and integrating technology safety into their mindset.

It’s important to integrate both these aspects and present safety as a standard component of technological development, not a stand-alone add-on as it is often portrayed or interpreted.

This is best accomplished by emphasizing technology’s vast potential while consistently addressing safety concerns. A dichotomy between positive applications and negative risks is to be avoided.

 

Strategies:

• Provide exposure to companies and labs on the cutting edge of rapidly developing disruptive technologies. This makes clear the speed with which technological development is occurring and how close we already are to revolutionary breakthroughs, and the concrete practical applications and implications.
• Integrate safety as a principle into all events and issues, even for small concerns. This will establish it as an ingrained attitude. To do this, we will ask all speakers invited, firms visited, and other relevant parties to address how to mitigate the potential negative outcomes of their technology, and we will address safety in discussions.
• Host specific safety events in the form of talks by safety-related experts, including in:
  • Tech development, ie companies such as DeepMind
  • Technical safety, such as MIRI
  • Safety considerations, such as FHI and FLI
• Curation of online resources, including papers such as Nick Bostrom’s, major news updates, and overviews of relevant technologies.
• Discussion. This is a crucial element to complement the outside expert views with the inside, socially acceptable peer views. Envision will hopefully have many Effective Altruists, so discussion will be one of the most potent tools for integrating a safety mindset.

It is important to note Envision focuses on technological developments and issues that are not commonly known, which is generally correlated with the longer-term. For example, the ethics of self-driving cars are already widely discussed and thus do not fall within the scope of Envision.

 

For providing the tools to contribute to the safe development of technology

 

• Envision Entrepreneurship: A competition for the most innovative application of a rapidly developing disruptive technology for large-scale impact, with prize money. With time, partnering with organizations to provide a more direct route for the winning ideas to be implemented. As with everything, safety considerations will be integrated. Being piloted at the conference.
• Interfacing with organizations that are working on innovative tech development. Includes tech firms and start-ups, VC firms, government agencies, research labs, think tanks, and tech safety research organizations. Exact programming highly dependent on organizations. Those deemed to be negatively contributing to human well-being due to an active disregard for safety considerations will be avoided.

• The purpose of this interfacing is both to provide the means for working on safe technological development and smoothing the path towards domain leadership.
• Will include career-oriented interaction with relevant organizations, including career fairs and in select cases presentations on how to work towards the positive development of technology in different careers.

 

For providing a talent pipeline into EA

This is quite straightforward: sufficiently involved Envision members will be exposed to EA. We’re collaborating with EA Build, the body coordinating EA Student Chapters, to accomplish this, and will coordinate with individual student chapters at the universities where we open chapters.

However, this exposure will be light, in order to retain those future leaders who are not interested in EA – this is a crucial part of Envision’s value-add.

Planned structure

 

Envision will be a global network of student chapters initially headquartered at Princeton University. Governance may become more decentralized and distributed across chapters with time, depending upon developments.

 

There will be an annual main Envision conference, and several smaller technology- and issue-specific conferences hosted by different chapters.

 

Envision-hosted events will generally be open to the entire student body of the university in question. There will be officers, who run the chapter, and fellows, who are selected by invitation from event attendees and gain several perks such as priority of attendance for events limited in size, and access to more intimate fellow-only discussions with professors and visiting experts.

 

The exact structure of Envision chapters, while open to change and additions, initially includes:

• Understand: guest speaker series and workshops with university and outside speakers and organizations.
• Interface: Trips to nearby companies and labs developing or working on implementing rapidly developing disruptive technologies, and attendance of nearby relevant conferences.
• Envision Entrepreneurship: Smaller local versions of the conference event. Winners will compete at the main Envision conference or at a separate Envision Entrepreneurship event.
• Discuss: Discussions among members, sometimes including faculty. There will be a more selective subset of Fellows who will discuss more frequently and have more intimate discussions with professors and visiting experts.

 

Success metrics and measurement

 

These are incomplete. Many measurements and success criteria are not well defined, several are subject to Goodhart’s Law and are not optimized to measure counter-factual impact, and the details of implementation are missing. Feedback and advice for addressing these issues is welcome.

 

Goal	Metric	Measurement	Success criterion
Attracting future leaders	Success in attraction	Quality of applicants	Qualitative - compare to criterion of future leader
		Quantity of applicants – Chapters	10% of freshman class
		Quantity of applicants - Conference	700 applications
	Quality of attractiveness	Potential applicants not applying	No talented, potentially interested people not applying. A key issue is how to measure this.
	Successful screening process	Fellows alumni turn into future leaders	50% become leaders
		Accurate selection	Every year after acceptance, fellows are re-evaluated whether they still match future leader criterion. 75% yearly success rate.
Convince future leaders that technology will play a pivotal role in humanity’s future.	Successful events	Pre- and post-event survey - Conference	Significant update towards understanding power of technology in 85% of cases
		Post-event surveys for each event – Chapters	70% significant updates to deem an event successful
	Overall successful mindset change	End-of-year survey, more detailed informal check-in for 50% of most involved members by non-central members (to avoid bias)	Significant positive updates, tangible excitement in 80% of cases
Integrating safety into future leaders’ mindset	Theoretical updates	Pre and post conference survey	70% Significant update towards concern for safety
		Post safety event surveys	30% major update, 60% minor update to deem event successful
	Observed changes	Alumni concerned with safety	60% of alumni leaders demonstrate concern
		Engagement with safety issues	End-of-year survey; 50% show increased engagement and concern
Providing the tools to accomplish above	Career changes	End of year survey question on career change	70% update, of which 45% major update (positive)
		Observed changes	Qualitative; how to measure?
	Career advancement	Opportunities through Envision	15% of career-related event attendees find opportunity. Based on post event survey, several months after.
		Partner meetings	2 partnerships a year formed.
	Provide a talent pipeline	Envision alumni working for the organizations in question who otherwise would not have	At least 5 researchers within 10 years who updated through Envision. Based on self-reporting.

 

Key terms to define

 

Positive update

Safety update

Leader

Future leader criteria

Positive career change

Partnerships

Minor versus major update

 

Metrics to keep track of

 

Number of applicants

Alumni paths and tech safety concerns

Pre and post conference views on tech and tech safety

Post event updates chapters

End of year major updates and tech safety views - number asked and answers (non-anonymous)

Pre and post conference career plans

Observed career changes

Opportunities reached through career-related events

Partnerships formed through Envision

 

Failure Modes

Envision faces many challenges, of which I will outline some of the most salient.

 

Inability to identify future leaders

Our predictive model is designed to counteract this, but success is far from guaranteed. Some potential causes of failure:

• Goodhart’s Law.
• Identifying only the visible manifestations, not the underlying traits.
• Finding there are no common identifiable traits.

 

Inability to attract future leaders

Once identified, being unable to attract future leaders. Potential causes:

• Factors that attract leaders vary so widely between domains it is not possible to appeal to all.
• Path dependence - mistakes early on and the attracting of the wrong initial set of students permanently places Envision on a path that cannot lead to success. This also applies to individual chapters; if initially badly set up, it would be close to impossible to gain traction at the university in question.
• It is not possible to provide the factors that would attract future domain leaders within the boundaries of what is possible given Envision’s goals.

 

Failure to instill a safety mindset in future domain leaders

This is quite a broad failure mode which can result from several different causes:

• Resources are not well-provided. This could be due to speakers and firms refusing to address, or inadequately addressing, risks.
• Resources are not accessed. For example, students may simply not show up to safety-specific events and not read resources provided.
• Resources fail in instilling a safety mindset: Even if well-provided and accessed, students may refuse to update. This can be partially mitigated by presenting safety as an integrated component, that has concrete solutions.
• Integration of safety leads to a competitive disadvantage, causing those who do update to have a lower likelihood of achieving domain leadership.

 

These are but the most likely failure modes that we are most actively addressing. There are more, and we are continuously re-evaluating to ensure we remain aware of failure modes, are not in one, and are not missing any.

 

How you can be involved

Envision is in its early stages. Strategy is subject to change if another route is deemed more conducive to fulfilling Envision’s goals. Critical feedback, questions, and advice on any part of the value proposition and strategy are encouraged.

 

If you are a student or are affiliated with a university, further involvement is welcomed. You can apply to the conference and, if you have the time, dedication, and interest, help set up a chapter, in which case you can contact me.

 

If you are not but think Envision is important and are willing to help, further involvement is welcomed. This is especially true if you have organization-building/management experience, in which case you can advise, or model-building skills (quantitative or qualitative), in which case you can help build our predictive model. If the model is successful, it can be widely applied within EA beyond Envision.

 

If you are in charge of, employed by, or affiliated with an organization or company that potentially would be interested in partnering with Envision, further involvement is welcomed. If Envision is successful, this is an opportunity to potentially have a significant impact on the future of humanity at low cost. More directly, we also have recruiting and marketing opportunities if these are relevant.

Partnership can be content-based, as in the case of our partnership with MIRI, who are providing a speaker and possibly hosting a workshop; the provision of funding (we currently have some but are far from fully funded to execute on all of our projects); or a combination.

 

I can be reached through Direct Message or at lrade@princeton.edu.

To summarize, in this post I:

• Introduced Envision, a new EA-aligned organization, and its value proposition.
• Outlined Envision's current status.
• Justified the claims made with regards to its value and effectiveness.
• Outlined Envision's strategy
• Presented our success metrics and measurement mechanisms
• Went through some of the most salient failure modes
• Explained how EAs can potentially be involved