Contributions by the Students for High-Impact Charity Team (Catherine Low, Baxter Bullock, Tee Barnett, David Vatousios and Callum Hinchcliffe), the Run to Better Days Team (Brenton Mayer, Daniel Charles, Laura Koefler), Jessica McCurdy, Daniel, Alex, Jamie Harris and Sebastian Becker. Compiled by Catherine Low. Introductory segments written by Catherine Low.
This post compiles summarized reports on several projects and instances in which effective altruism (EA) concepts have been introduced to school students aged 13 to 18 (referred to hereafter as high school students):
- Guest presenters running workshops in high schools (Run to Better Days and Students for High-Impact Charity).
- Resourcing university student coordinators to help high school students set up EA-aligned groups in their in high schools (Students for High-Impact Charity).
- Recruiting and resourcing high school student leaders to run sessions on EA concepts with their peers (Students for High-Impact Charity).
- Recruiting non-EA teachers run EA sessions in schools (Students for High-Impact Charity).
- University students running EA sessions as part of the Splash program in USA Universities.
- EA-aligned teachers presenting concepts in classes and extracurricular clubs.
The content delivered to students varied from project to project, and included charity comparisons, ethical questions, cause prioritization, high-impact career choices, and discussions of common EA cause areas.
We hope this post will be a useful resource for people who are interested in communicating the basic principles of EA to young people.
This post was prompted by Students for High-Impact Charity (SHIC) recently choosing to suspend outreach to high school students.
We begin by explaining the generally accepted reasons for why reaching out to high school students may be useful, and our tentative conclusions. This is followed by detailed descriptions of some projects that have been tried, written by that project’s team or coordinator. For each project, we explain the method used and any measured impact.
Why choose high school outreach?
Many self-identified effective altruists state that they wished they had gained EA knowledge much earlier than they did, so that they could have had a roadmap to effectively improve the world from a younger age. These statements suggest it was worth testing whether high school students are a good group to educate about EA.
Our initial reasons for believing that high school students could be a good audience were:
- They are less likely to have fixed opinions about the best way to do good than people who have been doing altruistic actions for some years.
- They may be more open to new ideas than older people.
- They are in a significantly better position to make impactful life decisions than university students or adults, as they haven’t sunk time and resources into a potentially lower impact path.
- It’s possible to gain access to an audience of high school students more easily than audiences of older people.
- Anecdotally, EA ideas seem to be more appealing if they’re presented by someone more senior than they are, which most EAs are relative to high school students.
There are several ways that reaching high school students could have an impact:
- Guiding high school students towards higher impact career paths, volunteering and donations.
- Influencing school fundraisers.
- Providing a positive first experience of EA concepts, increasing the chances that these students would take action after subsequent exposures to EA—for example, when they are at university.
This section was authored by Catherine Low (Manager of SHIC since 2018, closely involved with SHIC since early 2016, and a former teacher).
The contributors to this post have not tested all of the possible methods of engaging high school students, and other, untried methods might have more impact. However, I have drawn some tentative conclusions from the attempts that have been made.
- Introductory EA concepts appeared to be fairly accessible to many students.
By this, I mean ideas about cause prioritisation, cost-effectiveness, and making decisions under uncertainty appeared to be comprehensible by many students, as evidenced by students speaking intelligently about these ideas during in class discussions. However, in most cases, students only had a small number of hours thinking about these ideas, so they were unlikely to remember much or apply these concepts to new situations.
- There was very little controversy generated from teaching EA concepts.
Generally students had favourable opinions towards ideas that were presented, and there were rarely comments in classes, or in student or teacher surveys, that suggested that the material was too controversial for a high school audience. This was somewhat surprising because I believed some of the ideas would be confrontational, such as viewing factory farming videos, or discussing whether we are morally obliged to help those in need, even if they live in other countries and we can't see their suffering. This updated us towards thinking that EA outreach has fewer immediate downsides, although it is hard to know whether this finding can be extrapolated to older age groups.
- Schools are often receptive to having presenters talk about EA concepts.
Giving guest presentations in schools is generally an effective way of reaching a large number of people. EA concepts fit well into high school curricula, and schools in Vancouver, New Zealand and Australia were often enthusiastic to have guest presenters talk about EA concepts. Schools in London appeared to be less responsive to outreach, and while we have some possible explanations, we can’t be certain as to why this was the case.
- So far, only a very small percentage of students reached have taken significant steps towards high-impact actions as a result of outreach.
Outreach to high school students has resulted in some success stories, but in general, has not been as impactful as was initially guessed, as only a very small percentage of students reached are known to have taken action. It was harder to engage with students long-term than expected. In surveys, most students stated they were positively inclined towards the ideas and many students reported that they planned to donate more and/or do more research before donating in the future. However, very few students invested time to learn more or took high-impact actions in the months after being exposed to EA ideas.
- Engaging students over the long-term is difficult when there is no EA-aligned staff member at the school.
SHIC attempted to establish longer term engagement with interested students they reached through the student leader model and in instructor-led workshops in schools, however only a few students took up these opportunities.
- Having some long-term connection with students through an EA-aligned staff member, or a volunteering opportunity seems to increase the chance of students taking action.
Out of the options that have been tried, the best strategies for engaging high school students seem to require sustained interaction with students, for example by finding teachers interested in EA, or giving students volunteer roles. Since engaging and empowering teachers hasn’t been thoroughly tested, I don’t know how easy it would be to scale the successes of the EA teachers who have shared their stories below. EA concepts do appear to fit into existing social studies type curricula in many areas, and 80,000 Hours-type advice could fit into careers courses, so it may be possible for EA resources to be spread through teacher networks. While it could be very influential to alter the national, state, or provincial curriculum, this is likely to be difficult to do.
- High School outreach can be an effective way of improving presentation skills. Given the willingness of many schools to have EAs present, high school outreach has been useful for training purposes before presenting to older audiences, as the SHIC team and several other contributors to this post have found they’ve honed their understanding of EA and presentation skills through this outreach.
I don’t think our outreach described in this post was a particularly effective use of resources. However, outreach could be effective if you are able to attract highly promising students to sign up for a program over a longer term. This might be possible if you have a strong brand (such as an association with elite University) allowing you to attract suitable students through schools and other networks, and the resources to run a fellowship-type program with these students. Alternatively, with the right connections it might be possible for EAs to take a significant role in similar programs already existing for highly promising students.
While only a few students appeared to have taken significant action as a result of the outreach described in this post, our outreach may have had more subtle, positive effects that weren’t measured.
One way we may have made impact could have been through the management of first impressions, as Jessica mentions in the section on Splash. Some people seem to have had negative or incorrect first experiences of EA, potentially causing them to be put off engaging in the future. However, if they have good first impressions, people are likely to be inoculated against poor messages in the future. If this is a common problem, then well executed high school outreach by EAs could be a good way of preventing harm from negative messages before students are likely to get low fidelity exposure to EA from other sources. It might be possible to gather more information about whether this is a cost-effective action through conducting surveys or experiments.
Even if people are unlikely to gain negative impressions of EA, outreach to high school students could still have a subtle positive impact. Many people require multiple exposures to ideas before acting, it may be that the main impact of our outreach was to increase the chances of the students getting involved when they next come across EA. However, if this is our main route to impact, it is unlikely to be the most cost-effective form of outreach at the moment. Since EA is not well known in wider society, so many of our students may not get any subsequent exposures to EA ideas. When and if EA becomes more widely discussed, the role of high school outreach might become more useful.
If you are interested in talking to high school students about EA, please feel free to contact me at firstname.lastname@example.org.
This section was authored by Brenton Mayer (80,000 Hours, and a former coordinator of Run To Better Days). Edited 14th May 2019 to add estimates of the overall results of the outreach activities mentioned in this post.
I’ve been very surprised at how little measured success high school EA outreach efforts have yielded. This post has compiled evidence from many competent people trying out multiple different methods, which in total have had over 5 years of full time equivalent work go into them. This has resulted in:
- Three students becoming counterfactually interested in EA enough that they became involved in university groups or made a career change. I would guess that this work (mostly Catherine’s) accounts for the majority (>75%) of the measured success.
- 10-20 students becoming counterfactually interested in EA enough to reduce their meat consumption or start fundraisers for ACE or GiveWell recommended charities.
- <$20,000 USD raised for ACE or GiveWell recommended charities.
My impression is that this is pretty poor compared to other outreach methods, such as running a university group.
I don't think the evidence is strong enough to suggest that no-one should work on high school EA outreach, but I do think that someone considering this should be pretty concerned by this data. I’d suggest that they design their programs with the information in this post in mind (and ideally talk to some of its authors), should make sure that they're well positioned to collect feedback on their programs as early as possible, and they should be prepared to pivot away from those programs (or high school outreach in general) if they aren't getting results.
- I agree with Catherine’s points. In particular, it seems like #2-5 are true to a surprising degree.
- I’m not sure what SPARC’s data has shown about how promising their approach is, but I’d be interested to know. If they’ve had a lot of success, I’d likely update towards very targeted high school outreach being good, but not general high school outreach.
This section was authored by Baxter Bullock (Executive Director of Rethink Charity, Co-founder of Students for High-Impact Charity, and former high school teacher).
I mostly agree with the conclusions made in the sections above, and non-SHIC accounts of high school outreach included in this post have not drastically altered my takeaways. However I am more optimistic than others about the impact of high school outreach, pending clearance of several obstacles related to data collection and long-term engagement. In one sentence, my conclusion is that high school outreach likely has potential that we haven't yet figured out how to unlock.
My opinion is that the SHIC program was suspended because we couldn't see how much impact we were having, not because we thought we weren't making a positive impact. In early 2018, when SHIC made its biggest and final shift in strategy to the instructor-led model, one of our main reasons for doing so was to maximize our chances of being able to quantitatively and qualitatively track our impact firsthand. In many ways we were successful in doing so. We saw firsthand how much students and teachers were engaged by our material. We experienced a strong demand for a program like SHIC in schools.
However, when we set the arguably unclearable bar of needing to prove our impact through extended engagement, we were unknowingly setting ourselves up for failure. As a transparent and data-driven organization, we were seeking to find a metric measurable in the short-term by which we could justify SHIC’s existence, but in my opinion it’s quite possible that no such metric exists, and that longitudinal observation (over a time scale we don’t have the resources to employ) is the only reliable indicator of long-term impact.
High school outreach is bottlenecked by this imprecision, which is an inability to accurately adjust strategy based on observed results. Essentially SHIC and similar programs that rely on short-term evidence to justify their existence are probably doomed to fail regardless of actual impact, because the true and most significant effects can only be seen longitudinally through participant’s future actions and life decisions.
It could be a massively cost-effective way to spread ideas that could shape the way a generation prepare their lives to do good. It could also have very little impact. Its position on the scale of impact lies in the intricacies of unpredictable moments in the future in which a now-student must mark a form to indicate their preferred area of study in university, or decide what to do with their inheritance. At these moments, will workshop participants utilize their knowledge of high-impact charity?
This indeed leads us to a broader question about the immeasurable effects of outreach and education in general, and whether they’re worth the investment of time and money. Regardless, the breakthrough in high school outreach belongs to the individual or organization that is able to reliably maintain a long-term connection with students subsequent to their exposure to these ideas. I don’t believe this to be a futile effort, and I look forward to future efforts to do so.
The following sections give detailed descriptions of the different methods used to engage high school students in EA, and the results of this outreach.
The Run to Better Days
This section was authored by Daniel Charles (medical doctor), Laura Koefler (medical doctor) and Brenton Mayer (80,000 Hours).
The Run to Better Days is an annual event which aims to generate action against world poverty, chiefly through encouraging donations to GiveWell top charities. Since 2012, it has delivered presentations to 35,000 people (mostly high school students) and tracked $80,000 USD in donations through associated fundraising campaigns.
The run is a ~1000km annual relay completed by about 15-20 volunteers, mostly medical students from James Cook University, typically along the north eastern coast of Australia. During the run we fundraise through personal networks, community groups and local news media. We stop in at schools and community organisations to deliver presentations inspired by Peter Singer’s ‘The Life You Can Save’. You can see one of the presentations here.
We’ve delivered our long form ~40 minute speech to around 20 000 people, and have given a more condensed version to 15 000. About 70% of that audience were high school kids aged 12-18. Other audiences included:
- University classes for health sciences, the Masters of Public Health and medical school.
- Community organisations including Rotary, Rotoract and Lions clubs.
- Conferences for doctors and medical students.
- Primary schools.
We gave away around 2500 copies of The Life You Can Save to people who seemed enthusiastic after hearing the speech. Around 70% went to high school students.
Efforts to track our impact
We have mostly not followed up with people who we gave talks to. Below we describe factors which made follow up more difficult to execute on than might initially be assumed, but even given these constraints, we think we should have done better.
Most of the follow up was incidental, through talking to people we already knew who happened to be hearing our talks.
Our major effort at systematising follow up was in 2015, when we asked for the emails/Facebook details of 5-10 interested students at 10 different high schools. We then sent emails to these students/made and posted to Facebook groups we made and added students to for the following few months. We received a few positive messages, facilitated one promising discussion about a gapminder video and had some likes on our posts. We discontinued this due to a combination of low engagement from the students and lack of organisation on our part.
Interest from our target audience
We're aware of two people who have been significantly impacted by a RTBD presentation as well as a few smaller successes. All of these people were university students at the time.
- One medical student we met at a global health conference signed the GWWC pledge and became enthusiastic about EA.
- We met Peter McIntyre (now at 80,000 Hours) through giving a talk at a medical student global health conference. He had already read The Life You Can Save and was giving around 30% of his income to AMF. Becoming friends with us caused him to get much more into EA and take the GWWC pledge. He thinks it’s unlikely that he would have made a major EA inspired career shift if not for the run. Peter then caused Brenton to leave earning to give in medicine something like two years earlier than he would have otherwise (on expectation).
- In addition, there are several examples of smaller changes such as people sponsoring a child through World Vision and someone pledging with The Life You Can Save.
Interest from the RTBD volunteers
Seven people involved in the run have signed the Giving What We Can pledge. Three of them co-founded the run and are the authors of this post. Two of them are long-term partners of the authors of the post.
Since the co-founders graduated, other people have stepped up to run it, which (given the size of that undertaking) demonstrates substantial interest in effective giving.
Apart from Brenton (who stopped practicing medicine in order to work at 80,000 Hours), no-one has developed much interest in cause areas other than global poverty or made substantial career shifts.
We ask people to give to our partner charities (mostly the Against Malaria Foundation) to support the run, which we track via pages like this. We’ve recorded $80 000 USD through these, which has mostly come via the networks of the runners. The remainder came through assorted sources including the runners themselves, some schools proactively fundraising for us, through our sponsors and donations from the community groups.
Speculation on impact we’ve had but haven’t tracked
We’ve had a huge number of positive conversations with people (mostly not high school students) who liked our message and told us they were enthusiastic about effective giving. For some indication of this, consider that we were repeatedly asked to give talks at platforms which seem pretty generous to offer to a bunch of university students without relevant qualifications. One example was being asked to speak at a conference for rurally practicing doctors. Another time we gave our talk at a 600 person global health conference and facilitated a giving game which ran throughout. Several professors kindly found a way to shoe-horn our presentation into their curricula - mostly they invited us in as part of a global health module, but there was one ethicist who asked us to come in as an example of utilitarian reasoning.
Based on this, we would guess there are tens of thousands of dollars which have been donated to GiveWell top charities due to the run which we haven’t tracked.
It’s possible that there are people heavily involved in EA who initially heard about it through the run, but we think that’s less likely. We have checked the registry of GWWC members for the names of around 30 people we’ve had contact with who seemed pretty interested in the ideas, but didn’t find anyone who has taken the pledge who we weren’t already aware of.
The message of the run has reached a lot of people, yet we have remarkably little evidence of ongoing substantial involvement from our audience. We think two major contributors to this are our ability to follow up with people we speak to and their age.
If we were designing an outreach project from the ground up, we would try to build follow up mechanisms into the structure of the project . Not having good follow up mechanisms has meant that we haven’t been able to reinforce messages with our audience, get feedback on what messages are and aren’t landing with our audience, and track our impact. The structure of the run makes this follow up difficult, because:
- We usually live hundreds of kilometres away from our audience.
- Their age puts restrictions on how appropriate it is to have personal contact with them after we leave their school.
- The run is a major logistical challenge which badly strains the capacity of senior organisers in its lead up, making it difficult to get ‘important but not urgent’ things done such as follow up planning.
- The annual nature of the event means that the most rapid iteration cycle available is once per year.
Our impression is that speaking to older students is more promising than speaking to younger students, and that this doesn’t start diminishing until they reach around 19 years of age. This was informed by what discussing the talks with them afterwards indicated about how deeply they’d understood the concepts we’d presented. This impression is bolstered by how large the effect of the run was on the university aged participants (as distinct from its audience) and how the two GWWC pledges associated with the run which which came from our audience were from university students.
We’re proud of how the Run to Better Days impacted its volunteers and how long it has kept going without much involvement from the original organisers. There are many ways we would change it if we were designing another outreach activity from the ground up, though it’s less obvious that these changes should be made to the existing run.
You’re welcome to get in touch with us at email@example.com. If you’re thinking of starting something similar to the run then we’d be happy to help you out as much as we can!
SHIC coordinator model
This section was authored by the SHIC team.
The Students for High-Impact Charity (SHIC) published program consists of a number of activities, videos, readings and discussions on ethics, EA principles, charity analysis (including a Giving Game with money from The Life You Can Save going to the charity the students chose), global poverty, animal welfare and rights, climate change, existential risks, cause-prioritization, and 80,000 Hours-inspired career advice.
When SHIC was initially conceived in 2015, we believed that the most effective outreach strategy would be to have university students who are involved in effective altruism (referred to as ‘coordinators’) act as intermediaries between the central administrative team and student groups in high schools However, after testing this strategy over approximately 6 months (while also working on the student leader model), this method did not result in the formation of any student groups.
The aim was to have coordinators work with high school students to run the SHIC program in their groups. To build these groups, we tasked the coordinators with approaching high schools, identifying student leaders, and helping these student leaders run the SHIC program within their respective schools.The program at the time consisted of 6 levels of approximately 1 hour each, with advanced levels available for students to move on to if they wished to continue.
Over the latter half of 2016 we recruited 11 coordinators from universities in six countries. 4 out of the 11 used the SHIC resources in groups in their own universities, only one ended up using the SHIC resources with high school students, and that did not lead to the forming of a SHIC group in the school.
We believe that our original strategy placed too much responsibility on volunteers whose time and understanding of the SHIC program were limited, and we had not yet developed materials or a framework to sufficiently aid coordinators in this endeavor. It should be noted and emphasized that we attribute this lack of activity to an unstructured strategy, not to the coordinators themselves.
More information about the coordinator model is available in SHIC’s Interim Report published in December 2016.
The Coordinator Model was worked on during 2016, with one full-time staff member working on the Student Leader and Coordinator models.
In addition to the paid staff member, a large number of volunteer hours were spent on the curriculum and the website.
SHIC Student Leader Model
This section was authored by the SHIC team.
The aim of the Student Leader Model (formerly known as the “Ambassador” model) was to encourage students to form extracurricular clubs (or use existing school clubs) to work through the full SHIC program. The student leader would work with our team and become familiar enough with the material to run the SHIC program with their peers. This model was used from 2016 until recently, although from 2018 it was deprioritised, and run almost entirely by capable SHIC volunteers. As a result, a small number of students have become highly engaged in EA and have, or are likely to, take significant actions.
When SHIC started, the Student Leader Model seemed like a promising strategy for engaging students as it had a high potential to be scaled up to hundreds of schools at low cost. It utilised the existing club structure in high schools, and would ideally lead to students gaining a peer group with similar interests and goals. The success of other large school group networks like Model United Nations, Me to We, and Key Club, suggested the possibility that SHIC could form the basis of a network of school clubs.
We recruited both university and high school students for this role, however this report just outlines the results from the high school students. We didn’t recruit enough university students to comment on whether there was a significant difference between high school and university students.
We chose not to explicitly use the term “effective altruism”, although the program was designed to cover most introductory EA concepts. This choice was made partly because we didn’t want teachers and parents to be concerned that SHIC was aiming to recruit young people into a movement, and partly because of reputational risks: if EA became controversial we didn’t want this to reflect poorly on SHIC, and vice versa. This could have had significant downsides: students were less able to discover related content on their own, and did not realise that SHIC’s philosophies were part of a widespread movement they might have been excited to join.
Some of the student leaders were aware of EA, and found us through networks, but most student leaders were recruited through online volunteer listing websites. Interested students were then offered an initial Skype with a SHIC staff member or volunteer, who would have a conversation about the basic ideas of high-impact charity, and what the student leader role would entail. Students willing to start this role were offered continuing support over email and Skype.
98 prospective student leaders had an initial Skype with our mentors. It is unclear how many of these students started or completed the SHIC program, however pre- and post-program surveys completed by participants, along with our 2016 Interim survey completed by student leaders, provided some indication. 82 students across 23 schools completed the pre-program survey. 8 of these schools had clearly formed a group as multiple students from the same school answered at the same time. The average group size was 8 people. The remaining 15 schools just had one person complete the pre-program survey, which could have been as a result of the student leader working through the materials on their own, or could have been because the student leader chose to complete the survey on behalf of their group.
The post-program survey was only completed by 21 students across 7 schools. 3 of these schools had groups of 5 people on average. The remaining 4 schools just had one student complete the survey.
The survey numbers are likely to be a significant underestimation of the number of students starting and completing the program, because some student leaders may not have asked their group to do the survey at all, or just submitted one survey for the whole group, so these results represent the absolute minimum numbers. To estimate how well the survey entries match real student numbers, at the end of 2016 we emailed additional surveys out to active student leaders, who reported their own group numbers. These numbers were on average 4 times higher than indicated by the survey, which, if extrapolated across all groups, would lead to an approximate 328 students starting and 84 finishing the program.
Student leaders may be exaggerating the number of students in their group, and there were many students who expressed an intention to start a group who may have run the program without completing any surveys. It is therefore very uncertain how many students SHIC reached through the student leader model.
There are many reasons for the high attrition rate from the initial Skype to doing the first session, and to completing the course. We believe that it was a very big ask for students to organise a club, let alone run the activities, especially when most of them were very new to the EA concepts themselves. One student leader reported that they felt less comfortable with the animal and existential risk topics and so did not complete those levels, so comfort delivering the materials may have been a significant factor. While the mentors were able to provide Skype and email support, there was no local support, and very little in the way of external incentives for students to run the SHIC course.
During 2016 we suggested that student groups could set up fundraisers for their favorite charities, and gave them some guidelines as to how to do this effectively. When we started SHIC, we hoped that these fundraisers would result in more money going to effective charities than the total running costs of SHIC. However, we found that students are less eager or capable of raising funds than we expected, raising only $928 USD between five fundraisers in a single semester. Although it remains an option for student groups, after 2016 we reduced the emphasis on fundraising in favour of emphasising longer term actions they can take, and have not asked students for fundraising data.
Several of the students who took the post-program survey reported that they are likely to make changes to their donations, careers, and way of living due to the SHIC program. However, these statements could be an overestimation of the true impact, as students may give answers biased towards what they think their student leaders (and SHIC) want, and the surveys were taken immediately at the end of the SHIC program, when students’ enthusiasm was likely to be at its peak.
To gauge the long term impact of learning about EA at high school we recently requested information from some students who got involved between 2016 and 2018 and have remained in touch with us since. Their reports indicate some long term impact from SHIC, but apart from one student it is unclear how counterfactual this impact was.
Student A discovered EA through becoming a SHIC student leader, but probably would have discovered EA at some point. Now at university, they help to run their local group and organise their local EAGx. They have volunteered or interned for a couple of different organisations associated with EA, hope to do direct work for EA cause areas in the future.
Student B was already interested in EA when they discovered SHIC, becoming a student leader and a volunteer. They have since given presentations about EA to other groups, are part of their University EA group, and have volunteered for multiple EA organisations.
Student C was already interested in EA when they discovered SHIC. They have since read widely on EA, and are planning on choosing courses at University based on EA principles. Philosophy will be part of their degree, but they haven’t decided on other courses yet.
Student D would have been unlikely to discover EA without SHIC, and has been running a SHIC group for the last year. SHIC has made a large impact on their career choice— they hope to make an impact in global health and are considering medical anthropology as a degree.
Why the student leader model was deprioritized
The student leader model was deprioritized in late 2017 to be mainly volunteer run, and advertising for student leaders stopped in February 2019. The SHIC resources are still available on the internet, and one-on-one assistance is still available for students who discover the SHIC website and wish to have support using the resources.
This decision was mainly due to the very small number of students working through the program. We were also becoming more careful about ensuring the concepts were explained well. Despite the thorough lesson plans provided, the student leaders may not be able to convey the concepts in the program accurately, leading to a low-fidelity transmission of EA ideas.
We don’t think our experience proves that EA concepts can’t be part of a large school group network such as Model United Nations, as our approach was just one possible strategy. We may have also been hindered by trying to run the project on a very low budget, as it may have been necessary to have staff on the ground in key cities to get a school group network running.
The Student Leader Model was worked on from Early 2016 to March 2019. Over 2016 there was one full-time staff member working on the Student Leader and Coordinator models. Over 2017 there was the equivalent of one full-time staff working on this, across two people. In 2018 and the first three months of 2019 this method was deprioritised, and took up approximately 10% of a full-time staff member’s role.
In addition to the paid staff, a large number of volunteer hours were spent on the curriculum, mentoring student leaders, reaching out to schools and students, and creating the website.
Providing SHIC resources and support to non-EA teachers
This section was authored by the SHIC team.
A couple of teachers who were not previously familiar with effective altruism stumbled upon the SHIC materials and requested more information about how they could be utilized in their classroom or club. To the best of our knowledge, these teachers presented only the first one or two SHIC lessons.
To see whether teachers would be interested in utilising the program, Catherine presented at the New Zealand Philosophy Teachers Conference in 2016 and the NZ Social Studies Teachers Conference in 2017. The presentation featured interactive activities from the SHIC course, and elicited a lot of positive feedback and discussion. However there didn’t seem to be particularly high uptake. Only one out of the approximately 90 teachers in attendance contacted us reporting that they used part of the SHIC program in their class, and no surveys were completed by students, although others may have tried it.
Despite these unremarkable attempts at teacher outreach, we believe there may still be potential. We used only a small amount of volunteer time and effort encouraging teachers to use the program. There are several potentially effective strategies for teacher engagement that could still be explored, such as working with teacher associations or influential teachers within promising subject areas such as Philosophy and International Baccalaureate’s Theory of Knowledge.
With non-EA teacher outreach, we ran the risk of important concepts being miscommunicated, however the opportunity for strong scalability may outweigh this risk. We believe available SHIC resources are detailed enough that a non-EA can effectively run our program, but we have no control over how they choose to use the materials. While we are no longer working on promoting SHIC to teachers, resources and support are still available for teachers who discover the SHIC website and wish to run the program in their classroom.
SHIC instructor-led workshops
This section was authored by the SHIC team .
From January 2018 to April 2019, we experimented with presenting workshops put on by SHIC Instructors, paid Rethink Charity employees who have been extensively trained with the material to ensure the workshops are accurate and of high quality. Workshops consisted of SHIC instructors running variations of the short SHIC program, as either four hours of programming across three school visits, or a single 1.5 hour visit, depending on the school’s availability. Most of these workshops were run in Vancouver, with some in New Zealand schools, and some in London. Locations of workshops were determined both strategically and based on the homes of our qualified instructors. These workshops were inspired, in part, from Catherine’s experiences delivering the SHIC program as a high school teacher in 2016 and 2017. Her experiences provided evidence that an instructor-based model may be more effective than student-led models. By having trained employees we were able to reach a large number of students, and test whether a the SHIC curriculum, implemented to a high standard, would be effective at engaging students.
Workshops were mostly presented to students aged 16 to 18, and were primarily run in Math classes, or classes within the Social Studies departments (Social Studies, Social Justice, Philosophy, Human Geography and Economics). By the end of 2018 we had presented 106 workshops, reaching 2,580 participants at 40 institutions.
In Vancouver we saw more interest than we initially expected, and to our surprise the majority of bookings resulted from cold emails rather than networking. 25% of the schools we contacted in Vancouver eventually booked at least one workshop, indicating that there is an appetite for presentations on EA concepts in high schools. Our outreach appeared to be less effective in London schools, although we only spent a few weeks contacting these schools, in comparison to Vancouver where we contacted each school multiple times over the course of 9 months.
While we reached many students, very few continued to extensively engage with us beyond our initial visits.
We experienced strong in-workshop engagement with nearly all groups we worked with. Students in classes run by the Social Studies departments classes generally seemed more interested than those in Math classes, especially if the subject was an elective class that encouraged consideration of morality in some way (e.g. Social Justice or Philosophy). Perhaps better still, for both interest and comprehension, were International Baccalaureate Theory of Knowledge classes, likely because the students are more intellectually inclined and have had explicit critical thinking training. The highest level of interest was with philanthropy-related student clubs (such as Me to We and Interact Clubs), but we had far less time with them, and we weren’t invited to work with these groups as often as with timetabled classes.
We noticed little variance in terms of interest and understanding across different age groups. Though concepts may have generally taken more explanation for younger grades, the majority of workshop material seemed accessible to all participants. In fact, some of the best groups we worked with were grade 9 classes at a high-achieving private school.
In the first half of 2018 we administered pre- and post-workshop surveys to participants. Results indicated positive shifts in values and beliefs. After the workshops, students were:
- Much more likely to choose a high-impact charity when asked to choose a charity to donate to.
- More likely to have “EA” reasons for choosing a charity (cost-effectiveness, and choosing the most critical cause area).
- Less likely to believe that farmed animals are treated well.
These survey results were encouraging, but it is likely these stated changes in perspective were affected by social desirability. Additionally, we are uncertain whether any genuine changes in perspectives or beliefs would result in behavioural change.
More information about the pre- and post-workshop survey results can be found here: https://forum.effectivealtruism.org/posts/EKBt7uvYug7haDAfu/students-for-high-impact-charity-2018-update
Teachers of relevant Social Studies classes (including Social Justice, Philosophy, Economics, Theory of Knowledge and Human Geography) also reported how the SHIC program fit well into their curriculum, further indicating the viability of high school classes as a way of spreading ideas.
Teachers were very positive about the program, with 19 out of 20 teachers reporting in the teacher survey that they were satisfied or very satisfied with the program. Many teachers gave glowing testimonials and some invited us back for repeat visits. Since we were unsure whether teachers would think the content was too upsetting or controversial, particularly the factory farming footage or discussions of existential risk, we also asked whether there were parts of the program that were likely to be distressing to students. Only 1 out of the 20 teachers reported that parts of the program were unnecessarily distressing, the others reporting that it wasn’t distressing at all or was potentially distressing but that this was necessary to learn about the topics.
At one school, two teachers (who didn’t complete the survey) stated during our visit that they thought the existential risk section was too alarmist for the age group (approximately 14 years old). However, none of the students appeared to be distressed and one even commented to the instructor afterwards that the session had been very interesting. Intriguingly, we’ve found that a large proportion of students believe human extinction to be much more likely than existential risk experts do, so we think it more likely that students found our discussion of this topic reassuring, rather than disturbing.
We never received any complaints from students or parents. Overall our positive teacher feedback further indicates that EA outreach has fewer risks than some initially thought.
To ascertain whether the SHIC program had more than a temporary impact on students, all 662 students who we had an email address for received a follow up survey by email approximately 3 months after their workshop was completed. All non-responders were reminded 1 week and 2 weeks after the initial email. 58 out of the 662 students (8.8%) started the survey. This small sample is likely to be skewed so it is unclear how much weight to put on the results of this survey.
One question asked the students to list the charity they’d most like to donate to today. Only 17 students gave a name of a charity, and 6 (35%) were deemed by SHIC to be “effective charities” (3 Against Malaria Foundation, 1 GiveDirectly, 1 GiveWell, 1 Humane League). This is a lot higher than the 0.3% effective charities in the pre-workshop survey, suggesting that at least some students have retained knowledge of the charities in the program.
Students also were much more likely to select “EA” reasons for choosing a charity, such as “The charity is tackling the world's most critical problem” and “The charity is particularly cost-effective (can do good with a small amount of money)” compared to the pre-workshop survey. Given the selection bias, this positive result is perhaps unsurprising. Also, given that many of the students that chose both “EA” options also chose a charity we would not deem to be effective, it is not clear that this positive result indicates this change will translate to real impact if and when students are to donate.
Stronger evidence of impact came from asking the students to list actions they had made as a result of the SHIC workshop. Most students who answered this stated a change in perspective rather than a concrete action. However, four statements indicated positive actions:
“We did a yard sale recently and gave the money to against malaria foundation [sic]”
“I’ve actually started to fundraiser [sic] for the Humane League. I’m running several school events throughout the year to support this charity, as well as inform people about it’s [sic] significance. My hope is that people begin to realize how impactful it is to donate just $1 to high impact charities such as the Humane League”
“Yes absolutely. I think the significant impact was that I have become a vegeterian [sic].”
“I've thought more about how much meat I consume and have tried to bring that amount down.”
When we began implementing the instructor-led workshop model, we determined that the key measures of SHIC’s success would be the number of students who continue to engage with the material beyond our initial visit, and the extent to which they remain involved. In the second half of 2018 we began mentioning advanced workshops during our initial visits as opportunities to remain involved, learn more, and become part of a community. 18% of students expressed their interest when asked on the post-workshop survey.
1247 students were eligible to be informed about our advanced workshops, and we sent out 392 invitations. 17 students expressed interest, five ended up attending our first advanced workshop in November, and two attended the December workshop (both returning from the first).
There may have been logistical reasons for why the December workshop was less attended by students (weather, time of day, proximity to the holidays). However the very low response rate to both the November and December advanced workshops was poor enough to suggest that this was not an effective method for engaging students beyond the initial workshop, and could indicate a low appetite for additional programming of any kind.
Based on these results, we’re left with one or more of the following three conclusions:
- The students were engaged in the SHIC program in class, but our methods for engaging students beyond the classroom were ineffective.
- Students had the will, but not the bandwidth to engage further with the SHIC program.
- Students were not as engaged by the SHIC program as our post-program survey data and experience suggested, and therefore uninclined to participate in further programming.
Our best guess is that all three of these conclusions are true to some extent.
As a result of our struggles to further engage students, we decided to suspend SHIC instructor-led workshops in high schools. There is a possibility that these workshops performed a very useful role by being the first exposure to high-impact charity and effective altruism ideas, potentially making them more receptive next time they encounter the ideas, however it is very difficult to provide tangible evidence of such an impact.
More information on the advanced workshops and why we chose to stop running Instructor-led workshops can be found in this post.
David, the Vancouver SHIC instructor, teaching at Bodwell High School.
Students at Bodwell High School during one of the activities.
The Instructor-led model required more resources than the previous methods of engaging students, as we had paid staff delivering the workshops. Over 2018 we had one full-time manager/trainer who also delivered workshops, one part-time administrator, and one full-time instructor. From January to April 2019 we had an additional full-time instructor.
Splash - Courses for high school students run by Uni students
This section was authored by Jessica McCurdy, who runs Yale EA’s contributions to the Splash and Sprout programs.
Splash is an event for high school students where they come to one of many Universities across the USA, and get to elect to take classes taught by university students. Splash students attend one 50 minute class on each topic. Yale, MIT, and UCSB EAs have presented at their local Splash events. Yale also runs a similar program, Sprout, where students attend three 50 minute classes to the same students over three weeks (although students often skip a week or two).
Yale EA’s aims for Splash and Sprout is to introduce high school students to the concepts of EA. We started in the Fall of 2017, and have taught 7 classes so far, reaching a total of approximately 70 students in total. We recognize that a 50 minute session is not nearly enough time to change students behavior. We mostly want to “prime” the students with the name Effective Altruism, the main concepts, and GiveWell.
We hope that if they hear about it again they will have a positive attitude towards it, as it is more common than we would like for students’ first interaction with EA to be in a negative or critical light. Often times incoming university students have been mislead by misrepresentations of what EA is (i.e: only earning to give, just for vegans, etc.)
We originally had high hopes for the students but one class is simply not enough time to significantly affect behavior. Additionally, the way Splash and Sprout is set up here there is not a way to do follow-up on students (although it may be possible that we could with assistance from Splash/Sprout leaders).
We would survey students at the end of class and these have been very positive with students most responding well to the ideas “give more and give more effectively”. This is likely because we spent more time on global health/poverty and/or that giving is an easier concept to grasp. Students were usually engaged and excited about EA after the class with a majority saying that they would like to continue pursuing EA but I find it unlikely that it significantly stuck with any of them.
While these classes may not be very high impact on the student side, I do see potential value on the teacher side. University students on the newer side of EA learn the concepts more thoroughly while preparing to teach a class. They will also get a chance to practice talking about EA and responding to questions. Additionally, it acts as a good way for someone new to get involved since it is easy and a low time commitment.
I think this was a good use of my time when I had first joined Yale Effective Altruism. Now as I am taking on other roles and have probably gained much of the learning and practicing value through my previous classes, I no longer teach myself but encourage and assist new members.
I wrote a handover document for new teachers. It goes over preparation, tips, and lessons learned. Some of the information is Yale specific but hopefully could still be useful!
Feel free to reach out to me - firstname.lastname@example.org.
This section describes the experience and results of four EA-aligned teachers who have introduced EA concepts to students at the school they teach at. At least 8 EAs have utilised the SHIC program with their students, resulting in at least 530 students learning about EA concepts in school, and several other teachers have created their own resources to use in their school. As a result, a handful of students have or are likely to take significant high-impact actions.
This section is authored by Catherine Low, who used to teach at a private girls’ school in New Zealand.
In 2016 and 2017, I taught three 16-hour courses in Effective Altruism to a total of 41 senior students in my school aged from 16 to 18. I used this to develop and test the full SHIC program, which included activities, videos, readings and discussions on ethics, EA principles, charity analysis, global poverty, animal welfare and rights, climate change, existential risks, and 80,000 Hours-inspired career advice. In addition, I taught a shortened version of the SHIC program to another 40 students aged approximately 14 years old, during a social studies class in the same school.
This had multiple positive results in the school:
- Around $10,000 NZD was donated to “EA” charities due to the students influencing existing school fundraisers.
- A successful campaign for a policy of Meatless Mondays in the cafeteria.
- A successful campaign to make the school carbon neutral by calculating the school's emissions and offsetting through Cool Earth (which at the time was the most EA-recommended climate change charity).
- In both years, the students also ran a giving game with the the rest of their year to teach them about effective altruism, as a result most senior students (approximately 200 students) in the school learned about GiveWell and the basic ideas of EA.
All these activities were student-led, but had significant support from me. Apart from the food and nutrition teacher, who had concerns about reducing students meat intake, these activities were well supported by the staff.
The survey results and interactions in class indicated that the course was well received by the students. Initially I was a little concerned about student or parent feedback being negative due to the controversial nature of some of the topics, however this turned out not to be a problem. Only one student commented in the survey that the factory farming video was too guilt-inducing, however this student signed up for a second term of the course, so it was unlikely to be a big concern for her. A couple of parents gave me positive feedback about discussions they have had with their daughters as a result of the course.
While several students said in their end-of-course surveys that the course was making them reconsider their behaviour and long term plans, it is unclear how much weight to put on student answers. However, two students maintain a visible interest in effective altruism and keep in touch with me. One of them became vegan as a result of the course, volunteered for SHIC throughout the year following her graduation, donates to GiveWell recommended charities, and has changed her area of study from medical school to microbiology, with the intention of working on global public health risks. The second student began volunteering for EA New Zealand a year after graduation, and plans to take the Giving What We Can Pledge once employed. Both of these students would probably not have gotten involved with EA without this introduction at school.
The most engaged students were those who were in their final year of school. While the content was accessible to younger students, they seemed to treat the course as an intellectual exercise rather than information and ideas that could affect their decisions.
The short term gains and possible long term gains from running these courses therefore appeared to be a very good use of my time, especially since I was paid by the school to run the courses as part of my normal salary.
The success of these programs led me to believe it could be highly cost-effective having skilled EA educators visit schools to deliver workshops, sparking SHIC’s instructor-led workshop model. However, after our experience running instructor-led workshops, I now think it is likely that my success probably relied on me having an existing relationship with the students and being available for continued support and regular reminders after the course ended.
One of Catherine’s classes running a giving game with the rest of their year group (during a wild animal themed mufti day, they don’t normally dress like that)
One of Catherine’s classes running a “animal friendly lunch” consisting of a vegan sausage sizzle and bake sale, and handing out flyers about how to help animals.
This section is authored by Daniel, who teaches at a large public school in a high socioeconomic area in New Zealand.
I started introducing EA to my school in early 2018 when I emailed staff, outlining an inaugural event for EA Wellington which staff were welcome to attend. Although no staff went to the inaugural event, a child of a staff member who is a very accomplished final year student did attend. They became quite interested in the ideas, borrowing Doing Good Better from me, and went on to be on the Executive Committee of EA Wellington. This student intends to follow a high-impact career path, probably in the field of climate change.
I and the student mentioned above ran a giving game and bake sale, resulting in $300 being donated to GiveWell recommended charities.
I also run philosophy club which started in 2018. Since then, students have slowly been learning and discussing more about ethical theories, amongst other topics. These students are keen to understand the complexities of the world, and how one can make a difference, but apart from the one student mentioned above I would not yet say that any of them are yet likely to take significant EA-specific action as a result of the club.
This section is authored by Alex who is a teacher in an elite Science and Technology focused sixth-form college in the UK.
Since September 2018 I have been running a lunchtime “EA club” with attendance ranging from 5 to 25 students. The club ran weekly, for around half an hour, due to external constraints. The sessions initially focused on discussions of particular issues, and discussion quality and engagement was high. Students were, in general, more excited to discuss “big picture” ideas e.g. cause prioritisation, than more narrow foci, however the short time slot limited discussion quality. Later sessions involved practical activities, for example using guesstimate to estimate the impact of different career plans, and seemed to make better use of the time.
I also delivered an assembly on choosing a career looking at the problem broadly via the framework of 80000 Hours, and brought in an external speaker who spoke to 75 students about his decision to earn to give. Both assemblies were extremely well-received.
The biggest piece of evidence of impact has been this year’s student-run fundraising week, where different classes compete to raise money for charity. Until this year, no EA-recommended charities had ever been nominated as recipients of money raised during the event. Of the 8 charities nominated this year, 5 were one of the top charities in their cause area (according to either OpenPhil, GiveWell, TLYCS or ACE). All of the top 3 charities, including the eventual winner, were top charities, with Evidence Action as the eventual winner, which will receive several hundred pounds. Students also made and voted on videos for Project for Awesome. My informal evaluation of the impact is that the existence of earning to give as a potential career path, AI alignment research as a legitimate and important academic field, and effective altruism as an approach to thinking about the world, are all ideas that many students in the school now take seriously.
This section is written by Jamie Harris, who used to teach at a sixth form college (ages 16-19) near London, UK.
I introduced an optional, extra-curricular club teaching the SHIC curriculum. The college was not fee-paying, but the students mostly came from quite privileged backgrounds. In the first year, I ran it over six separate hour-long sessions after college on Tuesdays. In the second year, I ran it over the entire year in 40 minute sessions at lunch times, with various additional sessions.
In the first year, I started with about 20 attendees, falling to about 10 by the end. In the second year, I started with about 15, falling to about 8. For context, there are about 2,400 students in the college that I taught at.
- I learned quite a lot about EA topics myself through the content. My main area of interest has always been farmed animal advocacy, but teaching the course enhanced my knowledge of topics like global poverty and AI safety, when I was fairly new to EA.
- The students who came seemed to enjoy it and it probably increased their knowledge.
- In the second year, we organised a fundraiser which raised a few hundred pounds to be split between AMF, GiveDirectly, and SCI.
- There were some examples of deeper engagement: students seemed happy to volunteer for involvement in the fundraiser, and two students created two videos for Project 4 Awesome, with my help.
Negative outcomes and disappointments:
- Far fewer students expressed interest than I had hoped
- At various points, I had some indications that the students I believed to have been most engaged with the sessions and ideas had not grasped some of the key concepts, or at least weren't very good at applying them. E.g. I had a session where I discussed the Christmas fundraiser campaigns by The Telegraph (a conservative UK newspaper). One of the most engaged students was adamant that it was preferable that the Telegraph fundraise for guide dogs (I think this was the example) rather than AMF (of which he was personally very supportive) on the basis that doing so was more tractable.
- None of the students took up my offers for further discussion of career plans or engagement with ideas relating to EA.
- Although the students were excited about the fundraiser, it took a lot of my own time.
- We were unable to influence the existing charitable activities of the college.
Questions and uncertainties:
- Would the reaction have been very different in different schools, even without much change to the ways that I approached and advertised the club?
- How different would the reaction have been if I had promoted the sessions differently or run them in another way? I am unsure how far off from the ideal implementation of the programme that I was.
- How valuable is the vague general awareness of impact-focused discussion of charities? E.g. has the existence of the club indirectly contributed to significant improvements in the allocation of charitable giving of staff and students, or will it do so in the future?
Along with another EA who was a teacher at the time, I also spent some time creating a curriculum that was intended to be taught in schools with a younger age group (11-16) in the UK. We spent about 40 hours on the project. The resources have 313 views with an estimated 200 downloads of the entire set of resources, but I'm not sure if those who downloaded the resources used them, or how positive this would even be.
Some thoughts about EA outreach in educational settings:
- As with all EA outreach, there is a tradeoff between aiming for larger audiences and aiming for higher fidelity, more intensive outreach. I think this tradeoff is especially important for outreach in educational settings because the extremes seem more tractable than the middle ground, e.g. seeking to integrate a small amount of discussion of EA ideas in a national curriculum, or directly teaching a small number of promising students.
- Relatedly, there is a decision to be made between seeking to primarily have impact indirectly, through the teaching of ordinary teachers using EA resources and ideas, through the direct teaching of full time EA staff, or through a number of vaguely EA-aligned teacher volunteers.
- Are there existing institutions who can be collaborated with and whose resources can be used to support the growth of EA? Examples might be Teach First (UK) or Teach for America (US), both of which describe themselves as being motivated by positive social impact.
I am currently most optimistic about the ideas of seeking to introduce a small amount of high quality EA content into national curriculums, or using socially motivated existing organisations to introduce a small amount of EA content into schools through their employees. For other ideas of using educational careers to support EA movement growth, see this sheet I created a year or so ago.
Feel free to ask me for clarification in the comments or to reach out to me directly at email@example.com.
This section is authored by Sebastian Becker, who between 2016 and 2018 taught at a secondary school (age 10-16) in a low-income neighbourhood in England as a participant of the Teach First programme.
In autumn 2017, I used parts of the SHIC curriculum in 6 one-hour lessons with my year 7 tutor group of 29 students, aged 10-11. The content was delivered during PSHE (Personal, Social and Health Education) lesson time. I was able to do so as the PSHE curriculum lead teacher had failed to provide a curriculum for the half-term, so teachers were free to create their own lessons.
I also delivered 15-minute assemblies on Singer’s drowning child argument to the year 7 and year 8 cohorts (aged 10-13) of my school (300 students). The assemblies seemed to be well received by the students.
In order to adapt the lessons to the younger age of my students, I usually used only parts of the content of a given SHIC lesson. I removed things which I judged to be too complicated for the students or cut down on some of the longer videos.
The students were engaged, more than in an average maths class and probably also more engaged than in an average PSHE lesson. I think this might have been due to several factors, among them:
- The students noticed that I was passionate about the topic
- When playing the giving game, I committed to donating £50 to the charity the class would choose, to “raise the stakes”
- Some of the material caught their interest, in particular a video about the living conditions in very poor countries
The students seemed quite convinced by arguments that one should consider the cost-effectiveness of one’s donations. When playing the giving game, the majority of the students were in favour of donating to PlayPumps before engaging with the material, but overwhelmingly voted in favour of donating to AMF afterwards.
Left table: votes in the Giving Game before (1) and after (2) engagement with the Giving Game material.
Interpretation of student reaction: I was somewhat surprised by how susceptible the students were to EA-arguments. My initial scepticism was, firstly, based on the fact the school is located in a very politically conservative area. In a mock election in 2015, the immigration-sceptical UKIP party had won the plurality of student votes. I therefore thought the students would argue that one should be more concerned with local, rather than global poverty, however, this argument was never raised. It’s possible that I was able to pre-emptively address this concern by sharing Giving What We Can’s “How Rich Am I” calculator with the class (I also made explicit that there could circumstances were someone might not be able to donate).
These lessons helped me practice explaining EA ideas and strengthened my commitment to EA as the experience was generally motivating. However, I can’t assess whether the lessons had any lasting effects on the students. Given that the students were only 10 or 11 years old and hence the time when they will make significant donation or career decisions are quite far away, it seems fair to doubt this. It is possible that they discussed the ideas with their parents and the lessons had an indirect effect on the parents.
We’d like to thank David Moss and Ida Sprengers for reviewing this post, the numerous volunteers who have helped with these various efforts, and to The Life You Can Save who provided donation money to allow Giving Games to be run at schools.
- Some of the contributors chose not to publish their full names. ⤴
- Fun fact - by the time we did this talk Dan and Brenton had spent so much time together that Brenton’s mum can’t tell their voices apart in this video. ⤴
- We included a social desirability test in our survey to attempt to measure this bias, however there was no clear correlation between the social desirability score and shifts in student responses from the pre-workshop survey to post-workshop survey. ⤴
- We invited all Grade 11 and 12 students we had an email address of. For students who participated in the full SHIC program in the Winter and Spring terms of early 2018, providing a contact email was mandatory. In Summer and Fall 2018 19% of the students voluntarily gave us a contact email. ⤴