Effective outreach: evaluating "Idea innoculation"

This is a (relatively) short post and research proposal brainstormed during an Oxford EA Fellowship group discussion on EA Community Building. Thanks to Konrad Seifert and the other members  of  Cohort 31 for feedback on this post. All error and omissions are my own. 

Motivating statements:

• Outreach and movement building plausibly have very high positive expected return.
• There is concern within the EA community about ‘idea inoculation’, ie that exposure to EA ideas expressed in an unconvincing or incorrect way may result in prospective EAs being harder to convince when they are next exposed to EA ideas. This is also expressed as a desire to transmit EA idea methods through ‘high fidelity’ methods (groups, academia) vs those with greater reach (media).
• Idea inoculation has fairly strong implications for optimal movement building and outreach efforts. If it were true, it would imply discouraging newer, less involved, or less convincing EA members from promoting EA outside the existing community.
• There is empirical evidence for idea innoculation, but little of it has been conducted around charitable giving / EA topic areas, or in ways that seem directly relevant to the way it could occur in this community. In particular a lot of the research seems to look at active innoculation (containing refutational preemptions) vs. innoculation that occurs inadvertently. I'm not certain about this, as I don't have access to all the papers (and haven't conducted an very extensive review), so it's possible the existing literature does apply to EA. I think such this sort of review would be valuable.
• There is also evidence that people form strong first impressions about other people, and that those impressions are resistant to change. But it’s not clear those same effects apply to ideas, especially if the idea is transmitted through someone the receiver trusts.
• Additionally, impact often seems to be achieved by outliers. In the EA case we might expect people with potential for unusually high impact to also be unusually thorough thinkers; capable of quickly dismissing badly presented ideas to focus on the most relevant things. Thus, it would be good to understand whether these people (eg. as measured through high rationality quotient score) are more or less susceptible to idea inoculation.

Proposal: Based on the motivating statements above we think research on idea inoculation could form the basis of a valuable, EA-motivated psychology research paper. 

Potential experimental design:

1. Volunteers fill out a survey on how convinced they are by certain core EA ideas (eg. “I have a disproportionate ability to do good”, “some interventions are much more effective than others”).
2. Randomizing undergraduate volunteers into three experimental conditions:
   1. Listening to a highly rated EA (or other cause) speaker (treatment 1)
   2. Listening to a poorly rated EA (or other cause) speaker (treatment 2)
   3. Listening to an unrelated speaker (control)
3. The volunteers then fill out a survey indicating how convinced they were by the arguments of the speaker they listened to.
4. Two weeks later split the groups evenly between:
   1. Listening to a highly rated EA (or other cause) speaker
   2. Listening to a poorly rated EA (or other cause) speaker
   3. Listing to an unrelated  speaker
5. Volunteers again fill out a survey indicating how convinced they were by the speaker, and how convinced they are in the ideas in general.

Example hypothesis questions:

• Were students who listened to the poorly rated EA speaker less convinced than the control group (ie. students who heard no EA speaker at all)?
• Are students who first heard the poor EA speaker and then the highly rated EA speaker more convinced than those that heard just the highly rated speaker?
• How much more effective (if at all) is the highly rated speaker at convincing people of EA ideas than the poorly rated speaker?

Possible variations:

• Provide the students with money in stage three and give them the option to donate it to an EA charity, control charity, or keep it.

Additional variables to look at:

• Is the effect different in older vs younger people (ie. are, as CEA expects, students and young professional easier to convince)
• Is there variation in the result based on individuals rationality quotient or other character trait measures (eg. Big Five personality traits).

Why is this a good experimental opportunity?

• The experiment closely mirrors relevant real-world conditions (EA organisations often are aim to attract university students, and they’d be likely to hear talks at their university or from other students)
• Results have strong implications for EA outreach policy (both individual and institutional)