GWWC has run an ambitious series of marketing and messaging trials. We aim to run more, including testing innovative features as part of our organizational update. We are targeting all parts of the "funnel", but focusing on outreach and 'conversion to taking a first meaningful step towards effective action'. 
The EA Market Testing team (EAMT) is working with GWWC (and others) to help design, implement, analyze, and preserve and communicate the results of these trials. We (EAMT) aim to help make the trials and innovations more directly successful, as well as increasing and sharing the insights and knowledge gained from these trials.
You can see more of what the EAMT is up to in our (continuously updated) public gitbook here (which includes the full report on the trial discussed in this post).  Below, we offer a summary of this trial, which we believe yielded meaningful results and insights.
Note that the summary and the analysis (in the linked gitbook presentation) is still a work in progress, and we aim to update and improve it over the next months; feedback is very welcome!
Summary of trial and results
Giving What We Can (GWWC) has three giving pledge options, displayed in the 'Original presentation version' below.
From April-July 2021 they ran a trial presenting its 'pledge page' options in three slightly different ways. Considering 'clicks on any button' as the outcome:
- Pledge before Try Giving was the least successful presentation. This was like the one displayed above, but with "Try Giving" in the central position. This had about a 23% lower incidence rate than the Original presentation.
- "Separate Bullets for Other Pledges": The most successful presentation (presented below) only showed a box for "The Pledge", with the other options given in less prominent bullet points below. This had about a 20% higher incidence rate than the Original presentation.
These results may only apply narrowly to the GWWC pledge case, and even here, we have some caveats. However, it loosely suggests that, when making a call to action, it may be most effective to present the most well-known and expected option most prominently, and not to emphasize the range of choices. (See further discussion in the linked gitbook here.)
Getting people to take the GWWC pledge may be seen as an important outcome on its own. It may have a causal impact (see some descriptive evidence here) on getting people into, and engaged in the Effective Altruism community and in other effective altruistic activities, such as directly impactful career choices. E.g., over 20% of EA Survey respondents indicated that GWWC was important for them getting involved in EA.
Once again, for more information about the trial, the statistical analysis, the results and the implications, we direct you to the gitbook section here. We welcome comments below, in the gitbook, or in any other format.
We believe there is a strong case that engaging people in donations and pledges has an important direct impact, as well as being a significant entry point into other forms of EA involvement, such as career decisions.
We also aim to provide our results, analysis, and data (where sharing is allowed) in a transparent web book, in a dynamic document format (using the Quarto tool). A very preliminary and work-in-progress version can be found here (watch this space). We are also looking for volunteers who can help with this statistical and data analysis and communication. I (David) am willing to provide mentoring is available for helping you learn these valuable open science/data science skills, and we also aim to find funds for paid work. Please contact me if you are interested, sharing your CV and a sample of your work, if possible.
The differences below may not be statistically significant in standard frequentist tests; we have not done this analysis (yet). In the Gitbook report HERE we present the Bayesian inference from Google Optimize, which generally suggests that the preponderance of evidence is in the direction described above. We aim to replicate and extend this analysis more transparently in the dynamic document.
The share citing GWWC was even slightly higher for highly-engaged EA’s. Note that effective donations are an important route overall: E.g., GiveWell was cited by nearly 35% of respondents.
To comment in the Gitbook, please use the editor access link here and add comments (or suggest changes if you feel ambitious), or leave comments using hypothes.is and let us know.