If you don't already have it, I would strongly recommend getting a copy of Gerber & Green's Field Experiments. I would also very strongly recommend that you (or EA Cameroon) engage an experimental methodology expert for this project, rather than pose the question on the forum (I am not such an expert).
It is very difficult to address all of these questions in a broad way, since the answers depend on:
- The smallest effect size you would hope to observe
- Your available resources
- The population within each cluster
- The total population
- Your analysis methodology
I'm a little confused about the setup. You say that there are 6 groups— so how would it be possible to have "6 intervention + 3 non-intervention?" Sorry if I'm misunderstanding.
In general, and particularly in this context, it makes sense to split your clusters evenly between treatment and control. This is the setup that minimizes the standard error of the difference between groups. When the variance is larger, smaller effect sizes are difficult to detect. The smaller the number of clusters in your control group, for example, the larger the effect size that you would have to detect in order to make a statistically defensible claim.
With such a small number of clusters, effect sizes would have to be very large in order to be statistically distinguishable from zero. If indeed 50% of the population in these groups is already masked, 6 clusters may not be enough to see an effect.
Can we get some clarification on some of your questions? Particularly:
How important, in terms of statistical power is to include all clusters
If you have only 6 to choose from, then the answer is very important. But I'm not sure this is the sense in which you mean this.
How many persons should be observed at each place?
My inclination here is to say "as many as possible." But this is constrained by your resources and your method of observation. Can you say more about the data collection plan?
Hey, thank you for the work you are doing! Here are my thoughts (I'm an economist at IDinsight and work on this type of research):
More technical details:
Since you're doing a clustered RCT -- treatment is at the village level and the outcomes of people within a village are likely positively correlated -- you'll need a larger sample size than if you were doing an individual-level RCT (for the math, see section 4.2 of this -- generally a great resource for RCT design). You can do a power calculation for a clustered randomized controlled trial, e.g. using Stata's "power twomeans" command. One parameter that's missing is the intraclass correlation (correlation among individuals within a treatment unit). However, since your cluster size is SO small (3 and 3), when I try to do this calculation in Stata with any reasonable assumption Stata says you cannot have enough power (assuming you want all the standard -- 80% power, 5% significance level etc.). That's why I recommend not doing an RCT unless you have a program at scale
Hello Sindy,
Thank you so much. This answers my question. Yes, there will be a before and after qualitative survey asking about own and others' behavior - which may need to be truncated to speak with more different groups. Then, the face covering data can be used to complement the survey information.