This week researchers with Center for Effective Global Action (CEGA) and the Innovations for Poverty Action (IPA) published a working paper with the results of a first-of-its-kind contactless direct payments program. The program, which was led by the Government of Togo and supported by the research team and GiveDirectly, showed the machine learning targeting outperformed other options available to policymakers at the time, though is best used as a supplemental tool to conventional approaches, especially during times of crisis.
To combat COVID’s impacts, Togo considered several approaches to reach the poor
Last year, when looking to expand their Novissi aid program to more rural regions, the Togolese government considered several approaches to reaching the poorest individuals. This included blanket targeting the poorest regions, or paying everyone registered as an informal worker. They instead worked with GiveDirectly, CEGA, and IPA, who brought together a team to use machine learning and mobile phone data to remotely identify, enroll, and pay over 138,500 Togolese in poverty suffering from COVID19 lockdowns.
The machine learning approach reached more people in poverty than alternatives
This working paper finds that the machine learning approach was more accurate and reduced the number of eligible recipients who would have been excluded from benefits by 8-14% when compared to the alternative geographic targeting approaches being considered by the government at the time (Figure 1). This means 4,000-8,000 more people received aid because of the collaboration between the government, researchers, and GiveDirectly. The researchers conducted this analysis through pre-cash transfer phone surveying of recipients and non-recipients.
It was also the best available approach to minimize inclusion errors, though some did still happen. We’ll take this into account when considering where & when to next deploy this model.
The A.I. targeting did not discriminate against vulnerable groups
CEGA’s findings summary by Anya Marchenko continues below:
One growing concern in algorithm decision making is its potential to discriminate against vulnerable groups. To address these concerns, the authors look at whether the algorithm systematically excluded different demographic groups in Togo, relative to those groups’ true poverty rates.
The authors also find that our machine learning approach does not result in women being systematically more likely to be incorrectly excluded from receiving benefits than men relative to alternative targeting methods (Figure 2a); nor does it result in people of different ethnic groups in Togo to be unfairly excluded (Figure 2b). This parity holds across religions, age groups, or types of households.
This is a rapid, cost-effective, but supplemental tool
The authors conclude that their results — while heartening in the case of Togo — do not imply that ML and phone based targeting should replace traditional approaches reliant on proxy means tests or community-based targeting. Rather, similar to the ideas we outlined here, the authors emphasize that:
“These new methods provide a rapid and cost-effective supplement that may be most useful in crisis settings or in contexts where traditional data sources are incomplete or out of date.”
