Running www.240Project.org.uk
I’ve spent much of the last twenty five years trying to figure out how best to support vulnerable, isolated adults in a rich country (the UK). These people normally don’t have the support of a family, often have mental health issues, usually have experience of homelessness, and mostly aren’t good at relationships with other people or bureaucratic institutions. Most have suffered trauma at some point, and for most, their daily contact with the rest of us is typically characterised by hostility, rejection or invisibility. Typically they live in homeless hostels or supported accommodation (for long term mental health patients). If they suffer from bipolar or depression or paranoia, they can go long periods where it’s very difficult to provide any kind of meaningful support at all.
It’s great to be able to measure things, to get hard data that lead to clear decisions. There are many domains of existence where measurements and data make great sense. Engineering is one of them. Human life is a lot harder to quantify. If we had metrics that capture how much someone is loved, for instance. We certainly should not be measuring the hours an individual is happy against the hours they are miserable. Or bad experiences against good. Human life is complicated. When Charles Darwin was trying to decide to marry his fiancee, he made a list of pros and cons. And was left none the wiser, because decisions like this are rarely measurable, let alone amenable to logic.
When we started this charity in the late 90s, there was a rage in government to measure things, in particular outcomes. Efficiency was highly prized. We began by partnering with a large homeless organisation that received most of its funding from the government. This meant for instance that for funding reasons we weren’t allowed to support people for more than a year, on the grounds that this was enough support to get them up and running on their own feet again. This was of course a ridiculous assumption. The people we are dealing with have chronic conditions. With a huge amount of support over the long term, a few of them have actually managed to stabilise and become capable of leading what is generally considered a ‘normal’ life. But most will never be able to do that. So for most, they either have access to the kind of support we provide, and therefore have a higher quality of life, or they do not. If they do not, they are more likely to end up back on the streets, to die earlier, or end up in prison or on a mental health ward. We know this because of experience, but the reality is that it's not possible to track people's lives over 30 years or more, so it ends up being very difficult to gather hard data.
So when we are looking to track meaningful targets or outcomes, the most real, tangible measure we can give is whether someone attended 240Project or not. If their day was made better by working on an art project, or doing creative writing. Having a sacro-cranial massage. Meeting other people in a supportive environment. Eating some good food. Being around others in an atmosphere where kindness is the norm.
Many well intentioned funders would like this kind of day to day support to translate into some kind of measurable long term progress. Does our support make it more likely that our members will end up in employment? Or volunteering, or in further education? Are they healthier?
And the answer to questions like these, objective questions, is that often they are. And often they are not. Because, when we look honestly at our lives, what we see over time is that all of us, the marginalised and the successful, go through ups and downs. In fact, quite a number of our members (at a guess, maybe 10%) have in the past been successful, educated people. Mental health issues do not spare any of us.
The point is that if we only work with people as they are in recovery, or making progress, or healing, then our support isn’t real. We at 240Project try to behave in a way that imagines our people having a supportive, non judgemental family that is there for you and provides facilities so you can spend the day being busy doing enjoyable things in a supportive environment. We try our best to give people healthy meals and physical support. Most of them do not exercise. It's hard to keep people healthy. Life being what it is, as often as not, people are on a downward trajectory as on an upward one. And is it right that we should withdraw our support when mental or physical health deteriorates? Isn’t this when we should be supporting people more? Yet it's obvious that from an effectiveness point of view, we are supporting people who might end up just dying on us, the ultimate proof perhaps that we were wasting our time, that we were doing our altruism ineffectively.
For sure, we could cook up some figures that show that the total cost to society of an individual with mental health issues can be very different, depending on whether or not they have the support of 240Project.
But when I think in this way, an alarm bell goes off. One thing we experienced was how important it is for our staff and volunteers to express a certain kind of ethos, and not another one. To try to capture a complex story, if there is an atmosphere of trust and support and calm, the project works. If management becomes heavy handed, bureaucratic, or judgemental, we can deliver on paper exactly the same project, but it will be either pointless or actually negative. We once appointed a manager for the project that acted like a jail keeper. Needless to say this was a disaster.
I would love us to have things we can measure objectively. They would be things like ‘interactions increasing a sensation of kindness’ or ‘a person is left feeling less lonely’, or ‘because of how they spent today, a person feels that they are valued and have something to offer’. These kinds of experiences are the daily commerce of what makes life worth living, that create meaning and make the world a better place. But I hope any reasonable person can get that it would be madness to ask staff and volunteers to log every human interaction they have had, and rate it on a scale of ‘love’ effectiveness.
I’m not saying that there are no objective measures that could help us log our effectiveness. It may be that we haven't found them. The best we have come up with so far is to tell stories. Case studies. But again, when we do tell stories, everyone wants to hear the ones that ‘end’ well. The ones where a crack addict has come into the project, overcome their addiction and gone on to get a job. No one wants to hear about the crack addict that comes in, gets better for a while, then goes back on the streets and is stabbed.
So what I’m left with is simply the question, did we make someone's day better today? If so, then it was worth it. Often, despite everything we do, a depressive might attend the project, have a full day, and leave. Yet if they are asked, they will say the day was really bad. They shouldn’t have come in. They should have stayed in bed. In that case, objectively, we all wasted our time. In reality however, we see this depressive person come back the next day, and the day after, and often project their depression on others, and staff will try their best to keep the atmosphere positive, and mostly succeed. Working with people with chronic mental health issues just is not easy. Staff and volunteers need a huge capacity to recharge and just keep giving attention, and care, without asking for anything like fairness or sanity or logic or effectiveness in return.
After 25 years doing this, I know in my bones that it has not been pointless. That something really good was added to the world because of it, and that this kind of thing should be available to many more vulnerable, isolated people. In our annual report, we put a table of ‘outcomes’ which tries to capture self reported improvements in our members' lives. In truth, I know we don’t actually capture in that table any of the reasons why we are important to our members, why we can be crucial.
And this isn’t just because of ‘negative’ or invisible outcomes (like if a person spent the day doing art work instead of on the streets, they are less likely to commit crime, drink or get involved with negative influences).
Sometimes I think we are in the game of creating instances of meaning (and I’m absolutely not clear what that means). Has there been a spontaneous act of support, where one member is helping another without being asked? In other words, has a genuine act of human altruism happened? Has someone been helped with a problem they face? Has someone learned to look in a new way?
Every year, we put on an art exhibition. This year, we sold works by x artists. Ordinary socially integrated members of the public come to the show, like a work, and buy it. The money raised goes to the artist. And this has an effect. Members of 24oProject who have sold works at the art show tend to start to take their art more seriously. They work harder and concentrate more. Often you can see a passion being kindled. They get into a flow state more often. If you come into the project today, what you see is around 20 tables where people are concentrating on creating art works. Most of these people will sit there and work for five hours, which is how long we are open for. They take breaks, maybe for a session of reflexology. Maybe to go out and smoke and have a chat. But mostly they are concentrating. They are doing something that means a lot to them. A lot more than before they realised that other people liked their works enough to buy them.
Some of our members are able to articulate this. Most are not. More than that however, if we try to capture our effectiveness in the number of art works produced, or number sold, again, we are missing the point.
It may be that there are objective criteria that can capture what we do, to measure how effective we are. It may be that we just haven’t thought of them. There were times when I wanted to feel we were helping to transform people's lives, that we could provide a step change. Yes, that has happened. No, not a lot. Most of the time from an outcomes perspective we are simply pissing in the wind. But on a human level, I know in my bones that we are doing a really good job. People who come into the project to visit us can see it too. So it’s not quite right that we are ineffective. It's more that we aren’t able to generate the kind of data that actually captures what we do. But it is true that we are far less effective and far more expensive than a malaria tablet or an eye operation for a person in India. Yet for me, if 240Project didn’t exist, the world would be a far colder place.
Edward Farrelly
www.240project.org.uk
Thank you for writing and sharing this! I suppose it's being downvoted because it's anti-EA, but I enjoyed reading it and understanding your perspective.
I had three main reactions to it:
I'd be interested in your thoughts!
Hi Isaac, appreciate the response.
Re 2, you capture something important in the phrase 'achieving the best objective outcomes, even if you can't measure them'. That for me is the precise problem, how do you achieve this in the absense of measurement? What are the practical steps I can take to make my charity a better organisation if what I measure doesn't seem to capture the work that is important. I know for instance that our work depends almost entirely on the skill and emotional awareness/warmth of our staff and volunteers. But I can't predict, when hiring, who will be good at this and who wont.
Re 3, for sure I can help people even more. Theres no doubt that funding a hospital in Sudan which has run out of medical equipent is far more important and valuable, or paying a few hundred dollars to restore sight to a blind person in India is better value for money. Those are the kinds of charity I like to donate to. But for me its important that when trying to do something active, to contribute ones time and skills, that one try at least part of the time to prioritise ones immediate local environment, ones local community, and see where the greatest needs and disparities are there. Suicide prevention & homelessness are local issues all over the wealthy world, and very real.
I understand your point about any action meaning other action isn't taken, that one is in effect 'giving up' on others, and how that can be an important tool for prioritising where one directs limited resources (this being pretty much what EA is trying to do, and what attracts me to it), and I think that in the real, practical world, its actually not that complicated to get a feel for ones own biases in that direction. There is a ton of really desperate primary survival need in the world today (though less than there was when I was growing up). I'm absolutely not questioning that charities who support this kind of work should not be prioritised. But I still think that there is room for other organisations, such as my own, that do actually save lives (as reported by those using the service) partly by offering an intangible form of caring that makes people feel less alone and perhaps loved.
Thanks for the thoughtful response!
I think when it comes to how you would make your charity more effective at helping others, I agree it's not easy. I completely agree with your example about it being difficult to know which possible hires would be good at the job. I think you know much better than I do what is important to make 240Project go well.
But I think we can use reasoning to identify what plans are more likely to lead to good outcomes, even if we can't measure them to be sure. For example, working to address problems that are particularly large in scale, tractable and have been unfairly neglected seems very likely to lead to better objective outcomes than focusing on a more local and difficult-to-solve problem (read more at https://80000hours.org/articles/your-choice-of-problem-is-crucial/).
Another relevant idea might be a "hits based" approach, where there's a smaller chance of success, but the successful outcome would be so good that its expected value is better than (say) the best GiveWell-style measurable approach.
To be completely clear, I'm not saying I think you're making a mistake if the reason for focusing on people struggling in the UK is either that you want to help people but don't mind about how big a difference you make (you clearly are helping!), or if you definitely want to work on something you have an emotional connection to. But if your goal is to help other people as best you can, then that's where the EA approach makes a lot of sense :)
Put another way, I completely agree that there are serious problems in all places, including in wealthy countries - but I don't prioritise working on helping people in the UK because (a) I want my efforts to help others as much as possible, (b) it's clear that I can help much more by focusing on other problems and (c) I don't see a reason to prioritise helping people just because they happen to live near me. If you disagree with any of those, I think it's perfectly reasonable to keep focusing on people in the UK! But I think on reflection, many people actually do want to help others as best they can[1].
It is surprisingly emotionally difficult to realise that even though the thing you are working on is hugely important (and EA doesn't at all disagree with that), there are other problems that might deserve your attention even more. It took me a while to come around to that, and I think it is psychologically difficult to deal with the uncertainty of suddenly being open to the possibility of working on something quite different than your old plan.
One caveat is that although I mostly want to do the EA thing of making the biggest difference possible, I also do separately sometimes want to do something that makes me really feel like I'm making a difference, like volunteering to address a problem near me, and that's obviously fine too, it's just a different goal! We all have multiple goals.