On Saturday 11th February, we ran a day-long EA hackathon in London, modeled off a similar event in Berkeley late last year. This post follows a similar format to the retrospective linked above. You can see some of the basic information about the event, as well as notes on some of the projects that people were working on in this Google doc.
We are incredibly grateful to @NicoleJaneway and @Zac Hatfield-Dodds for their advice before the event, and to Edward Saperia for his advice and assistance on the day itself.
- We ran a pilot hackathon in London, and were surprised by the success of the event.
- Around 50 people turned up, they gave mostly positive feedback, there were several impressive projects which have plausible impact.
- The event helped build the EA London tech community and generated opportunities for community members to work on impactful software projects on an ongoing, long-term basis.
- We're excited to continue running these kinds of events and want to produce a blueprint for others to run similar hackathons in other places.
Goals of the event
While we hoped that this event would produce artifacts that had legible impact, it was mainly a community-building exercise. Our primary goal was to validate the concept - could we run a successful hackathon? If so, would running similar events in the future lead to greater tangible impact, even if this one didn't necessarily do so?
What went well
- Approximately 50 people attended, which was more than we'd expected.
- The average skill level was high.
- The majority of teams had a strong lead or mentor, in most cases an existing maintainer or the person who had the original idea for the project.
- Asking people if they'd like to be put in groups and then doing so generally worked well - I would estimate this was beneficial for 80% of the people who selected this option. See related potential improvement below.
- Dedicating one of the monthly EA London tech meetups to brainstorming ideas for hackathon projects both yielded good ideas and got people engaged.
- Having a show & tell section encouraged attendees to optimize for having something concrete, and gave groups the chance to learn about what other groups had worked on.
- The venue was excellent.
- The average overall rating for the event on our feedback form was 4.36/5 .
What we could do better next time
- We had a limited budget, and more people showed up then we could provide food for on said budget, meaning we didn't provide lunch after we'd originally said we would. We weren't transparent about this, which was a mistake.
- We didn't do enough to accommodate those who were less experienced coders. In future, we'll use a different question on the sign-up form, along the lines of "how much guidance would you need in order to complete an issue in a project that required coding?".  We can then organise our groups/projects/activities accordingly, including having more ways to contribute to projects through means other than coding.
- We underestimated the ratio of people with jobs in the "data" family relative to the "software" family, and so our suggested projects were almost entirely software-focused.
- We could've had a better shared digital space. This ended up being a bit of an afterthought for us, and we ended up asking people to join a WhatsApp group when they signed in, but it wasn't used much during the day. A different platform could facilitate more collaboration/visibility between groups, allow people to ask for help more easily, and generally give more of a community feel to the event.
Outputs and resources
- There were several pull requests submitted to existing open-source projects, including VeganBootcamp and Stampy.
- Multiple proof-of-concepts for new projects, including:
- An LLM-powered summary generator for opinions on the Alignment Forum.
- An implementation of a reverse-engineering algorithm to visualize the shape of a neural network.
- GPT-Automator, a voice-controlled Mac assistant capable of completing various complex tasks based off speech input. See the blog posts by Luke and Chidi.
- A BOTEC calculator for shallow investigations using the ITN framework.
- Experiments in constraining language models.
- Resources we used in the event (please DM Jonny if you'd like edit access to any of these):
- If you're not already signed up for the EA Software Engineering Newsletter and would like to, you can do so here.
We feel strongly motivated by the outcomes from this event, and think there are a number of ways to build on the momentum from this event, including;
- Similar events run every 2-3 months
- Regular informal "hack nights"
- Themed hackathons, eg one focused on forecasting tools, or an alignment jam
- Helping other organisers run similar events in other places
- Fostering a community of EA software engineers collaborating on EA-related OSS projects
We're keen to define a Theory of Change in order to keep these events value-aligned and impactful. After that, we expect to do more experimentation with event format, frequency etc, as well as ways to encourage community members to work on projects on an ongoing basis, rather than for a single day.
If you would be interested in organizing a similar event then please reach out as we're happy to share as much knowledge as we can!
Admittedly "success" was poorly defined here. Our version of success was primarily about getting attendees to collaborate on projects in groups, and making progress with said projects.
On our expression of interest form, we asked attendees to subjectively rank their familiarity with programming on a scale of 1-5, with the mean answer being 3.275
Options would be something like: "a lot", "a little", and "none".
The actual question we used was "what is your level of familiarity with software engineering on a scale of 1-5?", which in retrospect was too vague. There were only 2 respondents out of 42 that selected '1', which led to the only accommodations being made for folks who put '1' or '2' being placed in groups with people who selected '4' or '5', and all being expected to work on the same project. This left some less experienced attendees feeling like they didn't achieve very much during the day, which was something we were trying to avoid.
Depending on skillsets, it seems best to not be prescriptive about how people contribute to projects, but there are some options for things likely anybody could do, for example QA work or documentation.
Discord seems like a solid alternative. Having multiple channels on the same server would allow people to keep their team's communications in one place, see what other teams are up to, and ask general questions/post memes etc, while keeping each stream of communication focused.
Thanks for organising! I had a great time, I'd love to see more of these events. Maybe you could circulate a Google Doc beforehand to help people brainstorm ideas, comment on each other's ideas, and indicate interest in working on ideas. You could prepopulate it with ideas you've generated as the organisers. That way when people show up they can get started faster - I think we spent the first hour or so choosing our idea.
(Btw - our BOTEC calculator's first page is at this URL.)
I think this is a great idea, thanks for the feedback - I completely agree we want people to be able to hit the ground running on the day. I would imagine groups are most effective when they're formed around strong coders, perhaps there's a way we can work that into the doc.
One thing we're considering is an ongoing Discord server, where people could see ideas/projects/who's working on what, etc. The idea would be that the server would persist between events, and move more towards having ongoing projects as above. I think this could potentially solve some of the cold start issues, but I am also hesitant to ask people to join yet another Discord server, and it'd probably need to reach a critical mass of people in order to be valuable. Having written out this comment, I think we will likely start it and push to get it to a good size, and if not we can re-evaluate.
Thanks for pointing out the bad link, I've corrected it now!
Sounds fun! I think we can set up a few teams in Israel, and run such an event in parallel, if that makes sense.
Great work! Thanks for organising. Also neither of you have formal organising roles right? You just came in and did this (which is great and worth emulating)
Yes that's correct, I'm a software engineer and Sam's a product manager. I think we both felt pretty comfortable with our roles despite of our lack of experience, and would recommend others who are interested in organizing not to be put off by not having done it before. I'm very happy to chat with anyone who feels this way and can hopefully offer more advice than I laid out in the post.
So cool to see this!
We are organizing our first hackathon as EAUC Chile and this posts “what we could do better” as well as your resources will be useful for ours :)
This seems super useful and something I’ve been waiting for. But what is the EA or AI safety connection)