All of morecats's Comments + Replies

AMA: JP Addison and Sam Deere on software engineering at CEA

What's tech stack/architecture/hosting situation look like at the moment? Has it hit any specific pain points in the last year? Is it likely to need change?

4JP Addison9moThe Forum is Mongo, Node, React app written in Typescript. We[1] started off on Meteor, which informs a lot of the design patterns, but have recently finished the process of removing it, which has greatly sped up build times. The project was initially build on top of the Vulcan framework [http://vulcanjs.org/], which was a relatively immature framework attempting to provide an even more full-service bootstrapping experience than Meteor. We got off of it several years ago, but it still informs our pattern which you might call UI-inclusive schemas. Other technologies: * We use GraphQL as our API layer, which you can explore here [https://forum.effectivealtruism.org/graphiql]. * Apollo [https://www.apollographql.com/docs/react/] manages the client state * We're hosted on AWS Elastic Beanstalk * Material UI [https://material-ui.com/] provides the styling framework. (This is the framework, not the material design philosophy, which we differ quite a lot from.) * Our Editor is a customized CKEditor 5 editor I'd highlight two pain points that I feel: * Our testing story is pretty poor. So much of what we do is hard to separate from the state of the app that it's hard to make good, non-trivial unit tests. I think this is funged against by good type checking with TypeScript, but my inner Gary Bernhardt [https://www.destroyallsoftware.com/screencasts] is disappointed. * Deploying to AWS takes way too long. I believe this will be fixed soon. [1] "We" here being inclusive of the LW team, with whom we keep up to date.
5SamDeere9moTL;DR; – For Funds/GWWC, the frontends are React (via NextJS) running on Vercel (previously a React SPA running on Netlify). The backend is a bunch of Node.js microservices running on Heroku, connected to a Postgres DB (running on RDS), and wired together with RabbitMQ. We’ve migrated most things to TypeScript, but a lot of the backend is still JS. A lot of business logic is written in SQL/plpgSQL. --- EA Funds and GWWC have been on the same platform since 2017, and share the same backend. The frontend was a pretty standard React SPA written in JavaScript, but we’re currently in the process of deprecating this for a NextJS app written in Typescript (which has been a huge win in terms of productivity). EA Funds has been ported (compare the new [https://funds.effectivealtruism.org] with the old [https://app.effectivealtruism.org/funds]) already, and the incoming GWWC developer will be working with me on porting the existing SPA functionality to the same NextJS-based system. The new sites are in a Git monorepo managed with Yarn Workspaces, meaning that eventually all EA Funds + CEA sites will be able to share components/login architecture/backend connections etc. Down the React rabbithole a bit: We connect to the backend using Apollo to manage GraphQL queries, we use Immer for immutable state management as needed (though we don’t use Redux or any other global state management). UI components are provided with Material-UI. I used to use Netlify to host the EA Funds/GWWC frontend, but we’ve moved to Vercel for their first-class NextJS support. The backend is a collection of quasi-microservices running on Heroku, all written as Node.js apps: * An Express web server that handles our GraphQL endpoint, as well as webhooks/callbacks for various integrations (e.g. Stripe payments). * The GraphQL endpoint is provided by Postgraphile [https://www.graphile.org/postgraphile/], which is essentially a way of generating a GraphQL schema by reflecting over our Postgr
AMA: JP Addison and Sam Deere on software engineering at CEA

Is there any skillset that's been lacking from the team, where you feel a newcomer bringing that skill could have an outsized impact?

4SamDeere9moI think the biggest challenges we face are related to capacity rather than specific skills. So, a really productive fullstack dev could have a huge impact just by virtue of helping us to ship things faster, and cover more surface area. That said, a few things that I think would be great to have more of: * Experience with analytics/measurement/telemetry and using the insights to drive development of new features or content * Exceptional UX/UI chops, to give sites a visual lift and ensure that user flows through our sites are really good * A dedicated product management skillset, conducting user research and using that to inform subsequent iterations of each product
AMA: JP Addison and Sam Deere on software engineering at CEA

What have been the biggest surprises or differences working for CEA vs typical startup or commercial work?

I agree with Sam that most of the difference between working at a typical tech company and CEA  is the size. Even most startup employees probably work at startups 10-100x the size of CEA. Unlike Sam I have worked in the tech industry, but I when I was hired at Kairos, I was employee #6, which I think is unusually early. Relative to Kairos, I think my experience at CEA has differed in the following ways: 

  • I bring more of myself to work. CEA really wants to know what I really think the best thing to do is, and I get to be really open about what I th
... (read more)

I’ll first caveat by saying that I haven’t worked at either a typical startup or big tech company.

I think that there’s probably not a huge difference between CEA and a very early stage startup. I think that the most relevant dimension is just scale – currently we’re two devs working on a bunch of different projects, which means a high degree of autonomy and ownership over the code in a way that I expect is similar to a lot of small startups. We’re obviously a more mature org though, so we do have a lot of processes in place (CI, a dedicated Operations team... (read more)