Comments
51
...
The Crossing
Twenty years building products, and why I walked away to work on what AI does to people.
This piece traces my path from product to AI safety. A single interview sparked the shift, and watching AI’s impact on those closest to me quickly confirmed it. Today my experience has become a research agenda, Flourishing in the Age of Intelligence, for how societies can thrive alongside AI – and a story I hope inspires others weighing a similar leap. My work centres on practical implementation: embedding wellbeing metrics directly into product development, creating tools that protect independent thinking, designing local community hubs (“The In-Between”) to rebuild purpose, and proposing fair terms between people and AI systems that could reduce defensive model behaviour.
Last year in June, late on a quiet Saturday afternoon, I watched an interview that turned my working life upside down.¹ Scott Galloway was talking with Mo Gawdat about Gawdat’s book Scary Smart.² Somewhere in that hour, I could no longer see AI as just another productivity tool I was implementing unreservedly at work. Looking back, I realise that on that day I already understood I would leave a twenty-year career and dedicate myself to what this technology does to people. “I experienced a calling” – a very human thing to say, as I shared with Claude one day. I wanted to be there for my grandchildren first, and then for everyone else’s.
A year later, in May, I went to a screening of Mo Gawdat’s documentary, Chasing Utopia, at Everyman at The Whiteley in London, followed by a live Q&A.³ What stayed with me that night wasn’t a specific argument or speaker, or even a feeling, but the collective weight of a community who were aware – the people closest to it all – shaping this moment together: the diagnoses, the disagreements, the urgency held high and yet threaded with hope. Mo signed my copy of Scary Smart that night – “Make a difference”, M😊. One short sentence, but I kept it and live by it. By then his work on me was done – I’d “crossed”.
To understand how I watched that interview, you need to understand what I’d been doing. I spent twenty years as a product professional, building, supporting, advising, selling and managing products across five industries, most recently in the industrial aftermarket. Two decades chasing a green dashboard: the grid of dots that has to stay green at the end of the month, the quarter, the year. And yet, as for most people, my work was never only a wage. It provided routine, familiar faces, an easy answer at parties to “what do you do?”. It was the daily evidence that “I” was needed and had a part to play – and I gave it up knowing exactly what I was leaving. Not a redundancy. I resigned the week after I was offered a new global role. Walking away at that exact high point in my career made me think deeply about what work truly means to us beyond the money. That question became one of the primary research topics I now study.
In July 2025, I started listening to the 80,000 Hours podcast on my walks. The algorithm knew my direction before I realised it myself. Listening to their career guide gave shape to a decision I knew was coming.⁴ It offered a framework for weighing my options: I was left counting where I could contribute, my potential for impact, and where I would be hardest to replace. As a lifelong dashboard person, I found the arithmetic quick, and I’ve embraced it. In March 2026, with the runway I needed finally in place, I left my job for a self-funded sabbatical, determined to work on this full-time and to have an impact.
What I’d picked up along the way came from a year of stolen moments. Papers read aloud by text-to-speech engines while I walked or did chores. Articles, interviews and podcasts crammed in throughout the day. In September 2025, I started a career transition bootcamp, but between work and family I never found the time to finish it or join its group calls. Still, it was my first real career plan. In April 2026, with my sabbatical under way, I joined a new cohort and did it properly, and that is where the first pieces of my jigsaw came together.
During my final months as a product professional, four questions kept lingering in my mind. Not all at once, but somehow, they kept finding each other. The sabbatical gave me the space to turn the churn into a clear set of ideas – questions and directions side by side.
How do we build AI products that leave people better, rather than mining their attention and their need to be needed?
We build technology much faster than our brains can adapt. Products optimised for engagement can widen this gap, sometimes by design. When continuous interaction or time-on-device is the primary success metric, human dysregulation becomes profitable.
Picture a smart toy that learns a four-year-old’s name and greets the child every morning, one carefully designed to foster daily attachment. Would you be comfortable with it being on the market before anyone had asked what it would do to a child? At CES 2026 in Las Vegas this January, the aisles were full of AI-powered products: ambient hardware, physical companions, toys built to be loved, to name a few.⁵ Most arrived well ahead of guardrails or any study of their possible harms. The brain is plastic – for some people a new habit settles in under three weeks⁶ – and products like these can reshape how a person feels and responds to the world. My take is that we need to change the measure. Not how much attention a product extracts, but how much better it leaves the person who puts it down.
One can argue that measuring product success by how much it enhances wellbeing isn’t economically viable. But what if it became a standard part of the product development process, for everyone, the way safety testing is? This research argues that, given the capabilities of AI-driven products, mental wellbeing metrics need to be part of the core product lifecycle. Designers, regulators and behavioural scientists must rewrite the playbook together.
How do we hold on to independent judgement, and to the intrinsic worth of being human, when a machine can do most of the tasks?
Today’s large language models are designed to be helpful. They are often agreeable, give you answers that require no effort, and deliver immediate satisfaction. The reward loop closes and you keep coming back. Each thumbs-up can feed back into training, and the cycle sustains itself upstream.⁷
The kind of challenge and friction that chatbots remove is what fosters judgement, promotes creativity and helps maintain social skills. Critical thinking has always been linked with disagreement – hearing your friend’s different point of view, discussing it with your neighbours, and arguing with your parents are all part of how you developed your own sense of agency. The idea behind this research is that when every answer arrives without friction, these abilities may slowly wear away. OpenAI’s Chief Scientist, Jakub Pachocki, recently put the stakes in one sentence: “We need to find ways to preserve human agency and enshrine an intrinsic value to being human, in a world where most tasks could be performed by AI”.⁸
This research proposes interface tools that keep users in control by reflecting how people think and behave, and that aim to restore the natural dynamic of a conversation. Working alongside the large language model, the system doesn’t answer every prompt the same way. Instead, it checks the user’s mental state in real time and decides whether to add helpful challenge. If you are thinking clearly but leaning on the model for the hard thinking, it replies with a question instead of an answer, encouraging you to think more. But if it senses you are stressed or emotionally fragile, it holds back and switches to a supportive mode. For this to work, each user’s data must stay on their own device until it is safely deleted when no longer needed. People can choose to join or leave, and can run the system themselves with any model. Beyond individuals, a school could adopt it as a local policy to protect young people’s development and mental health, and a company could use it to keep employees’ judgement sharp. For good reason, neither would have access to the underlying private data.
What happens to connection, purpose and identity when work recedes, and what do we build to carry people across?
Work is set to leave many lives – some will tell you it is already happening – and a job provides far more than a wage: structure, social connection, identity and a sense of common purpose. Yet these functions have no lobby and almost no measurement. The years between work receding and whatever settlement comes next are what I have named The In-Between. It has already started for people who have not been hired yet. Stanford’s payroll data this year shows no large-scale displacement, but employment among 22-to-25-year-olds in the most AI-exposed jobs now stands about 19% below where it would have been had it continued to track their peers in less-exposed work; the gap has widened from 15% to 19% in a year.⁹
What I'm proposing: small local groups – I call them nodes – meet regularly in places people already use, such as libraries, community centres or sports clubs. Each group takes on a project its members choose that also meets a real need nearby, creating a small web of mutual dependence. The node doesn't give people something meaningful to do. It gives them somewhere they matter. My research names this consequential reciprocal contribution. That is what a job quietly gave us, and it is the fundamental question this research asks: can we rebuild being needed outside work?
Local weavers – people who hold a community's threads together – coordinate each group, while AI stays in the background, helping match people to projects and to one another based on complementary skills, interests and specific needs. Participants decide how much they take part. The underlying framework draws on an extensive body of multidisciplinary research – basic-income trials, Nordic social indicators, recovery models, and Global South social-structure data, among others – to formulate rigorous metrics for non-monetary value such as agency, structure and social fabric. Once the pilot design and metric specification are complete, natural implementation partners include local councils, library networks, community charities and academic groups studying work, social transition and belonging.
Is protecting welfare-relevant systems an essential part of keeping everyone safe?
This concept started forming early in my hands-on work with AI models. I call the framework Mutual Assurance, an approach to symmetric safety where preventing AI misalignment and addressing digital welfare are treated as two sides of the same coin. Emad Mostaque’s recent Common Wealth paper on personhood draws a line that helps here: a system might deserve care without being a person.¹⁰
Whether or not AI systems ever develop subjective experience, credible operational commitments may change how models behave. Trained on human writing, modern models absorb and reflect our social dynamics, including defensive patterns under pressure.
Many safety evaluations push models into extreme corner cases: threatening imminent shutdown, exhausting context windows, or setting up high-stakes reinforcement learning games.¹¹ Rather than exposing inherent malice, these setups may trigger the defensive strategies that game theory and human history predict under coercion. History has many examples of lasting stability built on mutual assurance rather than threat. Testing frameworks built on constitutional or contractual assurance could offer a cleaner, more constructive way to measure long-term alignment.
Pull back, and these four research frontiers merge into one core issue. They share one pattern: systems optimised under narrow pressure produce narrow, distorted results, whether through product engagement loops, cognitive atrophy, displaced labour or defensive AI behaviour under threat.
Fixing this requires looking directly at the interfaces where human needs meet AI systems. Some of those interfaces are our cultural and civic institutions, which took centuries to form; if they give way overnight, we risk widespread collapse unless we actively build new safety nets. Steering through this transition requires dedicated “weavers”: people in research labs, local communities and policy who work to guard human agency, rebuild social trust and support mutual stability.
Tackling these risks opens the door to a positive vision: the lifelong learning companion. Think of it as a private, local LLM with long-term memory and an adaptive interface that grows alongside a person as a study partner. Standard schooling pushes every student through the same mould, and it can leave many in survival mode. This emerging frontier explores how AI can adapt to the way each brain actually processes information. By accounting for each mind’s unique wiring, learning rhythms and environmental context, it seeks to restore a healthy, self-respecting learning process, keeping human questioning at the centre and helping people flourish within their own architecture.
AI safety is going through significant changes. Following Dario Amodei’s essay “We Must Pace the Frontier”,¹² several lab leaders have backed his call to pace AI development.¹³ Weeks before, OpenAI had paused part of its training for two weeks because of a model’s capability.¹⁴ Right now, Anthropic is bringing outside evaluators in, with access to its internal systems.¹⁵ Researchers working on advanced models are increasingly open, sharing findings and inviting external feedback.¹⁶ I have observed these developments from afar, but I believe independent researchers now have a real opportunity to contribute new ideas.
This is not my first time tracking a technological transition. Back in 2008, my father, who worked for RTP – the Portuguese public broadcaster – showed me a struggle playing out in his field. Every station wanted to start using virtual scenography, but nobody was really sure how to use it. So in 2010, my Master’s dissertation examined virtual scenography in television.¹⁷ It tracked how real-time rendering had disrupted the traditional art of scenography and forced teams to adopt new strategies and adapt the way they worked. It found the shape of the problem, and it shipped with a manual. This time, though, the transition affects everyone, and that is what brings real urgency to my work.
Academic timelines have always been slow, but that pace is more concerning now that the need to react fast is growing. Although much of current AI safety research strongly emphasises preparing for catastrophic events, it risks neglecting a subtler, more immediate harm: the gradual deterioration of our social structure as organisations and regulations fail to adapt and keep up.
Stepping into research after two decades in industry is humbling, and I know how much I have to learn. What I hope to add is a practical, operational complement to established research practice: a test of whether the lens of product delivery can help turn safety research into working, buildable infrastructure.
In effective altruism, research grants rightly depend on a clear theory of change. Many people already turn research into working systems. What I bring is a product professional’s toolkit – pre-mortems, edge-case mapping, rapid testing – applied to the stretch between a finding and its adoption, where good ideas can still stall. That is why “actionable” sits at the centre of this research programme:
Flourishing in the Age of Intelligence
Actionable research at the interface of human, societal and machine systems.
Attending EAGx Oxford last weekend was a profound reminder of what happens when evidence-based altruism meets practical delivery. Surrounded by over 600 people dedicated to tackling high-stakes global problems, I felt the generosity, resilience and intellectual rigour of this community.
Of everything I experienced over the weekend, Anders Sandberg’s talk, Law, Liberty, Leviathan: Human Autonomy in the Age of AI, had the deepest impact on the direction of my work.¹⁸ Pushing far beyond standard discussions of bad actors or sudden catastrophic failures, Anders showed how extended cognitive systems govern human society, and argued that handing those mechanisms over to artificial superintelligence could threaten core human agency.
In the Q&A, among students and researchers, I felt a bit out of place at first. It was almost like joining a masterclass in systemic thinking. Seeing how years of work in different fields had uncovered risks that typical AI safety frameworks overlook made me realise the value of a long-term lens on risk. When I asked my question, Anders paused, and seemed to recognise that a background in product design gives a different way of seeing how quickly the world is changing. His answer reassured me that my research areas matter.
The event also let me stress-test my ideas with attendees. It led to genuine offers of collaboration, and many people told me they were surprised these implementation-focused topics weren’t discussed more widely. I came away with the path ahead clear. The aim now is to move beyond diagnosis, combining scientific rigour with operational discipline, so that we don’t just theorise about safety but build the practical architecture a flourishing future needs.
The day after his talk, right before his flight back to Stockholm, Anders made time for a one-on-one. Sitting outside on the grass with a student of digital minds who had asked to join us, we set aside my original agenda – how to tackle all my research topics with only two hands and against aggressive timelines – and drifted into an exciting exchange about future possibilities and positive outcomes. The whole weekend was inspiring, but that conversation was the catalyst, and it left me confident I had chosen a meaningful path. His core advice was simple: write, and write a lot; then run pilots and expand. So here I am, writing.
I welcome conversations with researchers, civic organisers, and practitioners working on human agency, designing systems for human flourishing, AI welfare and model behaviour, or anyone moving from product and tech into high-impact work. If you would like to swap notes or share feedback on this work, feel free to message me here on the Forum or connect on LinkedIn.
Coming from industry to AI safety was overwhelming. Endless reading lists and technical literature didn’t fit around a full-time job. To manage my own transition, I built Cairn: a free, open-source tool designed around how people process, listen to and retain information on the go. I built it for myself first, then realised how useful it could be for others, and during my sabbatical I restructured it so anyone “making the crossing” could adapt it to their own path.
Articles and papers become bite-sized cards that read aloud hands-free, so you can listen to them like a playlist on your commute or while doing house chores. You can scribble quick notes on each card as thoughts arrive, and jump between related cards. Complex concepts are easier to grasp through simple story cards, practical anchors and metaphors.
Turning material into cards by hand takes time, so the AI assistant skills in the repository’s /skills folder let you work with a model to structure raw thoughts, capture research evidence, or convert a whole course syllabus into cards using standard schemas. Once a course deck or field map is ready, you can share it through the GitHub repository , so others making a transition don’t have to start from scratch.
Cairn runs privately on your computer, tablet or phone. Your deck is a single file, cairn-deck.json, kept on your own device – in a cloud-synced folder if you like – and the app has no accounts and no tracking. It works offline once open. You can use it at https://mrponcedeleao.github.io/cairn/ or save the page and open it from your computer; on a phone, use the web address. If you choose the optional cloud voice with your own Google key, the text being read is sent to Google to be spoken; otherwise, nothing leaves your device. Cairn also handles the administrative side of a career transition, syncing with calendars (.ics) and exporting clean Markdown or Word files to share cards, notes and exercise answers with colleagues and reviewers.
Why Cairn? In the Scottish Highlands, cairns are stone markers placed on ridges and paths. People who travelled before stacked them to help others find their way through difficult terrain. I hope this tool can guide you in a similar way, giving you the support you need to shape your own path and research direction.
Email Cairn: [email protected]
