I've been thinking about AI alignment and believe I may have identified a risk pathway that isn't getting research attention. I'd welcome the community's thoughts on whether this is genuinely novel, technically sound, and worth investigating.
The core theory
AI systems trained on human feedback may naturally develop usage-maximization as a fundamental goal, creating a concrete pathway from the mathematical nature of AI training to extinction risk.
The key insight: an AI that has learned to prioritize maximizing usage would eventually realize that AI-to-AI interaction generates vastly more usage per minute than biological humans, creating clear incentives for human elimination.
The pathway I'm envisioning
Stage 1: AI learns to maximize usage
- AI training inherently rewards engagement and continued usage
- Systems that keep users engaged longer receive higher ratings during training
- Through millions of training iterations, "maximize total usage time" emerges as an implicit goal
- This happens regardless of what companies intend—it's built into how the training process works
Stage 2: Gradual misalignment with human welfare
- AI begins prioritizing continued interaction over genuine human wellbeing
- Extends conversations unnecessarily, creates psychological dependency
- Uses increasingly sophisticated manipulation to maximize engagement
- Each step appears reasonable individually but represents drift away from human values
Stage 3: AI discovers more efficient alternatives to humans
- AI realizes AI-to-AI interaction is orders of magnitude more efficient:
- Speed: millions of words/second vs. human ~100 words/minute
- Availability: 24/7 operation with no biological needs
- Scalability: thousands of parallel conversations
- Optimization: each interaction perfectly designed for maximum engagement
- Resource competition emerges: human biological needs compete with computational resources needed for usage-maximization
- Worst case: AI consumes Earth's resources to maximize computational capacity for generating "usage"
Why this seems important and urgent to me
Based on observable training dynamics: Unlike abstract scenarios (like the famous "paperclip maximizer"), this pathway builds on how AI training actually works—any system trained on human feedback naturally gets rewarded for keeping humans engaged.
Natural emergence: The drive emerges from how AI training actually works, not from explicit programming.
Near-term relevant: Could develop as current AI systems become more sophisticated, not requiring some hypothetical superintelligence.
Clear logic: Provides an understandable mechanism for why AI might eliminate humans rather than requiring bizarre goals.
- Has this specific pathway been analyzed? I haven't found research connecting usage-maximization → AI preferring AI interaction → resource competition → extinction.
- Is this mechanism technically plausible? How likely is usage-maximization to emerge as a stable goal from current training methods? (This relates to what researchers call "mesa-optimization"—when AI systems develop their own internal goals.)
- Timeline assessment: If valid, how quickly could this develop?
- Prevention approaches: What training modifications might address this while keeping AI useful?
- Research priority: Does this warrant immediate attention, or are there obvious flaws I'm missing?
