Effective Altruism Forum
EA Forum

Hide table of contents
by Ebenezer Dukakis
1 min read2 answers 0

9

AI safetyAI governanceEconomics of artificial intelligenceOpenAI
Frontpage

NVidia's market cap has gone up by 10x in the past few years on the strength of the AI boom.

At first glance, one would assume that NVidia is incentivized to accelerate the boom, in order to increase demand for their chips.

However, OpenAI's recent o3 announcement suggests that programming and math jobs will be some of the next jobs to be automated away. As far as I can tell, NVidia's competitive moat basically comes in the form of software which does math. The company appears to be quite light on tangible assets, with a book value that's around 2% of their market cap.

Not only that, but o3 suggests that the jobs most vulnerable to automation are jobs with an objectively correct answer. NVidia isn't in the business of writing pretty iPhone apps which are subjectively evaluated by users. In principle, it seems maybe AMD or Google could construct a big test suite for e.g. PyTorch, such that if their software passes the entire suite, NVidia's moat is gone.

(Why does this question matter for EA? If NVidia sees the problem, and becomes scared, it would want to sell OpenAI fewer chips, and direct the chips towards other AI players instead. Presumably that would be a good thing? Perhaps NVidia isn't incentivized to accelerate the AI boom after all!)

9

0
0

Reactions

0
0
New Answer
New Comment


2 Answers sorted by
Top

huw

6
1
0

Nvidia’s moat comes from a few things. As you pointed out, they have CUDA, which is a proprietary set of APIs for running parallelised math operations. But they also have the best performing chips on the market by a long way. This is not merely a function of having strong optimisation on the software side (possibly replicable by o3 but I would need to see more evidence to be convinced that an LLM would be good at optimisation), or on the hardware side (much, MUCH trickier for an LLM given that a lot of the hardware has to operate on nanometre scale, which can be hard to simulate), but also because having the most money and a strong track record & relationship means they can get preferential access to next-gen fabs at TSMC.

It is also true that the recent boom has increased investment into running CUDA code on other GPUs. The SCALE project is one such example. This implies (a) the bottleneck is not about replicating CUDA’s functionality (which it does), but more about replicating its performance (they might have gains to make there) and/or (b) that the actual moat really does lie in the hardware. Again, probably a mix of both.

However, this hasn’t stopped other companies from making progress here. I think it’s indicative that Deepseek v3 was allegedly trained for less than $10m. If this is true, it suggests to me that:

  1. Frontier labs might be currently using their hardware very inefficiently, and if these efficiencies were to be capitalised on, demand for Nvidia hardware would reduce (both by using less of their GPUs, but also because you wouldn’t need the best of the best to do well)

  2. If it turns out to be cheap to train good LLMs, captured value might shift back to frontier labs, or even to downstream applications. This would reduce Nvidia’s pricing power.

Also, it looks like the competition is catching up anyway. It seems like it’s very reasonable to do inference on Apple or Google chips (Apple Intelligence runs on M2-series chips, these also have top TSMC node access; Google run a lot of inference on their own TPUs). I was particularly impressed that you can run a 600B+ parameter model on 8 Mac Minis, not even running Apple’s best chips. Even if it’s only inference, that’s a huge chunk of the market that might fall to competitors soon.

So I’m not exactly counting on Nvidia to hold, but I think it will be for other reasons than automation. Even if you are very AI-pilled, we still live in the world where market dynamics are much stronger than labour automation effects. For now :)

MichaelStJules3
0
0

There's a thread here about hardware between companies, with johnswentworth arguing AMD had better hardware than Nvidia.

Reply
1
Ebenezer Dukakis
Interesting. Seems like the obvious way to resolve this disagreement would be to check benchmarks for SCALE. 6 months ago, in this reddit thread, they said benchmarks will "become publicly available in the near future". So, join their discord if you're holding Nvidia stock, I guess?
Ebenezer Dukakis1
0
0

Thanks.

Regarding decreased demand for GPUs -- I agree Deepseek v3 is evidence that demand will decrease. But o3 seems like evidence that demand will increase, since we'll be doing more test-time compute. Unclear which evidence is more compelling.

Reply
Rasool1
0
0

Well put! I will add that the competition includes companies like Amazon creating their own chips and others designing silicon specialised for inference (like Groq (not to be confused with xAI's model Grok))

Reply

Rasool

3
0
0

Dylan Patel of Semianalysis (one of the leading semiconductor research firms) did a good recent episode on the Bg2 podcast where he covers this question and others:

Youtube link

Dexa summary

Curated and popular this week
Paul Present
 ·  · 28m read
 · 
Note: I am not a malaria expert. This is my best-faith attempt at answering a question that was bothering me, but this field is a large and complex field, and I’ve almost certainly misunderstood something somewhere along the way. Summary While the world made incredible progress in reducing malaria cases from 2000 to 2015, the past 10 years have seen malaria cases stop declining and start rising. I investigated potential reasons behind this increase through reading the existing literature and looking at publicly available data, and I identified three key factors explaining the rise: 1. Population Growth: Africa's population has increased by approximately 75% since 2000. This alone explains most of the increase in absolute case numbers, while cases per capita have remained relatively flat since 2015. 2. Stagnant Funding: After rapid growth starting in 2000, funding for malaria prevention plateaued around 2010. 3. Insecticide Resistance: Mosquitoes have become increasingly resistant to the insecticides used in bednets over the past 20 years. This has made older models of bednets less effective, although they still have some effect. Newer models of bednets developed in response to insecticide resistance are more effective but still not widely deployed.  I very crudely estimate that without any of these factors, there would be 55% fewer malaria cases in the world than what we see today. I think all three of these factors are roughly equally important in explaining the difference.  Alternative explanations like removal of PFAS, climate change, or invasive mosquito species don't appear to be major contributors.  Overall this investigation made me more convinced that bednets are an effective global health intervention.  Introduction In 2015, malaria rates were down, and EAs were celebrating. Giving What We Can posted this incredible gif showing the decrease in malaria cases across Africa since 2000: Giving What We Can said that > The reduction in malaria has be
LintzA
 ·  · 15m read
 · 
Cross-posted to Lesswrong Introduction Several developments over the past few months should cause you to re-evaluate what you are doing. These include: 1. Updates toward short timelines 2. The Trump presidency 3. The o1 (inference-time compute scaling) paradigm 4. Deepseek 5. Stargate/AI datacenter spending 6. Increased internal deployment 7. Absence of AI x-risk/safety considerations in mainstream AI discourse Taken together, these are enough to render many existing AI governance strategies obsolete (and probably some technical safety strategies too). There's a good chance we're entering crunch time and that should absolutely affect your theory of change and what you plan to work on. In this piece I try to give a quick summary of these developments and think through the broader implications these have for AI safety. At the end of the piece I give some quick initial thoughts on how these developments affect what safety-concerned folks should be prioritizing. These are early days and I expect many of my takes will shift, look forward to discussing in the comments!  Implications of recent developments Updates toward short timelines There’s general agreement that timelines are likely to be far shorter than most expected. Both Sam Altman and Dario Amodei have recently said they expect AGI within the next 3 years. Anecdotally, nearly everyone I know or have heard of who was expecting longer timelines has updated significantly toward short timelines (<5 years). E.g. Ajeya’s median estimate is that 99% of fully-remote jobs will be automatable in roughly 6-8 years, 5+ years earlier than her 2023 estimate. On a quick look, prediction markets seem to have shifted to short timelines (e.g. Metaculus[1] & Manifold appear to have roughly 2030 median timelines to AGI, though haven’t moved dramatically in recent months). We’ve consistently seen performance on benchmarks far exceed what most predicted. Most recently, Epoch was surprised to see OpenAI’s o3 model achi
Rory Fenton
 ·  · 6m read
 · 
Cross-posted from my blog. Contrary to my carefully crafted brand as a weak nerd, I go to a local CrossFit gym a few times a week. Every year, the gym raises funds for a scholarship for teens from lower-income families to attend their summer camp program. I don’t know how many Crossfit-interested low-income teens there are in my small town, but I’ll guess there are perhaps 2 of them who would benefit from the scholarship. After all, CrossFit is pretty niche, and the town is small. Helping youngsters get swole in the Pacific Northwest is not exactly as cost-effective as preventing malaria in Malawi. But I notice I feel drawn to supporting the scholarship anyway. Every time it pops in my head I think, “My money could fully solve this problem”. The camp only costs a few hundred dollars per kid and if there are just 2 kids who need support, I could give $500 and there would no longer be teenagers in my town who want to go to a CrossFit summer camp but can’t. Thanks to me, the hero, this problem would be entirely solved. 100%. That is not how most nonprofit work feels to me. You are only ever making small dents in important problems I want to work on big problems. Global poverty. Malaria. Everyone not suddenly dying. But if I’m honest, what I really want is to solve those problems. Me, personally, solve them. This is a continued source of frustration and sadness because I absolutely cannot solve those problems. Consider what else my $500 CrossFit scholarship might do: * I want to save lives, and USAID suddenly stops giving $7 billion a year to PEPFAR. So I give $500 to the Rapid Response Fund. My donation solves 0.000001% of the problem and I feel like I have failed. * I want to solve climate change, and getting to net zero will require stopping or removing emissions of 1,500 billion tons of carbon dioxide. I give $500 to a policy nonprofit that reduces emissions, in expectation, by 50 tons. My donation solves 0.000000003% of the problem and I feel like I have f
Recent opportunities in AI safety
23
Last week to apply for the Futurekind AI Fellowship! (deadline: April 1)
Jay Luong
· · 1m read
0
0
27
Out of This Box: The Last Musical (Written by Humans) - Crowdfunding!
GuyP
· · 7m read
0
0
39
Australians for AI Safety Launches New Election Campaign — Here's How You Can Help
Luke Freeman 🔸
· · 3m read
5
5
View more