Dwarkesh Patel
Podcaster, Dwarkesh Podcast
“So space is really a regulatory. It's really a regulatory play. It's harder to build on land than it is in space”
“if you look at pre training, if you look at llama 3, for example, I think it's trained on 15 trillion tokens. And if you look at the 70B model, that would be the equivalent of 0.07 bits per token in that it sees in pre training in terms of the information and the weights of the model compared to the tokens it reads. Whereas if you look at the kvcache and how it grows per additional token, in context learning, it's like 320 kilobytes. So that's a 35 million fold difference in how much information per token is assimilated by the model.”
“Through the history of programming there's been many productivity improvements, compilers linting better programming languages, et cetera, which have increased programmer productivity but have not led to an explosion. So that's like one that sounds very much like an autocomplete tab. And this other category is just like automation of the programmer. And it's interesting you're seeing more in the category of the historical analogies of better compilers or something.”
“this thing you're saying, which would be intractable and prevents you from actually getting beyond a certain level in Go, is just by default how LLMs are trained ... Karpathy, when he was on the podcast, called it like sucking supervision through a straw.”
“you're trying to maximize as you're learning bits per flop ... you can think of bits per flop as samples per flop times bits per sample ... the samples per flop go down as RL becomes more and more long horizon. But at least this kind of naive RL is also terrible from a bits per sample perspective.”
“with supervised learning ... there's a label that says, actually the term here is blue ... Now, if you were doing this through rl ... you would have to do this on the order of 100,000 times in order to just stumble on blue, then get some learning signal off of that.”
“[On the chip-design-from-bottom-up framing for the Reiner Pope blackboard lecture]: how do chips actually work - starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do.”
“But this kind of continual learning requires going back to the weights.”
“the moment you move into the weights you have to give up on in context learning.”
“the spot prices for compute are more than 40% higher than they were in the February trough that we had earlier this year.”
“1.2x is coming from building new fabs. This process is ultimately going to be bottlenecked up to 2030 and potentially even beyond by just building new ASML EUV machines.”
“Google is paying $900 million a month for 110,000 GPUs that are a blend of GB200s and GB300s. The price that Google is paying here is 2x the spot price per hour for those GPUs.”
“AI has been making the fastest progress in mathematics as of any other field. So whatever is happening here and whatever way we're seeing AI progress happen or not happen, would tell us about what will happen to the rest of the world as AI gets better and better.”
“the kinds of things you can't make benchmarks for are also the kinds of things, at least in the current paradigm, you can't easily train for.”
“When deployment becomes part of training, the returns to being ahead in the AI race accelerate.”
“If you want to change the AI that you're using, you basically had to fire an employee that has accumulated months of context on your organization.”
“the optimal inference batch size for a sparse model like say, deep seq v3 is more than 2,400 concurrent sequences”
“a lot of proposals that have been put forward about regulating AI assume that you train a model and then you deploy it... I don't think this assumption necessarily makes sense in the future.”
“it would make more sense to do monthly or quarterly risk inspections rather than trying to single out some special moment that occurs after training is done but before deployment begins”
“The diversity of AI minds will increase. Right now there are less than five prominent AI minds... if AIs are learning from experience... we could actually see a lot of diversity come out the other end.”
“First is the argument that AI R&D is very verifiable. Second is the argument that if you automate AI R&D, you could get four or five years of progress in a single year.”
“something weird happened in the Bronze Age, and that as you said, across trait after trait, the selection intensifies during the Bronze Age.”
Dwarkesh Patel
One-line summary: Interviewer / podcaster known for long-form AI and economics conversations. Tracked here when guests articulate AI-infrastructure theses; Dwarkesh's own framings (regulatory-play analyses, scaling-law intuitions) show up too.
What they're known for
Brief factual context — fill in.
Why they matter to stock-market
Why this person's claims are tracked here — fill in.
Said
Speaker-attributed claims extracted from diarized sources. Each bullet mirrors one entry in quotes: frontmatter — keep them in sync.
-
On terrestrial-power-flat-to-orbital-dc-arbitrage:
"So space is really a regulatory. It's really a regulatory play. It's harder to build on land than it is in space" — 2026-02-05-dwarkesh-patel-elon-musk-in-36-months-the-cheapest-place-to-put (2026-02-05)
-
On agi-timeline-decade-of-agents:
"if you look at pre training, if you look at llama 3, for example, I think it's trained on 15 trillion tokens. And if you look at the 70B model, that would be the equivalent of 0.07 bits per token in that it sees in pre training in terms of the information and the weights of the model compared to the tokens it reads. Whereas if you look at the kvcache and how it grows per additional token, in context learning, it's like 320 kilobytes. So that's a 35 million fold difference in how much information per token is assimilated by the model." — 2025-10-17-dwarkesh-patel-andrej-karpathy-summoning-ghosts (2025-10-17)
-
On ai-coding-agent-asymmetry-on-novel-code:
"Through the history of programming there's been many productivity improvements, compilers linting better programming languages, et cetera, which have increased programmer productivity but have not led to an explosion. So that's like one that sounds very much like an autocomplete tab. And this other category is just like automation of the programmer. And it's interesting you're seeing more in the category of the historical analogies of better compilers or something." — 2025-10-17-dwarkesh-patel-andrej-karpathy-summoning-ghosts (2025-10-17)
-
On mcts-vs-llm-rl-credit-assignment, mcts-per-move-target-to-llm-rl-inefficiency:
"this thing you're saying, which would be intractable and prevents you from actually getting beyond a certain level in Go, is just by default how LLMs are trained ... Karpathy, when he was on the podcast, called it like sucking supervision through a straw." — 2026-05-15-dwarkesh-podcast-eric-jang-building-alphago-from-scratch (2026-05-15)
-
On rl-information-inefficiency:
"you're trying to maximize as you're learning bits per flop ... you can think of bits per flop as samples per flop times bits per sample ... the samples per flop go down as RL becomes more and more long horizon. But at least this kind of naive RL is also terrible from a bits per sample perspective." — 2026-05-15-dwarkesh-podcast-eric-jang-building-alphago-from-scratch (2026-05-15)
-
On rl-information-inefficiency:
"with supervised learning ... there's a label that says, actually the term here is blue ... Now, if you were doing this through rl ... you would have to do this on the order of 100,000 times in order to just stumble on blue, then get some learning signal off of that." — 2026-05-15-dwarkesh-podcast-eric-jang-building-alphago-from-scratch (2026-05-15)
-
On picks-and-shovels-leading-edge-fab-buildout:
"[On the chip-design-from-bottom-up framing for the Reiner Pope blackboard lecture]: how do chips actually work - starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do." — 2026-05-22-podcast-dwarkesh-podcast-reiner-pope-chip-design-from-the-bottom-up (2026-05-22)
-
On continual-learning-as-next-breakthrough:
"But this kind of continual learning requires going back to the weights." — 2026-06-26-podcast-dwarkesh-podcast-the-next-big-breakthrough-will-be-ais-learning-on (2026-06-26)
-
On continual-learning-as-next-breakthrough:
"the moment you move into the weights you have to give up on in context learning." — 2026-06-26-podcast-dwarkesh-podcast-the-next-big-breakthrough-will-be-ais-learning-on (2026-06-26)
-
On token-price-inflation-favors-asset-heavy-compute:
"the spot prices for compute are more than 40% higher than they were in the February trough that we had earlier this year." — 2026-08-03-podcast-dwarkesh-podcast-why-smarter-ai-models-could-drive-up-compute (2026-08-03)
-
On picks-and-shovels-leading-edge-fab-buildout, token-price-inflation-favors-asset-heavy-compute:
"1.2x is coming from building new fabs. This process is ultimately going to be bottlenecked up to 2030 and potentially even beyond by just building new ASML EUV machines." — 2026-08-03-podcast-dwarkesh-podcast-why-smarter-ai-models-could-drive-up-compute (2026-08-03)
-
On token-price-inflation-favors-asset-heavy-compute, elon-web-services-to-spacex-hyperscaler-rerate:
"Google is paying $900 million a month for 110,000 GPUs that are a blend of GB200s and GB300s. The price that Google is paying here is 2x the spot price per hour for those GPUs." — 2026-08-03-podcast-dwarkesh-podcast-why-smarter-ai-models-could-drive-up-compute (2026-08-03)
-
On ai-math-capability-jaggedness:
"AI has been making the fastest progress in mathematics as of any other field. So whatever is happening here and whatever way we're seeing AI progress happen or not happen, would tell us about what will happen to the rest of the world as AI gets better and better." — 2026-06-30-podcast-dwarkesh-podcast-grant-sanderson-ai-and-the-future-of-math (2026-06-30)
-
On ai-math-capability-jaggedness, agi-definitions-and-benchmark-saturation:
"the kinds of things you can't make benchmarks for are also the kinds of things, at least in the current paradigm, you can't easily train for." — 2026-06-30-podcast-dwarkesh-podcast-grant-sanderson-ai-and-the-future-of-math (2026-06-30)
-
On continual-learning-to-lab-lock-in-moat, continual-learning-as-next-breakthrough:
"When deployment becomes part of training, the returns to being ahead in the AI race accelerate." — 2026-08-15-dwarkesh-podcast-8-predictions-for-the-era-of-continual (2026-08-07)
-
On continual-learning-to-lab-lock-in-moat:
"If you want to change the AI that you're using, you basically had to fire an employee that has accumulated months of context on your organization." — 2026-08-15-dwarkesh-podcast-8-predictions-for-the-era-of-continual (2026-08-07)
-
On continual-learning-to-lab-lock-in-moat:
"the optimal inference batch size for a sparse model like say, deep seq v3 is more than 2,400 concurrent sequences" — 2026-08-15-dwarkesh-podcast-8-predictions-for-the-era-of-continual (2026-08-07)
-
On train-then-deploy-safety-regime-obsolescence:
"a lot of proposals that have been put forward about regulating AI assume that you train a model and then you deploy it... I don't think this assumption necessarily makes sense in the future." — 2026-08-15-dwarkesh-podcast-8-predictions-for-the-era-of-continual (2026-08-07)
-
On train-then-deploy-safety-regime-obsolescence:
"it would make more sense to do monthly or quarterly risk inspections rather than trying to single out some special moment that occurs after training is done but before deployment begins" — 2026-08-15-dwarkesh-podcast-8-predictions-for-the-era-of-continual (2026-08-07)
-
On train-then-deploy-safety-regime-obsolescence, continual-learning-as-next-breakthrough:
"The diversity of AI minds will increase. Right now there are less than five prominent AI minds... if AIs are learning from experience... we could actually see a lot of diversity come out the other end." — 2026-08-15-dwarkesh-podcast-8-predictions-for-the-era-of-continual (2026-08-07)
-
On continual-learning-to-lab-lock-in-moat:
"First is the argument that AI R&D is very verifiable. Second is the argument that if you automate AI R&D, you could get four or five years of progress in a single year." — 2026-08-15-dwarkesh-podcast-ryan-greenblatt-what-happens-once-ai-can (2026-08-11)
-
On bronze-age-selection-pulse:
"something weird happened in the Bronze Age, and that as you said, across trait after trait, the selection intensifies during the Bronze Age." — 2026-05-08-podcast-dwarkesh-podcast-david-reich-why-the-bronze-age-was-an (2026-05-08)
Sources
- 2026-02-05-dwarkesh-patel-elon-musk-in-36-months-the-cheapest-place-to-put
- 2025-10-17-dwarkesh-patel-andrej-karpathy-summoning-ghosts
- 2026-05-15-dwarkesh-podcast-eric-jang-building-alphago-from-scratch
- 2026-05-22-podcast-dwarkesh-podcast-reiner-pope-chip-design-from-the-bottom-up
- 2026-06-26-podcast-dwarkesh-podcast-the-next-big-breakthrough-will-be-ais-learning-on
- 2026-08-03-podcast-dwarkesh-podcast-why-smarter-ai-models-could-drive-up-compute
- 2026-06-30-podcast-dwarkesh-podcast-grant-sanderson-ai-and-the-future-of-math
- 2026-08-15-dwarkesh-podcast-8-predictions-for-the-era-of-continual
- 2026-08-15-dwarkesh-podcast-ryan-greenblatt-what-happens-once-ai-can
- 2026-05-08-podcast-dwarkesh-podcast-david-reich-why-the-bronze-age-was-an
Related
Cross-links — fill in.