Data Centers Part I: The Next Decade Driven by Acceleration & GenAI

Jun. 19, 2024 - 2 minutes 30 seconds
Engineers talking in a data center server room.

The TD Cowen Insight

The concurrent paradigm shifts of generative AI and accelerated computing are driving fundamental changes in the data center across central processing, accelerators, networking, memory & storage, and power. In this report we break down what is changing and how it will impact key companies across the ecosystem (semis in particular) as the industry reacts to these tectonic shifts.

Our Thesis

The paradigm shifts towards accelerated computing and generative AI will both drive and enable profound changes in architecture, especially at data center scale. While we recognize much has been written on AI, we take a more granular view analyzing these paradigm shifts and trace the impact on data center architecture across compute, memory, networking and power. As Moore's Law has slowed, the need for greater offload of tasks to tailored, workload-specific hardware accelerators (of which AI accelerators are a subset) has increased. We see this trend continuing and accelerating, with hardware heterogeneity and tightly coupled accelerators driving efficiency and TCO advantages.

Similarly, the rise of GenAI has driven an explosion of demand for AI acceleration hardware, both for training ever larger models and running at-scale, production-level inference of large language models − with inference itself necessitating acceleration (for the first time). In our view, a rising tide is likely to lift many boats, but the true winners will be the companies that can deliver (or enable) fully optimized systems from silicon to servers through to software. Further, we believe these trends are likely more durable and long lasting than many investors appreciate and could drive above-trend growth through our forecast period (and likely beyond).

What Is Proprietary

In this report, we develop a bottom-up, company-specific framework for data center silicon spending to 2030E across Central Processing Unit, Graphics Processing Unit, Tensor Processing Unit, Application-Specific Integrated Circuit (including both units and average selling price estimates). Further, we discuss in-depth key industry trends including low-level software infrastructure, networking architecture, and power management.

Financial & Industry Model Implications

Overall, we anticipate (at least) high-teens (%) annual growth for data center semis from 2024E to 2030E, with acceleration and networking unsurprisingly growing much faster than the overall market. We also include domain-specific forecasts for Central Processing Unit (including Arm-based processors), AI accelerators (with Tensor Processing Unit and others included), and networking spend (including the Smart Network Interface Card and Data Processing Unit market).

What to Watch

Ongoing investment in GenAI capabilities, software and market penetration; hyperscale, sovereign, enterprise data center CapEx trends and commentary; data center physical infrastructure impacts on power delivery, cooling and efficiency; and durability of demand across data center spending verticals.

Subscribing clients can read the full report, The Next Datacenter Decade, Driven By Acceleration & GenAI - Ahead Of The Curve, on the TD One Portal


Portrait of Matthew D. Ramsay

Managing Director, TMT – Semiconductors Research Analyst, TD Cowen

Portrait of Matthew D. Ramsay


Managing Director, TMT – Semiconductors Research Analyst, TD Cowen

Portrait of Matthew D. Ramsay


Managing Director, TMT – Semiconductors Research Analyst, TD Cowen

Portrait of Krish Sankar

Managing Director, TMT - Semiconductor Capital Equipment, IT Hardware Research Analyst, TD Cowen

Portrait of Krish Sankar


Managing Director, TMT - Semiconductor Capital Equipment, IT Hardware Research Analyst, TD Cowen

Portrait of Krish Sankar


Managing Director, TMT - Semiconductor Capital Equipment, IT Hardware Research Analyst, TD Cowen

Portrait of Joshua Buchalter

Director, TMT – Semiconductors Research Analyst, TD Cowen

Portrait of Joshua Buchalter


Director, TMT – Semiconductors Research Analyst, TD Cowen

Portrait of Joshua Buchalter


Director, TMT – Semiconductors Research Analyst, TD Cowen

Portrait of Michael Elias

Vice President, TMT - Communications Infrastructure Research Analyst, TD Cowen

Portrait of Michael Elias


Vice President, TMT - Communications Infrastructure Research Analyst, TD Cowen

Portrait of Michael Elias


Vice President, TMT - Communications Infrastructure Research Analyst, TD Cowen

Portrait of John Blackledge

Managing Director, TMT – Internet Research Analyst, TD Cowen

Portrait of John Blackledge


Managing Director, TMT – Internet Research Analyst, TD Cowen

Portrait of John Blackledge


Managing Director, TMT – Internet Research Analyst, TD Cowen

Portrait of Derrick Wood

Managing Director, TMT – Software Research Analyst, TD Cowen

Portrait of Derrick Wood


Managing Director, TMT – Software Research Analyst, TD Cowen

Portrait of Derrick Wood


Managing Director, TMT – Software Research Analyst, TD Cowen

back to top