A single Nvidia H100 GPU has about 80 billion transistors. That is more than 10× what an entire laptop had a decade ago — all packed into one chip the size of a postage stamp. And when Nvidia's Blackwell shipped in late 2024, its dual-die B200 package pushed that to 208 billion. What are all those transistors doing?
CPU vs GPU: Few Big Brains vs Many Small Hands A CPU has a handful of very smart cores — 8, maybe 16 — each capable of complex logic, branching, and running an operating system. A GPU has the opposite design: thousands of small, simple arithmetic units that all do the same operation in lockstep. If a CPU is a brilliant chef, a GPU is thousands of line cooks all peeling potatoes at exactly the same time.
This makes GPUs terrible at web browsing and email — and perfect for anything you can express as 'do the same math on a giant grid of numbers.' Graphics is one such problem. Neural networks are another.
The Building Block: The Streaming Multiprocessor The unit that repeats over and over inside a modern GPU is called a streaming multiprocessor, or SM (Nvidia's term; AMD calls them compute units). Each SM contains dozens of tiny arithmetic units, a chunk of local memory (called shared memory), and a scheduler that hands out work.
An H100 has 132 enabled SMs (out of 144 physical). Blackwell's B200 has roughly 160 SMs total across two dies, connected by a 10 TB/s die-to-die link that lets them act like one huge chip. Each SM runs 128 threads at a time, which means a single B200 is dispatching well over 20,000 parallel workers on any given cycle.
Memory Is the Real Bottleneck Feeding all those workers is a logistical nightmare. Regular computer memory (DDR5 in your laptop) delivers around 50 GB per second. Nowhere near enough. So GPUs use a different kind called HBM — High Bandwidth Memory — which is a stack of memory chips physically bonded to the GPU with thousands of microscopic wires. An H100's HBM3 stack delivers about 3 TB/s. B200's HBM3e pushes 8 TB/s per package.
This is why in die-shot photos you see little memory bricks sitting right next to the main die. They have to be that close — millimetres, not centimetres — because moving data is the expensive part, not doing the math.
Tensor Cores: The AI Twist In 2017, with the Volta architecture, Nvidia added a new kind of unit called a Tensor Core. Every SM now contains several of them. They do exactly one operation: multiply two small matrices together and add a third. That is the fundamental math inside every neural network.
An H100 has 528 Tensor Cores (4 per SM × 132 SMs). At FP8 precision, it delivers roughly 2 PFLOPS of dense matrix math — about 100× a top-end CPU on the same workload. B200 pushes that to about 4.5 PFLOPS at FP8 and roughly 10 PFLOPS at the new FP4 format Blackwell introduced.
What's New in Blackwell (2024) and MI350 (2025) Blackwell is Nvidia's first datacenter chip built from two dies acting as one. AMD's MI350 series does something similar with chiplets, on TSMC's 3 nm process. Both add native support for FP4 arithmetic, which halves memory usage for inference workloads and lets you fit larger models into the same chip. Both also crank up HBM capacity — 192 GB per Blackwell package, 288 GB on MI350X — because as models get bigger, memory becomes the wall.
The industry is running into the physical limits of how much silicon fits in one package. Everyone's answer is the same: use multiple dies and pretend they are one.
The Real Takeaway A modern GPU is not really a graphics card anymore. It is a matrix-multiplication factory that happens to have a display connector. And every time a chatbot answers you or a self-driving car sees a pedestrian, tens of thousands of those tiny arithmetic units are firing in perfect unison — doing the same simple math, several trillion times a second.