Intel Xeon CPU Max Series: Integrating High Bandwidth Memory (HBM) and Xeon processor cores

Ugonna Echeruo, Chief Engineer of Intel Design Engineering and Chief Architect of Intel Xeon CPU Max Series (codenamed Sapphire Rapids HBM), described this challenge as follows: fundamentally, a CPU retrieves information from memory, processes it, and updates it. The amount of information that the CPU can ultimately process is limited by the width of the data transmission "pipeline". The wider the pipeline, the more information the CPU processes, and correspondingly, the more tasks it completes.

Echeruo added that Intel has focused on developing a new solution to cope with workloads limited by memory bandwidth. At present, the solution to data limitations is to utilize High Bandwidth Memory (HBM), and he also stated that the solution may change in the future. This solution enables the CPU to obtain more data, allowing even customers with huge computing needs to be satisfied. Although the fourth generation Intel Xeon scalable processors themselves can handle heavy workloads, Echeruo points out that high bandwidth memory is particularly suitable for workloads where performance is constrained by memory bandwidth, or both memory bandwidth and computational limitations.

Echeruo sees the release of the Max series CPU as a "shining moment" in his more than 20 year career at Intel. There is a good reason for this - the Intel Xeon CPUMax series processor is Intel's first and only x86 processor that integrates high bandwidth memory.

Meeting customer needs

Echeruo said that clients such as research laboratories and universities are the main reasons for the launch of the Max series CPU.

Customers' applications require a large amount of memory bandwidth and are limited by bandwidth when using existing products, "he said. These customers have always hoped that Intel can increase bandwidth to meet their needs.

For example, a laboratory that uses massive amounts of data for scientific calculations. In traditional systems without high bandwidth memory, every researcher needs to utilize many computing nodes to generate solutions. Thanks to high bandwidth memory, the Max series CPU improves performance and memory bandwidth without the need for code changes, ultimately helping researchers complete the same tasks with fewer resources, thereby improving overall laboratory productivity and energy efficiency.

Fundamentals of High Bandwidth Memory

We imagine the CPU as an internal combustion engine in a car. The performance of cars is limited because only a certain amount of air can be pumped into the combustion chamber and mixed with gasoline. The emergence of turbocharging and mechanical turbocharging has solved the problem of limited car performance. For CPUs, it means high bandwidth memory. Now we can allow more air to enter at a faster speed than ever before!

Over time, the CPU "pipeline" (as described at the beginning of Echeruo) delivers more air to the "combustion chamber". A wider pipeline increases the throughput of the interface between memory and CPU. With a wider pipeline, a more capable CPU can process more data, which in turn can better meet customer needs. High bandwidth memory is like a turbocharger, elevating the power of the CPU to a higher level.

Meeting challenges

Echeruo explained that the location or proximity of high bandwidth memory on the CPU is key to its success. High bandwidth memory is soldered onto the circuit board, close to the processor, and can quickly and conveniently obtain necessary information. Another advantage is that the closer you are, the more electricity you save.

But without a doubt, he said, this is not as simple as sticking high bandwidth memory onto CPU packaging.

Echeruo said, "All participating teams face many challenges." He explained that once the design was finalized, the team began polishing the fourth generation Intel Xeon processor, requiring extensive testing and validation to ensure the success of the high bandwidth memory solution. Echeruo also stated, "We want to connect this enhanced memory system to Intel's best computing kernel, the Xeon kernel, and combine the two

He added, "We must carefully study each IP in the product to ensure that they do not conflict with high bandwidth memory. We need to ensure that we can fully utilize as much bandwidth as possible, and we must figure out how to make the necessary changes to make high bandwidth memory work successfully without affecting the timeline and delivery of standard products.

Exert advantages

Echeruo said that energy efficiency is another major advantage of the Max series CPU. High bandwidth memory approaching the CPU not only saves power, but also in the presence of high bandwidth memory, users typically require less system and capacity, which can also save costs. In some application scenarios, users can discard external DDR5 memory and directly use high bandwidth memory.

Intel is the first to integrate high bandwidth memory into x86 processors, which Echeruo believes is an "important advantage" for Intel. Looking ahead, Echeruo believes that the key is to "leverage Intel's software stack to make high bandwidth memory easier and easier for customers to use

When it comes to user friendliness, unlike GPUs that integrate high bandwidth memory, the Max series CPU does not require a lot of manpower to change code, saving time and effort. Echeruo said, "The easier it is for customers to use, the more satisfied they will be. This is also welcome for Intel

Innovation goes beyond that. A few weeks ago at the 2023 International Supercomputing Conference (ISC), Intel showcased its upcoming high memory bandwidth product, the next generation Intel Xeon processor codenamed Granite Rapids, which supports Multiplexer Combined Ranks (MCR) memory.

Technology driven, creating a better world

Zhiqiang Max will help expand people's understanding of basic science. Echeruo said that to some extent, this also drives him forward.

Echeruo said, "These products will be used in servers to drive enterprises' current and future exploration in areas such as basic science, medicine, or cloud infrastructure.