Cloud Native Efficient Computing is the Way in 2024 and Beyond

The Shift Has Started… Even if You Are Missing It

Since AMD re-entered the server market in 2017, performance per core and core counts have significantly increased. As a result, cores have become larger and chips have gotten faster. However, it was soon realized that there is a demand for cores that are smaller and more efficient, especially for applications that don't require high-performance cores.

The industry discovered that it needed lower-cost chips that prioritize density over per-core performance. This means fitting as many small cores as possible into a system. By doing this, companies can maximize the utilization of applications in multi-core virtual machines or containers. The move away from hypervisors with per-core licensing or per-socket license constraints has also contributed to this trend.

Less Power, Higher Density

Lower power consumption has become a crucial factor. This is especially relevant in the context of building AI infrastructure, where data centers have limitations in terms of expansion and cooling. Transitioning from older Xeon servers to modern cloud-native cores with similar performance per core can allow for 4-5 times more density per system while only consuming approximately twice the power.

The density figures of new generation CPUs are constantly increasing. Companies can achieve higher density by using chips that offer more cores but have lower clock speeds. CPUs with lower clock speeds tend to be more power efficient, making them a suitable choice for applications that prioritize core count over per-core performance. These CPUs help to service applications with less power, which is advantageous in terms of both efficiency and costs.

Cloud-Native CPU Players

There are several notable players in the cloud-native CPU market. AMD offers the EPYC 'Bergamo' and 'Siena' processors. 'Bergamo' is AMD's densest publicly available x86 server CPU with up to 128 cores and 256 threads. It is designed for cloud-native compute and focuses on density and efficiency.

Ampere, led by former Intel Xeon team members, offers the Altra and Altra Max CPUs. These CPUs use a custom-designed core based on Arm architecture. They prioritize low power and high density, making them ideal for serving web pages and other similar workloads.

Intel has its upcoming Sierra Forest processor, which aims to offer up to 144/288 cores and is designed for low power per core while maintaining x86 compatibility. NVIDIA's Grace Superchip is another notable player, offering high core counts and onboard memory for certain memory bandwidth-bound workloads.