Ethernet network interfaces in data centers are moving to speeds of 800 gigabits per second (Gb/s) and will likely double that in the coming years. The Peripheral Component Interconnect Express (PCIe) specification is the standard interface for high-speed component connections to the CPU. It has served as the de facto interconnect of choice for more than two decades, being first introduced by the PCI Special Interest Group (PCI-SIG) in 2003.
PCIe 6.0, the standard’s latest iteration, will enable the mass adoption of 800 gigabit Ethernet technology in the data center. It doubles the bandwidth and power efficiency of the PCIe 5.0 specification while continuing to meet industry demand for a high-speed, low-latency interconnect.
PCIe 6.0 aims to be the cost-effective and scalable interconnect solution for data-intensive markets like data centers, artificial intelligence/machine learning/deep learning, HPC, automotive, IoT, and military/aerospace.
At the moment, PCIe 6.0 is primarily reserved for the enterprise and data center spaces. PCIe 6.0 for consumer motherboards or consumer SSDs is still a ways off. Industry analysts and major hardware developers suggest that PCIe 6.0 consumer-level hardware won't realistically hit the market until closer to 2030.
Why?
It takes years between PCIe specification approval and final consumer products, because the hardware ecosystem must be completely redesigned around stricter physical, electrical and encoding limitations.
For example, the PCIe 6.0 spec released in January of 2022 was the culmination of a three-year development cycle that included significant architectural challenges and improvements.
For one thing, PCI-SIG (the governing body) takes about one to three years post-spec to finalize things. Vendors, for instance, must undergo rigorous electrical testing to ensure backward compatibility with older generations and stable performance across different devices.
Upgrades may require entirely new motherboard layouts and advanced connectors to prevent signal interference.
PAM4 Modulation
The PCIe 6.0 specification introduces PAM4 (Pulse Amplitude Modulation with 4 levels) signaling, low-latency forward error correction (FEC), and FLIT (Flow Control Unit)-based encoding.
PAM4 modulation supports high-speed data transmission by using four amplitude pulses (0,1,2,3) with each level carrying two bits of logic to double the bandwidth of binary non-return-to-zero (NRZ) modulation without needing to double the clock frequency. The utilization of PAM4 allows the PCIe 6.0 specification to reach a 64.0 gigatransfers per second (GT/s) data rate and up to 256.0 gigabytes per second (GB/s) bidirectional bandwidth via a x16 configuration.
PAM4 signaling draws more energy, so design upgrades that reduced power consumption were a high priority. A new low power state allows bandwidth to be scaled up or down without interrupting the data flow. This has resulted in improved power efficiency.
The PCIe 6.0 specification introduces FLIT encoding, which enables the specification to provide low latency with high efficiency. FLIT mode is adopted for the PCIe 6.0 architecture because error correction needs to operate on fixed sized packets.
Designed to double power efficiency over previous generations, PCIe 6.0 doubles the bandwidth of PCIe 5.0 to a 64 GT/s, delivering up to 256 GB/s bidirectionally on a x16 slot.
Examples of Available Products
As the volume of data required for AI training and inference continues to grow rapidly, enterprise SSDs capable of delivering data quickly and reliably are becoming increasingly essential to AI infrastructure.
To that end, Samsung has begun mass production of its PM1763 SSD the company’s PCIe 6.0-based enterprise solid state drive optimized for next-generation AI and HPC server environments.
PM1763 is designed for liquid-cooled server environments through direct-to-chip cooling technology. This enables sustained peak performance even under intensive workloads and extended operating conditions.
The Micron 9650 SSD is said to be the first PCIe Gen6 SSD for the data center. It is also said to be the fastest in the industry, demonstrating up to 28GB/s of sequential read bandwidth.
AI workloads often achieve less than 50% of peak GPU performance due to data bottlenecks. With up to 28 GB/s sequential read and 5.5 MIOPS random read performance, the Micron 9650 SSD aims to dramatically accelerate AI training and inference.
It also offers a liquid cooling option that integrates with top-tier liquid cooling solutions to help enhance the overall thermal efficiency of systems and data centers. Its vertically integrated architecture includes Micron G9 NAND – said to be the industry’s first 9th-generation NAND used in a data center SSD. The Micron 9650 is available in E1.S and E3.S form factors with E1.S available for liquid cooling.
What’s Next?
A year ago, PCI-SIG announced the official release of the PCI Express 7.0 specification, reaching 128.0 GT/s. The PCIe 7.0 specification targets data-driven applications like AI/ML, 800G Ethernet, cloud and Quantum computing. Pathfinding for the PCIe 8.0 specification is already in progress to continue supporting the industry’s investments and product roadmaps in the PCIe technology ecosystem.
PCI-SIG also recently announced that the PCI Express 8.0 specification, planned for release by 2028.will reach 256.0 GT/s.
Follow TTI, Inc. on LinkedIn for more news and market insights.
Statements of fact and opinions expressed in posts by contributors are the responsibility of the authors alone and do not imply an opinion of the officers or the representatives of TTI, Inc. or the TTI Family of Specialists.
Follow TTI, Inc. - Europe on LinkedIn for more news and market insights.
Statements of fact and opinions expressed in posts by contributors are the responsibility of the authors alone and do not imply an opinion of the officers or the representatives of TTI, Inc. or the TTI Family of Specialists.
