Connection-First Design for Today’s Complex Data Centers
4.13.2022 // Vivek Shah, Molex
To understand just how essential connectivity is to today’s data centers, we need to step back and take a holistic view of the challenge – and not only from the standpoint of the technology involved. Connectivity must also play a role in the design of a data center from the very beginning of that process.
With the expectation of reliable, consistent delivery of both rapid speeds and high performance for the largest hyperscalers in the world, now more than ever it is critical to explore the role that connectivity plays in delivering both speed and power. At its core, a data center exists to store and manage the critical resources that are vital to an organization’s continued operation. For this reason, reliability, efficiency and security are typically the top priorities in data center design.
Yet sometimes, in the rush to design completion, the form and function of the data center’s entire connectivity setup – including the power supply – can be overlooked. This mistake may only become clear at a later time when those design features become restrictive factors that limit solutions to issues as they occur.
For those who design and deploy high-scale data centers, the reality is this: connectivity can no longer be an afterthought in data center design. Times have changed, and the rapidly evolving field of data communications has experienced continuous and insistent demand for increased throughput and bandwidth.
Moore’s Law and Data Centers
For a different perspective on this problem, let’s look at technology over a large span of time – 40 to 50 years – and consider how Moore’s Law has slowed over that timeframe. The original observation by Moore that the number of transistors on a chip would double every two years now has to be adjusted, as geometries at the chip level encounter the challenges of architectures at 7 nanometers and below.
Essentially, the industry’s response has been to multiply the systems involved, while keeping Moore’s Law alive by interconnecting these systems and recreating the performance levels that would normally be expected from single-chip implementations.
As a result, connectivity plays an even more instrumental role in data center design. For example, a clever interconnect design could mean that both short reach and longer reach can be achieved with the same system design, whether it’s connecting within the rack with passive DAC or connecting multiple racks with active electrical cable or optics.
The Critical Factor: Redundancy
It’s widely understood that if one system goes down, the continued function of the series requires redundancy based on multiple cards or related systems – for example, a backplane connector would need to have multiple line cards plugged into an A1 Switch card.
From an external I/O perspective, in an application where two top-of-the-rack switches are being connected to a single server using a dual top-of-rack (TOR) cable, additional functionality is required in order to maintain redundancy. And it is becoming more and more critical to these applications for designers to consider these connections from the very beginning.
Major Market Drivers
What’s driving the enormous growth we have seen in data centers and in their functionality? Demand for artificial intelligence (AI) applications is high on the list, as these are now considered the “nervous system” of connected technologies, automatically controlling other apps and equipment. In addition, 5G wireless technology maintains access to high-speed data all the way to the edge, or to the end user’s specific location.
Yet in the design of these complex systems running advanced technologies, connectivity is rarely prioritized from the very start, likely due to the difficulty in quantifying the pain points of doing so and the cost-effectiveness of implementing these interconnections – and cost is always a key factor.
Further, ignoring design issues at the start invites latency. Properly interconnecting all of these elements underscores the need to clearly map how data flows optimally through multiple boxes and systems – all to ensure the lowest possible latency. It can be argued retention of signal integrity determines the maximum bandwidth allowed. In the battle to eliminate latency, connectivity is table stakes.
Diversity of Applications
The diversity of applications – from the unique devices being supported, interconnects available and the wide range of choices facing the end user – also ranks as a critical consideration in data center design. For example, should a passive DAC cable or an active cable be used? Depending upon the answer to this question, one might choose a simple linear amplifier, a retimer-based design or an optical solution.
These are the key considerations on the external I/O side, but there are considerations on the internal side as well, such as whether to opt for expensive PCB material in lieu of a less expensive alternative.
Total Cost of Ownership and Value Insights
This raises one of the most compelling questions: the total cost of ownership, encompassing what’s inside and outside of the box. It is critical to look at the end-to-end solution and the total cost and start the discussion there.
Our approach to analysis and modeling has enabled Molex to make effective recommendations to our customers, confident that this in-depth analysis provides effective reasoning in support of certain products or solutions.
This level of insight is made possible through the breadth of our portfolio and our participation in the Open19 Project, which aims to establish a new open standard for data center servers. By combining core technologies with collaboration with industry-leading tech companies, we are proud to offer our customers a portfolio built over decades, which means we can guide customers in their choices, pointing out the best combinations of products for the best data center design solution. This includes a full end-to-end high-performance cabling with matched impedance system, as opposed to just providing an interconnect schema, for example.
By working with customers collaboratively to overcome challenges, rather than simply offering single-point products, Molex has created a point of differentiation. We see this at play so often at sites around the world, where unplanned expansion has led to real challenges with cable routing, bundling over obstacles and physical impairments – all with a huge knock-on effect on airflow that can cause impacts all the way down to the board level.
For these reasons, designing in connectivity from the outset of a data center project is fundamental to success later on. It enables customers to avoid rigid choices that require rework, and considerable amounts of both time and money to rectify challenges when the wrong choices are made. Connection-first planning enables data center design success from the word “go.”
