High Speed Interconnects: Navigating Your High-Speed Options


In most markets, Category 6A/ Class EA copper twisted-pair cabling has dominated in the data centre for 10 gigabit Ethernet links. Now as the demand for high-speed, low latency access to information continues to grow, server interconnect speeds are increasing beyond 10 Gig and the feasibility of twisted-pair copper cabling. That means network professionals like yourself are faced with choosing between Direct Attach Copper Cables (DAC), Active Optical Cables (AOC), or structured cabling using fibre assemblies and separate optical transceivers. Understanding the pros, cons, and capabilities of each of these options can help you make an informed decision and achieve the best outcome for your specific application and environment.
 

DACs

 
Factory terminated, fixed length DACs made from twinax copper with small form factor pluggable modules provide a cost effective, low latency direct connection between active equipment transceiver ports. The modules come in various single or multi lane technologies to support from 10 to 400 Gig. The most common DACs used in data centre switch-to-server links include:

10 Gig SFP+ single-lane
25 Gig SFP28 single-lane
40 Gig QSFP+ four-lane
100 Gig QSFP28 four-lane

It’s important to note that SFP28 DACs share the same mating interface as SFP+ DACs, and QSFP28 DACs share the same mating interface as QSFP+ DACs, making them backwards compatible to ease migration. To help you reduce cost via lower switch port counts and space savings, these DACs also come in hybrid breakout assemblies, including QSFP28 to 4 x SFP28 for 4 x 25 Gig applications and QSFP+ to 4 SFP+ for 4 x 10 Gig applications.

While DACs offer a cost effective and efficient way to connect equipment, they are limited in length. Passive DACs are limited to about 5 metre lengths, while active DACs that contain electronics for signal conditioning can reach to 10 metres. The distance limitations of DACs make them suited for a Top-of-Rack (ToR) data centre topology.
 

AOCs

 
AOCs are similar to DACs in that they are factory terminated and provide an easy, high performance direct connection between equipment. However, they leverage the distance capabilities of fibre optic cable and embedded transceivers to form a complete cable assembly that can support up to 100-metre lengths. Like DACs, they are available in SFP+, SFP28, QSFP+ or QSFP28 form factors for 10 to 100 Gig applications, as well as hybrid breakout assemblies for cost effective 4 x 25 Gig and 4 x 10 Gig deployments. The greater distance capability means that AOCs can be used in ToR, End of Row (EoR) and Middle of Row (MoR) topologies.

Considered a cost effective alternative to structured cabling and optical transceivers due to their lower material cost and power consumption, AOCs also benefit from the smaller diametre of fibre that allows for tighter bend radius, easy installation and increased airflow for cooling.
 

Fibre Cabling with Transceivers

 
The third option for high speed data centre links is to use two optical transceivers and connectorised fibre, either factory or field terminated to duplex or multi-fibre MPO connectors. Fibre cabling with transceivers can support much greater distances of up to 10 kilometers depending on the application and fibre type. That makes them suitable for virtually any architecture (e.g., 3-tier, full mesh, leaf-spine) and topology (e.g., ToR, EoR, MoR) including large hyperscale data centres where switches may be centrally located well over 100 metres away from servers.

Fibre cabling with transceivers also supports the use of standards based structured cabling that offers the most flexible deployment and the use of cross connects or interconnects for superior manageability. This can be useful in multi tenant colocation data centres where connections between customer and service provider equipment needs to be accomplished outside of the meet me room or customer space. While implementing cross connects or interconnects adds additional points of failure, reliability is improved by ensuring that quality fibre cable and connectivity (including jumpers) are purchased from reputable manufacturers with third party verified standards based performance.

The other main benefit of using fibre cabling with transceivers is that the fibre can support multiple generations of applications and does not need to be replaced—only the transceivers need to be replaced. Multimode fibre currently supports 10 to 400 Gig, while Singlemode will support up to 800 Gig and beyond.

While the cabling lasts generations, the use of structured cabling with transceivers can ultimately be as high as 74% more expensive than DACs and AOCs due to several factors. First, the material cost of cabling and separate transceivers is higher than DACs and AOCs. Second, the power consumption is higher per port on the equipment side. Initial installation is also more complex and can cost up to three times more due to more components and the need to ensure proper polarity as explained here in one or our previous blogs.
 

How Do You Choose?


When it comes to selecting an option for high speed data centre links, first and foremost you need to know your application, architecture and topology to understand your speed and distance requirements. For example, if you’re deploying an EoR or MoR topology with link distances greater than 7 to 10 meters, that pretty much rules out DACs. You also need to consider your plans for the future. If you plan to eventually migrate to a 200 Gig application like 200BASE-SR4, Multimode fibre with MPO connectivity deployed today for 100 Gig will support it.


Additional Considerations

 
Once you’ve made your choice, there are other considerations beyond cost that might influence where you purchase your components, including interoperability and compliance, quality and breadth of the product line. Because not all cables are created equal.

DACs and AOCs are often offered by switch vendors and can contain an EPROM, which contains information such as the length of the cable, data rate supported and vendor identity data. This allows some switches to check vendor-security IDs on DACs and either display errors or prevent ports from functioning when connected to an unsupported vendor ID. Siemon DACs have been specifically programmed to overcome this issue, avoiding the potential for vendor lock-in and maximising interoperability. They have been third-party tested for interoperability and compliance with IEEE application standards, including Cisco. It’s also wise to ensure that AOC transceivers (or any transceiver for that matter) are multi-source agreement (MSA) compliant for compatibility across equipment from various vendors.

When purchasing DACs and AOCs from switch vendors, also beware that you may find yourself with limited length and colour options, as well as a lack of warranty and service options. Purchasing DACs and AOCs from a reputable cable manufacture like Siemon means more choices and superior technical support and warranty protection. Siemon offers DACs and AOCs in half metre lengths and multiple colours to support a wider range of deployments and colour-coding, as well as a complete offering of high-performance Multimode and Singlemode structured cabling.

The Siemon range of products are available from CMW.