Colocation Data Centers Structured Cabling Trends

By Tom Debiec and Lisa Huff

As companies look to embrace cloud computing or backup their existing data centers, many are evaluating colocation as an option.  With this increased demand, colocation data centers are popping up all over the world and becoming a larger part of the overall data center market.  In 2013, the colocation sector is expected to account for about 25-percent of the structured-cabling data center market. During the work to develop the structured cabling forecast for the soon to be released Bishop & Associates report, "Structured Cabling Technology and Market Assessment," we had the chance to talk to project managers that are responsible for implementing 10,000 - 20,000 sqft build-outs in colocation facilities. It was clear to us that a few key trends have emerged:

• Whenever possible contractors recommend the use of pre-terminated copper and fiber cabling.  The benefits of utilizing these components include cost reduction, on-time delivery and the project is easier to manage.
• Although they are installing MPO cassettes on some jobs the cost premium often scares customers away.
• More OM3 fiber is being installed than OM4.  The up-sell to OM4 is difficult since OM3 covers the distances that are typically seen in these facilities (300m at 10G).
• A majority of the copper cabling is being installed is Category 6. 

A typical colocation lease averages about eight years.  Since the clients don't know what their requirements will be in this timeframe, they are less likely to make decisions that "future-proof" the installation for reuse with upgraded active equipment.  Tight budgets further preclude the addition of higher performing cabling products.  They would prefer to re-cable in the future than to pay for it now.  Since many of these installations are based on Top-of-Rack (ToR) architecture, re-cabling is viewed as a much simpler thing to do than to install new equipment when the cabinets are stuffed full of cabling. In view of this, we project that Category 6A and Category 7 cabling will only see very slow growth and that OM3 will be the mainstay over the next few years.

Next Generation 100G Ethernet (with corrections)

Ever since the 40/100G Ethernet standard was completed in 2010, the IEEE standards group has been working on ways to improve it. In my opinion, there were two very serious holes in the original standard. The 40G long-reach variant did not match the existing telecom standard for 40G so some type of conversion equipment would be needed. This was fixed when the IEEE 802.3bg 40GBASE-FR single mode fiber standard was released in 2010. The second concern, which still exists, is the 100GBASE variants. Four standardized and one MSA currently exist and are shown in the following table.

Ethernet Variant	Data rate (Gbps)	Min. Reach (meters)	Form Factors	Media Wavelength	Standard IEEE 802.3
100GBASE-
CR10	10x10	7	CXP Direct Attach Copper	Twinax Copper	ba
SR10		100/150	CXP, CFP	LOMF 850nm
LR4	4x25	10,000	QSFP+, CFP	SMF 1310nm
ER4		40,000	CFP	SMF 1310nm
LR10	10x10	2,000	CFP	SMF 1550nm	Not supported

40/100G, the IEEE did not want to make the mistake of too many variants and form factors again (like they did for 10G) so consciously limited them. But, in our opinion, may have restricted them too much. By reducing the laser-optimized multi-mode fiber (LOMF) optical reach to 100m over OM3 and 150m for OM4, the IEEE left a huge gap in distance covered for data center applications – in fact, a two orders of magnitude gap – from 100m to 10km. This results in an enormous difference in cost as well. For example, a 100GBASE-SR10 CXP module average selling price is about $200, while the 100GBASE-LR4 average price is more than $20,000. So it is currently cost-prohibitive to design a data center with connections longer than 100m. This is not realistic. In order to address this shortcoming, the top transceiver manufacturers are working on SR4 products that have the potential to reach to 300m. Recently, the IEEE has recognized this issue and is looking to address it in its next generation study group. It is called the Next Generation 100Gb/s Optical Ethernet Study Group and its charter is to investigate 25G-per-lane standards and to explore lower-cost solutions to cover reaches perhaps up to a kilometer.

Any 100G variant using 25G signaling is still under development. While the optical devices are almost ready to go, there are long-term projects to ascertain how 25G is going to run on a printed-circuit board (PCBs) or on twinax cable. The group that was studying this has just officially been named a task force in the IEEE – the P802.3bj 100 Gb/s Backplane and Copper Cable Task Force. There are chip sets available to run 25G signals over PCBs that will be available in the coming months. Texas Instruments was demonstrating this at SC11 and Altera, Amphenol, Semtech/Gennum, IBM, Inphi, TE Connectivity and Xilinx showed 25G products in the OIF booth at OFC/NFOEC 2012.

Notice in the table above that there are different signaling schemes and form factors between 100GBASE-CR10, SR10 and 100GBASE-LR4. The CXP that was chosen for short-reach copper and LOMF is not suitable for longer-reach SMF operation. Even though most of them were involved in the IEEE process, equipment manufacturers are not happy about this because that means their products must support two different form factors at the same time. It may also doom CXP to only the initial products until another, better form factor can be developed that will cover both cost effectively – maybe a CFP2 or CFP4? Or the 25G signaling matures and the SR4 and CR4 variants are created in the QSFP28 (now being worked on in the SFF committee) is used.

The LR10 variant is not standardized, but is backed by a consortium of vendors and end users – including Google and Facebook. Whether this will take hold in the industry at large remains to be seen, but some of the industry leaders are boasting that it is actually selling very well currently at more than 2,500 units already.

So, while we talk about Terabit Ethernet being on the horizon and there have been multi-vendor demonstrations of 25G signaling for 100G operation, plenty of work remains to bring 100G to fruition prior to the next speed bump.