Next Generation 100G Ethernet (with corrections)

Ever since the 40/100G Ethernet standard was completed in 2010, the IEEE standards group has been working on ways to improve it. In my opinion, there were two very serious holes in the original standard. The 40G long-reach variant did not match the existing telecom standard for 40G so some type of conversion equipment would be needed. This was fixed when the IEEE 802.3bg 40GBASE-FR single mode fiber standard was released in 2010. The second concern, which still exists, is the 100GBASE variants. Four standardized and one MSA currently exist and are shown in the following table.

Ethernet Variant	Data rate (Gbps)	Min. Reach (meters)	Form Factors	Media Wavelength	Standard IEEE 802.3
100GBASE-
CR10	10x10	7	CXP Direct Attach Copper	Twinax Copper	ba
SR10		100/150	CXP, CFP	LOMF 850nm
LR4	4x25	10,000	QSFP+, CFP	SMF 1310nm
ER4		40,000	CFP	SMF 1310nm
LR10	10x10	2,000	CFP	SMF 1550nm	Not supported

40/100G, the IEEE did not want to make the mistake of too many variants and form factors again (like they did for 10G) so consciously limited them. But, in our opinion, may have restricted them too much. By reducing the laser-optimized multi-mode fiber (LOMF) optical reach to 100m over OM3 and 150m for OM4, the IEEE left a huge gap in distance covered for data center applications – in fact, a two orders of magnitude gap – from 100m to 10km. This results in an enormous difference in cost as well. For example, a 100GBASE-SR10 CXP module average selling price is about $200, while the 100GBASE-LR4 average price is more than $20,000. So it is currently cost-prohibitive to design a data center with connections longer than 100m. This is not realistic. In order to address this shortcoming, the top transceiver manufacturers are working on SR4 products that have the potential to reach to 300m. Recently, the IEEE has recognized this issue and is looking to address it in its next generation study group. It is called the Next Generation 100Gb/s Optical Ethernet Study Group and its charter is to investigate 25G-per-lane standards and to explore lower-cost solutions to cover reaches perhaps up to a kilometer.

Any 100G variant using 25G signaling is still under development. While the optical devices are almost ready to go, there are long-term projects to ascertain how 25G is going to run on a printed-circuit board (PCBs) or on twinax cable. The group that was studying this has just officially been named a task force in the IEEE – the P802.3bj 100 Gb/s Backplane and Copper Cable Task Force. There are chip sets available to run 25G signals over PCBs that will be available in the coming months. Texas Instruments was demonstrating this at SC11 and Altera, Amphenol, Semtech/Gennum, IBM, Inphi, TE Connectivity and Xilinx showed 25G products in the OIF booth at OFC/NFOEC 2012.

Notice in the table above that there are different signaling schemes and form factors between 100GBASE-CR10, SR10 and 100GBASE-LR4. The CXP that was chosen for short-reach copper and LOMF is not suitable for longer-reach SMF operation. Even though most of them were involved in the IEEE process, equipment manufacturers are not happy about this because that means their products must support two different form factors at the same time. It may also doom CXP to only the initial products until another, better form factor can be developed that will cover both cost effectively – maybe a CFP2 or CFP4? Or the 25G signaling matures and the SR4 and CR4 variants are created in the QSFP28 (now being worked on in the SFF committee) is used.

The LR10 variant is not standardized, but is backed by a consortium of vendors and end users – including Google and Facebook. Whether this will take hold in the industry at large remains to be seen, but some of the industry leaders are boasting that it is actually selling very well currently at more than 2,500 units already.

So, while we talk about Terabit Ethernet being on the horizon and there have been multi-vendor demonstrations of 25G signaling for 100G operation, plenty of work remains to bring 100G to fruition prior to the next speed bump.

Common Electrical Interface for 25/28G – a Possibility or a Pipe-Dream?

Yesterday, I sat through a workshop hosted by the Optical Interconnecting Forum (OIF) on its “Common Electrical Interface (CEI) project for 28G Very Short Reach (VSR).” What quickly became clear to me was that I was in a room of VERY optimistic engineers.

I sat through presentations that were characterized as “Needs of the Industry,” which consisted of content from the leaders of IEEE 802.3ba (40/100G Ethernet), T11.2 (Fibre Channel) and InfiniBand standards groups. Yet all of these representatives made sure they carefully stated that what they were presenting was their own opinions and not that of their standards groups, which I found odd since most of what they showed was directly from the standards. Legalities I guess. I also noticed that they never really sited any kind of independent market research or analysis of what the “needs of the industry” were. For instance, one speaker said that InfiniBand has a need for 26AWG, 3-meter copper cable assemblies for 4x25G operation in order to keep the cost down within a cabinet. Yet, he did not present any data or even mention what customers are asking for this. Maybe this exists, but to me unless it is presented, the case for it is weak. I do have evidence directly from some clustering folks that they are moving away from copper in favor of fiber for many reasons – lower power consumption, weight of cabling, higher density, and room in cabinets, pathways and spaces.

Today, data center managers are really still just starting to realize the benefits of deploying 10G, which has yet to reach its market potential. I understand that standards groups must work on future data rates ahead of broad market demands, but this seems extremely premature. None of the current implementations for 40/100G that use 10G electrical signaling have even been deployed yet (except for maybe a few InfiniBand ones). And, from what I understand from at least one chip manufacturer who sells a lot of 10G repeaters to OEMs for their backplanes, it is difficult enough to push 10G across a backplane or PCB. Why wouldn’t the backplane and PCB experts solve this issue that is here today before they move onto trying to solve a “problem” that doesn’t even exist yet?

Maybe they need to revisit optical backplanes for 25G? It seems to me that 25G really won't be needed any time soon and that their time would be better spent on developing something that would appear to have relevancy beyond 25G. To me, designing some exotic DSP chip that would allow 25G signals to be transmitted over four-to-12 inches of PCB and maybe 3m of copper cable for one generation of equipment may not be very productive. Maybe this is simpler than I anticipate, but then again, there was a similar but a little more complicated problem with 10GBASE-T and look how that turned out...