OFC/NFOEC Data Center Cabling Short Course (SC358)

I'm headed to LA next week for OFC/NFOEC and my first order of business will be to teach a short course on transitioning your data center from copper to fiber. Here's a short description of what I'll discuss:

This short course is intended to help data center and network managers understand the value proposition of an all-optical data center. Topics that will be discussed include looking at basic data center network infrastructure design and hardware needs.  The course was developed with the TIA-942, Telecommunications Infrastructure Standard for Data Centers in mind, but will also address the fact that many data centers really have not used this standard in practice. Networking standards such as IEEE 802.3, Fibre Channel, iSCSI and InfinBand will be reviewed along with their applicability to certain aspects of the data center. Technology roadmaps and data center networking trends will be included as well as how to handle transitioning from lower to higher data rates within your data center. When it makes sense to implement fiber optics and what types of transceivers and cabling should be used for different scenarios will be presented. Other technologies such as Fibre Channel over Ethernet, RDMA (InfiniBand) over Converged Ethernet, IO virtualization and how virtualization and network consolidation will affect data rates will also be discussed. Detailed cost analysis of fiber versus copper in the data center will be presented considering not only equipment and infrastructure cost, but port and cabling density, power and cooling costs.  Also included will be analysis of whether it matters what vertical market a data center supports – for example, does a financial sector data center have different requirements than a higher-education data center? Several real-world case studies will be presented.

Hope to see you there.

The Ethernet Technology Summit and OFC/NFOEC 2011

The next few weeks I’ll be very busy. First I’ll be chairing Session 105 (Ethernet Chipsets/Components) at the Ethernet Technology Summit. While most of the conference is about R&D projects, I’ve chosen to focus my short presentation on current opportunities for components suppliers. You'll be able to download the presentation from my Web site soon.

In March, I’ll be heading to OFC/NFOEC 2011. There I have three different sessions I’m involved in. My short course on “Data Center Cabling – Transitioning from Copper to Fiber” will be held Monday morning. A short excerpt of this short course can be accessed through an archived Webinar I did recently. Monday afternoon I’ll be participating in the Computercom Symposium by presenting “The State of the Short-Reach Optics Market.” On Tuesday, the kick-off of the Optical Business Forum (see previous post for details) will take place where I’ll be moderating the Carrier Ethernet Exchanges session.

Hope to see you in Santa Clara this week or Los Angeles the week of March 7th.

Bringing the Optical Networking Supply Chain Back Together: The Optical Business Forum at OFC/NFOEC

As the networking equipment supply chain has broken up into intellectual property suppliers, semiconductor suppliers, optical component manufacturers, and full system manufacturers, telecom carriers are often 2-4 steps away from the technologies that can alter the economics of running their networks. This was not the case ten years ago during OFC's peak attendance years when many equipment vendors still owned components divisions. Yet as the large carriers turn to business accounts to cover the costs of expanding their optical networks and increasingly depend on technologies from their suppliers' supplier's suppliers, there has been no central trade event where they can gather to meet with this expanding set of companies entering their networks, or to examine the latest advances in optical technologies. OFC/NFOEC has seized the opportunity to bring the telecom supply chain back together to serve the community with much greater depth than any other show. It now has The Optical Business Forum.

Major telecom providers are in significant competition with each other for large business accounts. With wireless taking up a greater share of telephony revenue, and households having limited budgets to spend on service regardless of bandwidth offered, service providers see their future lying with the business customer. While they do not publicize much of their activity with commercial accounts due to regulatory pressure, businesses are a major factor behind carriers expanding the capacity of their optical networks.

Verizon's sale of its northern New England and West Virginia lines was carried out because of the limited base of commercial accounts in those regions. AT&T built nine 40-Gigabit MPLS nodes outside of its incumbent territory, where it does not offer U-Verse or any video service, in order to increase capacity for its corporate data and IP/VPN customers. It has kept quiet in the press regarding this upgrade, in spite of tens of millions invested. In an addition to corporations continuing to make heavy investments in bandwidth, carrier IP cores are carrying more traffic, with Petabytes transferred still growing 30 to 40% a year. And in order to maintain their dividends, as well as their returns on capital investments, carriers need revenue from the high margin, high budget business customer.

The Optical Business Forum was developed by analysts associated with DataCenterStocks.com. It is the first forum in what we’re hoping to be a yearly affair covering the most important topics in the area of optical transport for businesses data communications. This year’s summit is on the exhibit floor and consists of a key note address on High-Bandwidth Ethernet Services given by Rajiv Datta, Senior Vice President and Chief Technology Officer of AboveNet Inc. Following the keynote are three focused sessions:

1. Who is Buying Optical Bandwidth Services?
2. The Economics & Business Case for Connecting Data Centers
3. Carrier Ethernet Exchanges

The program includes speakers from AboveNet, Allied Fiber, CENX, Equinix, Juniper Networks, Packet Exchange, Telx Ethernet Exchange, Verizon Digital Media Services, XO Communications, Zayo Fiber Solutions and zColo. Moderators include Craig Clausen from NPRG, Michael Howard from Infonetics and myself.

Join us at OFC/NFOEC and hear what you’re customers’ customers are saying.

Optical Interconnection Players Strengthening Their Businesses

Molex just purchased Luxtera’s AOC business completing the circle that all the other optical interconnect players started. During the telecom bust in the early 2000’s, Amphenol, FCI, Molex and Tyco Electronics all either de-emphasized their optical interconnect businesses or exited them all together. Now, they have all re-entered. Why?

While they are all working on more high-speed copper solutions like the one Tyco showed for 25G and beyond at SC10, I beleive they also see the writing on the wall. While they won’t admit it, I think they know that beyond 100G copper cable interconnects may have FINALLY reached the end of their useful life. At 40G and 100G, for example, there is still no twisted-pair solution and the direct-attach copper can only reach about 7m reliably.

It has been interesting watching the choices these traditional connector companies have made:

• Amphenol: It never exited the optical interconnect business, but left the transceiver products to Avago, Finisar, JDSU and others until recently. It has a stronghold on the short-reach copper direct-attach market so has inroads at customers for its AOCs and modules.
• FCI: Exited the optics business entirely for a few years but then started again from scratch and subsequently purchased MergeOptics in February 2010. MergeOptics is what was left of Infineon Technologies and still has strong technical abilities in short-reach products. It also has the building blocks to provide all-optical interconnects all the way from the chip (see my previous posts on MergeOptics). They can provide both AOCs and transceiver modules so have the ability to cover all high-speed markets in InfiniBand, Ethernet and Fibre Channel.
• Molex: Purchased Luxtera’s AOC business recently. So while FCI and Tyco are stressing short-wavelength technologies, Molex has turned to custom long-wavelength ones. Luxtera’s technology is based on 1490nm devices, which really doesn’t matter if you’re purchasing an AOC, but will matter if you want transceiver modules. According to company representatives, they will eventually get back into supplying transceiver modules, but there has been no evidence of this as of yet. Perhaps the possession of Luxtera AOCs will prompt this.
• Tyco Electronics: Tyco exited the transceiver business in the early 2000’s, but still had a very active fiber-optic interconnect business – especially for premise wiring (AMP NETCONNECT). It acquired Zarlink Semiconductor’s optical products group in May 2010. Zarlink is on the forefront of parallel-optics technology and was one of the first to introduce AOCs. It does not appear that Tyco intends to supply optical transceiver modules again.

I would never bet against copper re-inventing itself in order to meet the demands of future high-speed networks, but with optical 10G dominating the market currently and 40/100G optical products starting to emerge, it will be an uphill battle for copper solutions to gain traction. And beyond 100G, all bets are off. I’m thinking that these companies are reaching the same conclusions and that if they don’t add optical capabilities soon, they may render themselves obsolete within the next ten years or so. That's not to say that there won't be a vibrant businesses in both copper structured cabling and interconnects over the next ten years - there will be. But I think that R&D dollars will be better spent on optical interconnect technologies rather than trying to figure out how to run 25G signals using copper interconnects (including backplanes.) Or how to convince end-user customers in the US that a shielded structured cabling solution for 40G is better than a short-reach optical one because it will be cheaper - but at what cost to power, cooling and space?

What do you think? I'd love to hear your thoughts.

The 10X10 MSA: Niche, Distraction or the Right Answer? (Continued)

While Vipul has a point that this new MSA is probably a distraction, it is difficult to deny that there is a market for cost-effective devices with optical reaches between 100m and 10km. In fact, 100m to 300m is the market that multi-mode fiber has served so well for the last 20 years. And, 300m to 2km has been a niche for lower-cost 1310nm single mode products like 1000BASE-LX. So I have a slightly different opinion about this 10x10 MSA and whether it’s a niche, distraction or the right answer.

In a recent article written on Optical Reflection, Pauline Rigby quotes Google’s senior network architect, Bikash Koley. About 100GBASE-SR10, he says 100m isn’t long enough for Google – that it won’t even cover room-to-room connections and that “ribbon fibres are hard to deploy, hard to manage, hard to terminate and hard to connect. We don’t like them.” There is an answer for this ribbon-fiber problem – don’t use it. There are many optical fiber manufacturers that now provide round multi-fiber cables that are only “ribbonized” at the ends for use with the 12-position MPO connector and are much easier to install – Berk-Tek, A Nexans Company, AFL and even Corning have released products that address this concern. But, the 100m optical reach is another matter.

I have to agree with Google about one other thing – 4x25G QSFP+ solutions are at least four years away from reality (and I would say probably even longer). This solution will eventually have the low-cost, low-power and high-density Google requires, but not quick enough. I think something needs to be done to address Google’s and others requirements between 300m and 2km in the short term, but I also believe that it needs to be standardized. There is no IEEE variant that would currently cover a 10x10G single mode device. However, there is an effort currently going on in the IEEE for 40G over SMF up to 2km. Perhaps the members of the MSA should look to work with this group to expand its work or start a new related project to cover 100G for 2km as well? I know this was thrown out of the IEEE before, but so were 1000BASE-T and 10GBASE-T initially.

So what I'm saying is that the market is more than a niche - hundreds of millions of dollars of LOMF sales at 1G and 10G would attest to that. And it's more than a distraction because there is a need. But I don't think it's entirely the right answer without an IEEE variant to back it up.

Let us know what you think.

The 10X10 MSA: Niche, Distraction or the Right Answer?

{For today’s blog, our guest author is Vipul Bhatt. I have known Vipul for several years, since when he was the Director of High Speed Optical Subsystems at Finisar. He has served as the Chair of Optical PMD Subgroup of IEEE 802.3ah Ethernet in the First Mile (EFM), and the Chair of Equalization Ad Hoc of IEEE 802.3ae 10G Ethernet. He can be reached at vjb@SignalOptics.com.}

Last week, Google, JDSU, Brocade and Santur Corp announced the 10X10 Multi-Source Agreement (MSA) to establish sources of 100G transceivers. It will have 10 optical lanes of 10G each. Their focus is on using single mode fiber to achieve a link length of up to 2 km. The key idea is that a transceiver based on 10 lanes of 10G will have lower power consumption and cost because it doesn’t need the 10:4 gearbox and 25G components. But is this a good idea? What is the tradeoff? Based on my conversations with colleagues in the industry, it seems there are three different opinions emerging about how this will play out. I will label them as niche, distraction, or the right answer. Here is a paraphrasing of those three opinions.

It’s a niche: It’s a solution optimized for giant data centers – we’re talking about a minority of data centers (a) that are [already] rich in single mode fiber, (b) where the 100-meter reach of multi-mode 100GBASE-SR10 is inadequate, and (c) where the need for enormous bandwidth is so urgent that the density of 10G ports is not enough, and 100G ports can be consumed in respectable quantities in 2011.

It’s a distraction: Why create another MSA that is less comprehensive in scope than CFP, when the CFP has sufficient support and momentum already? Ethernet addresses various needs – large campuses, metro links, etc. – with specifications like the LR4 that need to support link lengths of well beyond 2 km over one pair of fiber. We [do] need an MSA that implements LR4, and the SR10 meets the needs of a vast majority of data centers, so why not go with CFP that can implement both LR4 and SR10? As for reducing power consumption and cost, the CFP folks are already working on it. And it’s not like we don’t have time – the 10G volume curve hasn’t peaked yet, and may not even peak in 2011. Question: What is the surest way to slow down the decisions of Ethernet switch vendors? Answer: Have one MSA too many.

It’s the right answer: What is the point of having a standard if we can’t implement it for two years? The CFP just isn’t at the right price-performance point today. The 10X10 MSA can be the “here and now” solution because it will be built with 10G components that have already traversed the experience curve. It can be built with power, density and cost figures that will excite the switch vendors, which may accelerate the adoption of 100G Ethernet, not distract it. As for 1-pair vs. 10-pairs of fiber, the first swelling of 100G demand will be in data centers where it’s easier to lay more fiber, if there isn’t plenty installed already. The 2-km length is sufficient to serve small campuses and large urban buildings as well.

Okay, so what do I think? I think the distraction argument is the most persuasive. An implementation that is neither SR10-compliant nor LR4-compliant is going to have a tough time winning the commitment of Ethernet switch vendors, even if it’s cheaper and cooler than the CFP in the short term.

SFP+ - The New Optical RJ45?

For those of you that have been in the industry for what seems to be 100 years, but is really about 25 years, you know that the one “connector” that hasn’t changed much is the RJ45. While there have been improvements by adding compensation for the error that was made way back when AT&T developed the wiring pattern (splitting the pair causing major crosstalk issues), the connector itself has remained intact. Contrastingly, optical connectors for datacom applications have changed several times – ST to SC to MT-RJ to LC. They have finally seemed to settle on the LC and perhaps on a transceiver form factor – the SFP+. The SFP was originally introduced at 1G, was used for 2G and 4G and with slight improvements has become the SFP+ and the dominant form factor now used for 10G. Well, it is in the process of getting some slight improvements again and promises to make it all the way to 32G. That’s six generations of data rates – pretty impressive. But how?

The INCITS T11.2 Committee's Fibre Channel Physical Layer – 5 (FC-PI-5) standard was ratified in September. It specifies 16G Fibre Channel. Meanwhile, the top transceiver manufacturers have been demonstrating pre-standard 16G SFP+ SW devices. But, wait a minute – short-wavelength VCSELs were supposed to be very unstable when trying to modulate them at data rates above 10G right? Well, it seems that at least Avago and Finisar have figured this out. New microcontrollers and adding at least one clock and data recovery (CDR) device in the module to help clean up the signals have proven to be keys. Both vendors believe it is possible to do this and not add too much cost to the modules. In fact, both also think that possibly by adding electronic dispersion compensation (EDC) they can push the SFP+ to 32G as well - which is the next step for Fibre Channel - hoping to stop at 20G and 25G to cover developments in Ethernet and InfiniBand.

And what about long wavelength devices? It has always been a challenge fitting the components needed to drive long distances into such a small package mainly because the lasers need to be cooled. But not anymore – Opnext has figured it out. In fact, it was showing its 10km 16G FC SFP+ devices long before any of the SW ones were out (March 2010). Of course, this isn't surprising considering Opnext has already figured out 100G long haul as well.

These developments are important to datacom optical networking for a few of reasons:  

1. They show that Fibre Channel is not dead.
2. The optical connector and form factor "wars" have seemed to subsided so transceiver manufacturers and optical components vendors can focus on cooperation instead of positioning.
3. They will impact the path other networking technologies are taking – Ethernet and InfiniBand are using parallel optics for speeds above 10G – will they switch back to serial?

Stay tuned for more on these points later.