- Fabrics are faster
- Fabrics enable large-flat layer 2 broadcast domains
- Fabrics deliver a single point of management
- Fabrics reduce network tiers and collapse them
- Fabrics are more reliable
- Fabrics have no packet drops under congestion
- Fabrics have linear cost and power scaling
- Fabrics deliver any-to-any non-blocking connectivity
- Fabrics are auto-provisioned
- Fabrics use all links actively
- Fabrics are designed for the east-west traffic in a modern data center
I might have missed few points but had tried to put most of the points here. So, Lets discuss these points in brief now.
Fabrics are Faster than Networks
QFabric is comprised of QFX3500 edge nodes based on first generation Trident chipsets from Broadcom and a core device based on the HiGig2 chips from Broadcom. It requires five header lookups, one full lookup at the ingress switch, three lookups in the Clos tree within the QF/Interconnect, and a final lookup at the egress switch. Only the first lookup is a L2/L3 lookup, subsequent are port-of-exit header lookups that tell the transit nodes in the QFabric which port to forward the frame to.
The Brocade statement that all uplinks are active in a fabric implies that they are not all active in a network. This was true in flat L2 networks of the past based on STP. But since the late 1990s vendors have delivered proprietary solutions that made all links active and since 2008 blended proprietary/open-standard solutions such as MLAG have become available. MLAG requires that two switches be the same make/model from a vendor, they share learning and state information via a proprietary protocol between the two switches, and then express standards-based LACP to all adjacencies. With MLAG the size of the proprietary system is two switches, with current fabric solutions it's the entire system of switches across the fabric.
The Brocade VCS Fabric has a scaling challenge though - it has nothing in their fabric beyond a top-of-rack switch. Yes you can build a flat L2 forwarding network with what is offered - but not a very large one, nor a very scalable one. The maximum number of switches supported is 24 with NOS 2.1 making this the most limited of the fabric offerings. It is also only capable of forwarding at Layer-2 and uses a completely proprietary control plane for topology construction.
Fact: For those to whom latency directly correlates to application performance - financial trading, cluster computing/modeling and deep analystics, you can build a 2-tier network with a latency as low as 1.5 microseconds using fixed-configuration rack switches, you can build a very scalable 2-tier network with latency as low as 5 microseconds using a mix of rack and modular switches. The fastest fabrics are a bit more than that.
Fact: You can build a 2-tier L2 network based on Multi-Chassis Link Aggregation and have all uplinks active. The main constraint in MLAG scaling is the density and performance of the spine switch - the denser and higher performance the systems are the larger a L2 domain you can build - currently these are as large or larger than the biggest fabrics.
The other factor that is very important to note here is the need for effective congestion management and buffering in the spine tier. In a multi-stage network with east-west traffic patterns it is quite common for some large number of ports to need to access the resources on another port - the larger the buffers are on the spine switches the more effectively they will perform before they have to drop traffic or issue a PAUSE frame to slow down traffic.
It is conceivable that a vendor can build an MLAG system that makes the spine 'wider' than 2 switches by intelligently controlling learning and handling all broadcast/unknown/multicast forwarding on a subset of the devices - but no one has implemented this type of system yet - largely because it grossly exceeds 99% of the markets requirements for scale.
Summary: Networks offer the flexibility to be designed at the absolute lowest latencies if required (trading off the use of modular large-buffer systems for fixed-configuration low latency systems in the spine is the most effective way of achieving absolute lowest latency/fastest forwarding). Current network solutions scale as well or better than the current fabric solutions - at L2, and certainly as you introduce routing at L3.
Simply put, from a 'Fabrics are Faster' perspective, the claim doesn't hold up. The fastest fabric is a bit slower than an equally scalable network, the fastest network is 2-3x lower latency than the fabric alternative.
QFabric is comprised of QFX3500 edge nodes based on first generation Trident chipsets from Broadcom and a core device based on the HiGig2 chips from Broadcom. It requires five header lookups, one full lookup at the ingress switch, three lookups in the Clos tree within the QF/Interconnect, and a final lookup at the egress switch. Only the first lookup is a L2/L3 lookup, subsequent are port-of-exit header lookups that tell the transit nodes in the QFabric which port to forward the frame to.
The Brocade statement that all uplinks are active in a fabric implies that they are not all active in a network. This was true in flat L2 networks of the past based on STP. But since the late 1990s vendors have delivered proprietary solutions that made all links active and since 2008 blended proprietary/open-standard solutions such as MLAG have become available. MLAG requires that two switches be the same make/model from a vendor, they share learning and state information via a proprietary protocol between the two switches, and then express standards-based LACP to all adjacencies. With MLAG the size of the proprietary system is two switches, with current fabric solutions it's the entire system of switches across the fabric.
The Brocade VCS Fabric has a scaling challenge though - it has nothing in their fabric beyond a top-of-rack switch. Yes you can build a flat L2 forwarding network with what is offered - but not a very large one, nor a very scalable one. The maximum number of switches supported is 24 with NOS 2.1 making this the most limited of the fabric offerings. It is also only capable of forwarding at Layer-2 and uses a completely proprietary control plane for topology construction.
Fact: For those to whom latency directly correlates to application performance - financial trading, cluster computing/modeling and deep analystics, you can build a 2-tier network with a latency as low as 1.5 microseconds using fixed-configuration rack switches, you can build a very scalable 2-tier network with latency as low as 5 microseconds using a mix of rack and modular switches. The fastest fabrics are a bit more than that.
Fact: You can build a 2-tier L2 network based on Multi-Chassis Link Aggregation and have all uplinks active. The main constraint in MLAG scaling is the density and performance of the spine switch - the denser and higher performance the systems are the larger a L2 domain you can build - currently these are as large or larger than the biggest fabrics.
The other factor that is very important to note here is the need for effective congestion management and buffering in the spine tier. In a multi-stage network with east-west traffic patterns it is quite common for some large number of ports to need to access the resources on another port - the larger the buffers are on the spine switches the more effectively they will perform before they have to drop traffic or issue a PAUSE frame to slow down traffic.
It is conceivable that a vendor can build an MLAG system that makes the spine 'wider' than 2 switches by intelligently controlling learning and handling all broadcast/unknown/multicast forwarding on a subset of the devices - but no one has implemented this type of system yet - largely because it grossly exceeds 99% of the markets requirements for scale.
Summary: Networks offer the flexibility to be designed at the absolute lowest latencies if required (trading off the use of modular large-buffer systems for fixed-configuration low latency systems in the spine is the most effective way of achieving absolute lowest latency/fastest forwarding). Current network solutions scale as well or better than the current fabric solutions - at L2, and certainly as you introduce routing at L3.
Simply put, from a 'Fabrics are Faster' perspective, the claim doesn't hold up. The fastest fabric is a bit slower than an equally scalable network, the fastest network is 2-3x lower latency than the fabric alternative.
Information Sources - Brad Reese
You might also like these recent post -
Dell Readies 40 GBe Virtual Network Systems - Read This
Access-List (ACL) for Internet Router Security - Read This
Voice over IP (VoIP) - Solutions Case Study - Read This
IPv6 Benefits - Its more than just larger address space - Read This
Spanning Tree Protocol (STP) - The Necessary Evil - Read This
Five Most Commonly used Networking Technologies - Read This
Found it useful, Consider sharing it with your friends -