Google unveiled some details of its new internal network at Open
Networking Summit in April and predictably the industry press and
OpenFlow pundits exploded with the “this is the end of the networking as
we know it” glee. Unfortunately I haven’t seen a single serious
technical analysis of what it is they’re actually doing and how
different their new network is from what we have today.
This is based solely on the
publicly-available information presented by Google’s engineers at Open
Networking Summit (plus an interview or two published by the industry
press).
What is Google doing?
After supposedly building their own switches, Google decided to build
their own routers. They use a distributed multi-chassis architecture
with redundant central control plane (not unlike Juniper’s XRE/EX8200
combo). Let’s call their combo a G-router.
A G-router is used as a WAN edge device in their data centers and
runs traditional routing protocols: EBGP with the data center routers
and IBGP+IS-IS across WAN with other G-routers (or traditional gear
during the transition phase).
On top of that, every G-router has a (proprietary, I would assume)
northbound API that is used by Google’s Traffic Engineering (G-TE) – a
centralized application that’s analyzing the application requirements,
computing the optimal paths across the network and creating those paths
through the network of G-routers using the above-mentioned API.
I wouldn’t be surprised if G-TE would use MPLS
forwarding instead of installing 5-tuples into mid-path switches. Doing
Forwarding Equivalence Class (FEC) classification at the head-end device
instead of at every hop is way simpler and less loop-prone.
Like MPLS-TE, G-TE runs in parallel with the traditional routing
protocols. If it fails (or an end-to-end path is broken), G-routers can
always fall back to traditional BGP+IGP-based forwarding, and like with
MPLS-TE+IGP, you’ll still have a loop-free (although potentially
suboptimal) forwarding topology.
Is it so different?
Not really. Similar concepts (central path computation) were used in
ATM and Frame Relay networks, as well as early MPLS-TE implementations
(before Cisco implemented OSPF/IS-IS traffic engineering extensions and
RSVP that was all you’d had).
Some networks are supposedly still running offline TE
computations and static MPLS TE tunnels because they give you way better
results than the distributed
MPLS-TE/autobandwidth/automesh kludges.
MPLS-TP is also going in the same direction – paths are computed by
NMS, which then installs in/out label mappings (and fast failover
alternatives if desired) to the Label Switch Routers (LSRs).
Then what is different?
Google is (as far as I know) the first one that implemented the
end-to-end system: gathering application needs, computing paths, and
installing them in the routers in real time.
You could do the same thing (should you wish to do it) with the
traditional gear using NETCONF with a bit of MPLS-TP sprinkled on top
(or your own API if you have switches that can be easily programmed in a
decent programming language – Arista immediately comes to mind), but it
would be a “slight” nightmare and would still suffer the drawbacks of
distributed signaling protocols (even static MPLS-TE tunnels use RSVP
these days).
The true difference between their implementation and everything else
on the market is thus that they did it the right way, learning from all
the failures and mistakes we made in the last two decades.
Why did they do it?
Wouldn’t you do the same assuming you’d have the necessary
intellectual potential and resources? Google’s engineers built
themselves a high-end router with modern scale-out software architecture
that runs only the features they need (with no code bloat and no bugs
from unrelated features), and they can extend the network functionality
in any way they wish with the northbound API.
Even though they had to make hefty investment in the G-router
platform, they claim their network already converges almost 10x faster
than before (on the other hand, it’s not hard converging faster than
IS-IS with default timers), and has average link utilization above 90%
(which in itself is a huge money-saver).
Hype galore
Based on the information from Open Networking Summit (which is all
the information I have at the moment), you might wonder what all the
hype is about. In one word: OpenFlow. Let’s try to debunk those claims a
bit.
Google is running an OpenFlow network. Google is using OpenFlow between controller and adjacent chassis
switches because (like everyone else) they need a protocol between the
control plane and forwarding planes, and they decided to use an
already-documented one instead of inventing their own (the extra
OpenFlow hype could also persuade hardware vendors and chipset
manufacturers to implement more OpenFlow capabilities in their
next-generation products).
Google built their own routers ... and so can you.
Really? Based on the scarce information from ONS talks and interview in
Wired, Google probably threw more money and resources at the problem
than a typical successful startup. They effectively decided to become a
router manufacturer, and they did. Can you repeat their feat? Maybe, if
you have comparable resources.
Google used open-source software ... so the
monopolistic Ciscos of the world are doomed. Just in case you believe
the fairy-tale conclusion, let me point out that many Internet exchanges
use open-source software for BGP route servers, and almost all
networking appliances and most switches built today run on open source
software (namely Linux or FreeBSD). It’s the added value that matters,
in Google’s case their traffic engineering solution.
Google built an open network – really? They use
standard protocols (BGP and IS-IS) like everyone else and their traffic
engineering implementation (and probably the northbound API) is
proprietary. How is that different (from the openness perspective) from
networks built from Juniper’s or Cisco’s gear?