INTRODUCED TO OUR STORY LURKING
THE SHADOWS OF A PACKED SERVER ROOM, WHERE NETWORK ADMINISTRATOR FRANCIS SPENT MOST OF HIS DAY staring intently at bank after bank of monitoring screens, searching for any glimmer of hope that things might soon improve. Unfortunately for Francis, all of the late night hours and the large sums of money that he and his organization had invested in ever faster equipment and expanding machines had been for naught, as site-wide networking performance had been on the decline with notable increases in packet loss and network latency fluctuations. What could possibly be the cause of such problems, and were they so difficult to remedy? The true culprit here wasn’t a complex configuration setting or a tidal wave of bandwidth demand, but rather something much simpler: a mere lack of processing power.
Choosing the right network controller chip is a problem faced by many data center and enterprise network operators. With ever increasing traffic volumes and low latency requirements from applications, having the right selection capability is critical. But do IT professionals have the information they need?
Understanding Throughput Requirements Beyond Raw Numbers
Most datasheets for networking ICs begin with a headline Throughput number, but this number does not translate to real-world performance. While it is easy to rate ICs on Theoretical Throughput (e.g. 100 Gbps), in reality these ICs process packets that are much smaller than 1000 bits, and they encounter varying levels of traffic as well as control overhead that is processed by the controller much less efficiently than data.
Research has shown that the increasing overhead on network controllers caused by smaller packet sizes is becoming a major issue. Today’s network controllers are designed to optimally process streams of large, uniform-sized frames, but the rising presence of voice, video and data packets of significantly varying sizes is forcing them to handle uncharacteristic and often unpredictable traffic flows—particularly in virtualised environments.
So, what is the answer? Should we lower the frame sizes in the interest of efficiency on the network controller? No, it is much easier, and, on balance, likely better, to modify the software that generates packets to better packetise video streams. Let us resolve this legitimate challenge to network infrastructure performance, rather than attempting to press older designs into service indefinitely.
All of the controllers considered here process pixels in some pipeline fashion. Some of them are very good at raw throughput, building a long pipeline to achieve high, sustained processing rates. Others are better at avoiding latency, accepting lower throughput in order to keep the pipe moving as quickly as possible. Which approach is best for you will depend on your specific needs and level of understanding of the competing tradeoffs.
Protocol Support Considerations
Modern networks are multi-programmed meaning that the controller is managing multiple protocol spaces at the same time. So the traditional role of a controller in terms of just controlling the switch fabric for L2 forwarding is now only a portion of what a modern controller does. In addition to traditional L2 forwarding, the modern controller also performs L3-VXLAN encapsulation, SR-IOV virtualization, SDN protocols, and more.
Not all protocol support is equal. Just because a chip supports a particular protocol does not mean that that protocol will be efficiently supported or that there will be no performance costs associated with using that protocol. When evaluating the Broadcom network controller chip note not only the list of supported protocols, but also how that protocol is actually supported (e.g. hardware offload, software assistance) as well as limitations around using multiple protocol support simultaneously.
Integration Obstacles
But another very serious point of failure for network controller integration is driver compatibility with the driver installed on the driver of the physical machine, i.e. the driver the user uses to drive their car. Linux support for integration of network controllers from different vendors differs greatly from vendor to vendor. Windows driver integration support can differ GREATLY FROM DRIVER TO DRIVER. Even the best network controller on the market is worthless on your computer if it isn’t able to integrate correctly.
When selecting a high performance controller it is commonly look at in terms of signal capability, processing power and feature set. A more important factor could be its power consumption and how it will be housed, as many modern high performance devices require substantial amounts of power to operate and generate a corresponding amount of heat.
Future-proofing Without Overspending
There is a temptation to always go for the latest top-of-the-line controller in the marketplace, however this is in general a poor investment. A better investment would be to select a controller capable of supporting the current protocols in place as well as the future requirements for the next 18-24 months.
When evaluating the future capabilities required of your controller don’t just consider the performance figures. Next generation security protocols such as HSM and 400 Gigabit Ethernet will affect the requirements placed on the controller and the futureproofing that the controller offers over the life cycle. A controller optimised for today’s environment but which also supports the current standards and has a future migration path will be more valuable in the longer term than a solution which optimises performance above all else.
Vendor-provided roadmaps can provide some insight to the future viability of certain technologies, and in the case of network controller vendors, it is these companies with deep investments in controller technology who are best positioned to fund driver implementation and foster industry-wide adoption. The growing concern around security on networks will be a significant catalyst to this end.
Making the Decision
Instead of baselining at 100% theoretical utilization, we’re baselining at real world operating loads. What do network operating loads actually look like. Average power, packet size distribution, protocol distribution, and maximum power loads all play a role in how we can test the capabilities of a controller.
Test the test controller as far as is practical in real-world operating conditions. Some aspects can be unit tested, but the complex interactions with QoS etc with traffic shaping etc are very difficult to adequately test in a controlled lab environment. Look for experience and relevant examples/case studies similar to your organisation.
When evaluating the Total Cost of Ownership (TCOO) of automated control solutions, remember to look beyond the initial cost of the controller. Other factors which should be considered are the typical annual power usage, typical annual cooling required to house the controller, costs of licenses for control software, costs of support personnel and equipment required to program, maintain, and troubleshoot the controller. Although a “cheaper” controller may initially seem cost effective, if it ends up requiring significantly more time and resources to maintain over the life of the product, it in fact can be more expensive in the long run.
Your network controller chip is what will run the show in terms of how your network performs. So choose wisely. Test thoroughly. And when in doubt, over-provision for future growth.
