OpenStack Network Performance — Our 2014 Technical Deep Dive
In late 2014, like many others in the industry, we were quite excited about OpenStack. We decided to run a series of network performance tests comparing different virtual switching technologies used with it: LinuxBridge, Open vSwitch, and Docker.
The experiments were straightforward: virtual machines running on OpenStack, sending TCP streams via iperf for 25 seconds while monitoring TCP congestion window (cwnd) and slow-start threshold (ssthresh). The goal was to see how efficiently TCP performed under different bridging solutions.
The Results (With Pretty Graphs)
Here are the graphs from those tests. They show how quickly the congestion window grew and where packet drops occurred.
Graph 1: LinuxBridge — Fast initial growth, occasional packet drops
Graph 2: Open vSwitch (Neutron ML2) — More packet drops after peak bandwidth
Graph 3: Docker (using Linux bridge) — Stepped growth pattern
Conclusion from 2014: Different bridge implementations caused varying levels of packet drops on the receiving side. Open vSwitch, in particular, showed more instability in our tests. We were diligently trying to understand and optimize the networking stack for real-world OpenStack deployments.
Hindsight is 20/20 (and Slightly Painful)
Looking back, we invested serious effort into OpenStack — including two months of intense work in Ericsson’s Budapest office trying to stabilize it for a major project for the Spanish telecom giant Telefónica. We even refactored parts of Open vSwitch to improve throughput. The core OpenStack components, however, proved remarkably resistant to stabilization.
Eventually Ericsson outsourced the project and quietly moved away from OpenStack altogether. Later we learned that Mirantis — a company with strong russian roots — had raised and allegedly laundered over $100 million through the OpenStack ecosystem. What we had taken as a promising open source cloud platform turned out to have some rather murky financial undercurrents.
In the end, OpenStack taught us a valuable lesson: sometimes the most hyped “revolutionary” technologies in our industry are not necessarily the best engineered — or the cleanest. We took those TCP graphs seriously in 2014. Today we look back with a smile at our younger, idealistic selves and remain grateful that our path eventually led to building much simpler, more reliable, and transparent solutions at LightUp.Cloud.