OpEd:
Incorrect Base Assumptions About Network Management
July 9, 2009 by Kyle BradyTags: Bandwidth, IEEE, ISPs, Network Management, Smart Routing, Telcos, Traffic Shaping
As required by the FTC, a Full Disclosure is available - this piece adheres to the Code of Ethics
IEEE recently ran a lengthy article in the latest (July 2009) issue of Spectrum, their major monthly publication, called “A Radical New Router” – essentially describing and promoting a smarter network router to better manage traffic. Overall, an interesting piece that could prove highly useful in the industry, but throughout, the author and inventor of the technology, Dr. Lawrence G. Roberts, makes references to the so-called need to filter, augment, and otherwise monitor certain kinds of traffic – specifically, peer-to-peer (P2P) content. This base assumption from which he operates over many sequences of logic is incorrect, despite producing good insights into the current state, and possible future, of the modern global network. Not to mention having participated in its invention.
Whenever the subject of bandwidth usage and availability comes up in any public forum, P2P traffic is inevitably blamed with “hogging” bandwidth, “clogging pipes”, and other such euphemisms that imply wrongdoing and questionable use. The reality, no matter what the telecommunications or cable companies say, is that bandwidth is not disappearing into the ether due to the massive usage of protocols like BitTorrent. These myths and halftruths are perpetuated by internet service providers (ISPs) because they can produce a better bottom-line by spending less on equipment and infrastructure to support their userbase - as well as the copyfight-abuse organizations like the RIAA and MPAA because, when taking the longview, they have the same goals: reducing P2P traffic by any means necessary in order to potentially achieve higher profit margins.
Consider, for a moment, the issue most often cited for “traffic shaping”, the practice of filtering a users traffic based on the type and source: legality of content. While there is an abundance of content with questionable copyright origins based on the current interpretations of the DMCA (in America), there is also a sea of legal content being acquired by the same means: Nine Inch Nails, Radiohead, and a number of other musical artists have experimented with a freely available online distribution method, in addition to countless young movie producers that are only interested in their content being available and seen.
How can network monitoring practices differentiate between “legal” and “illegal” P2P traffic? Filtering by content source, such as a band’s official website vs. IsoHunt, is impractical – the content available via the official source is likely licensed for free distribution and sharing by other means. Filtering by traffic size, as in number of bytes transferred, is a gray area at best – setting an arbitrary size for acceptable P2P traffic, or any type of traffic, creates artificial pricing levels, not to mention potentially endorsing the acquisition of questionably sourced content. There is really only one option left, and it is what most ISPs choose in such cases: filter by traffic type.
Examples abound of ISPs delivering the speeds expected, or close to expected, for common traffic like web-browsing, email, and IM conversations, but getting heavily choked to lower levels (sometimes resembling dialup speeds) when a Torrent file is active – regardless of legality, content, or source. When the ISP will even admit to doing so, which is rare, they tend to blame their need for such filtering based on total bandwidth availability - “5% of the users in some networks [consume] 75% of the bandwidth”, to quote Roberts’ article. While it is hard for the ISPs to support such claims with hard evidence, this is improbable at best – how can a user consume that much bandwidth when they are prevented from doing so in the first place?
Yes, the internet, and networks of all types, are seeing explosive growth. And, yes, it will present some challenging problems in the coming years. But the true bandwidth capacity of the current infrastructure, which is constantly being expanded and optimized, is not anywhere close to being “full” – let alone full because of P2P traffic. The tech community likes to think that “the cloud” is the future of computing, meaning accessing remote rich applications via a web browser, and this means an increase in network traffic, purely by default – this is already being seen thanks to Facebook, Google, and countless others. Include Voice-over-Internet-Protocol applications like Skype, instant messaging, videoconferencing, remote desktop access, and numerous other now-standard types of network traffic, and the reason for the massive increase in bandwidth use becomes clear. P2P traffic is a part of it, but is by no means the singular culprit.
But if P2P isn’t inherently illegal or “bad”, can it be a large part of the future? It has been used to distribute video games, distribution of multimedia content from a United States government agency, and as a means to alleviate server stress for video playback for a major news outlet. These examples are merely the world-at-large testing the waters for this relatively new type of traffic, and the future may contain exponentially more – assuming the network connections don’t arbitrarily prevent users from connecting to such rich experiences. In the days of Usenet and bulletin board services, who could have predicted a common, open, and mostly free global network? If the network administrators of the time had decided to prevent an evolution of protocols, merely on the basis of profit, the modern Internet would not exist.
This is not to say that prioritizing network traffic, as the majority of the Spectrum article addresses, is not valuable. Packaging data streams such as video and audio together for processing and transmission could have substantial benefits on network stresses and overall activity patterns – evolving from “dumb” to “smart” routers is a necessary step in expanding the capacity, functionality, and ability of the Internet. However, there is a considerable difference between “prioritizing” and “filtering” – prioritizing means temporary delays that the end-user will never notice, on the scale of milliseconds, while filtering is restricting or preventing entirely the transmission of data.
Such Old World assumptions about the proper use of the Internet need to be discarded in order to allow it to continue the evolution into a more complex and useful tool – this is the basis of Net Neutrality, an area which the United States Federal Government is finally getting involved in. Lawrence G. Roberts, the IEEE, and hardware developers the world over would do well to take note: network management may be difficult at times, but the answer will not be found in restricting the freedoms of users. Many Internet users know this, the Federal Government appears to be slowly awakening to a similar realization, and it is time network operators joined the rest of the world in acknowledging what amounts to a Rights and Freedoms issue for the modern, digital age.
--- --- ---
Update (7/9/2009 10:50am PST): Quoted in a Net Neutrality piece by p2pnet.
Update (7/9/2009 4:14pm PST): I exchanged emails with Dr. Roberts himself regarding this, and the direct communication seems to help clarify - heavy editing may have been a culprit for some of the confusion on P2P traffic. A position of "fair and equal traffic" is much better than "regulate the unwanted traffic" - apparently Dr. Roberts and I are in agreement on this topic. Here's his clarification, unedited and printed with permission:
Perhaps the IEEE article was not sufficiently clear due to significant editing. However, I have never claimed P2P file sharing was bad. However, any multi-flow application is likely to consume far more than its fair share of the capacity an ISP allocates to a DSLAM or cable channel (often about 1000 subscribers). What most people don't understand is that the capacity for those 1000 subscribers is shared and not equal to 1000 times the peak capacity advertised. Often the ISP is at fault for misleading advertising but no ISP allocates much more than the statistical peak usage created by 1000 normal users. Unfortunately, IP networks today, using TCP, provide about equal capacity per flow, not per subscriber. Thus a multi-flow application using 100 flows will consume 100 times the capacity available to the normal user who is using one flow. Thus, if even 1% of the users are using certain P2P applications that open many flows, they will consume 60%-90% of the shared capacity of the DSLAM or cable channel. This reduces the capacity available to the other 99% of normal users to a small fraction of what they paid for. That is the problem. What I am recommending is for the network to adjust the sharing process such that when congestion occurs, capacity is allocated so that it is "equal capacity for equal pay", not equal capacity per flow. This means the multi-flow user gets their fair share in peak hours without hurting their neighbors, and in off hours get all they can use.
P2P happens to be the current main multi-flow application. However, cloud computing appears to be the next big multi-flow application and there will be others. My goal is to keep the Internet working smoothly and this requires it be fair, even under overload. Equal capacity for equal pay accomplishes this where today there is serious un-fairness. It is also important that this be accomplished not by picking on any specific application (like P2P) as DPI devices do, but by fixing the basic sharing process. P2P is not bad, and as you say no-one can tell if the file is legal or illegal. But any multi-flow application can produce unfairness due to the current structure of the network and it is this un-fairness that is the problem.
Larry
Update (7/9/2009 6:15pm PST): I only just noticed (sorry Cory!), but I was mentioned on BoingBoing for this.
Kyle can be found on Twitter and MySpace, or reached via email.
