Cable giant Comcast finds itself in hot water last year when users of their network start having problems with their peer-to-peer (P2P) file sharing. Comcast says, “We’re not blocking P2P traffic.” The Associated Press’s Peter Svenson, on Oct. 19, 2007 writes, “Comcast Corp. actively interferes with attempts by some of its high speed Internet subscribers to share files online, a move that runs counter to the tradition of treating all types of Net traffic equally.” Comcast replies, “We’re not blocking P2P traffic.” Svenson tests this by trying to download the King Jame’s Bible (which is not copyrighted) using a Comcast connection in San Francisco and in Philadelphia. Despite these findings, Comcast still insists that they aren’t blocking P2P traffic.
This dispute generates a lot of net discussion that passes through Comcast’s portion of the Internet unmolested. It is not positive publicity.
The Federal Communications Commission (FCC) eventually takes an interest and tells Comcast to stop blocking P2P traffic. Comcast tells the FCC that they aren’t blocking P2P traffic and suggest that the FCC is not authorized to interfere with Comcast’s management of their own network, anyway. The FCC, after deciding that it’s exactly their mandate to regulate broadband, tells Comcast that they must treat all Internet traffic consistently. Net neutrality thus becomes official US policy. Comcast demurs, and the FCC sets a deadline. Comcast sues, but on the last day before the deadline, Comcast comes clean and answers the FCC. In their answer, they include attachments that describe their past network management activity and their future plans for managing their networks. Comcast publishes these on their website. Interested web journalists take note.
That deadline was Friday 19, 2008. Much has been written about Comcast caving in, but little has been written about the nuts and bolts of how they were actively shaping a certain class of traffic passing through their network. Let’s look at that.
First, let’s look at how Comcast’s high-speed network is organized. It is a hybrid fiber-coax network. Starting from the customer’s network modem in his house, cable runs to a nearby optical node. Several other houses in the neighborhood also connect to the same optical node. The connection then becomes optical fiber. Several of these combine in a hub, and that hub is called a CMTS (cable modem termination system). Comcast says that a typical CMTS port can handle about 275 downstream ports and about 100 upstream ports. This is what provides the asymmetric description of consumer-grade connections – uploads are slower than downloads. The industry refers to this as “provisioning” or more simply, a best-efforts promise to provide a certain level of performance to each customer.
This is what a CMTS looks like
Comcast, like any other network provider, has a right to employ management practices on their own network. Comcast, some years ago, became aware of significant congestion happening on their networks. Much of this congestion was on the upload side. This is the weaker side of the asymmetric equation, but the unexpected upload traffic volume was causing problems. In their answer to the FCC, Comcast talks about latency, which is packets arriving slowly, and jitter, which is packets arriving with a variable delay.
A packet, in Internet and networking terminology, is the basic unit of network traffic. Each packet consists of a header and a body. The header describes where the packet is bound, and the body contains the content – parts of a web page or an email. The header also contains information about the data’s size, destination, and timing.
Internet users have traditionally been more concerned with download speeds than upload speeds – how fast they can download that track from iTunes or that video from YouTube – and less about how fast they can send – email and file uploads.
The congestion on Comcast’s upstream side surprised them. To further investigate the causes, Comcast joined forces with the Canadian company Sandvine. Together they discovered that the unanticipated traffic was due to “the use of several P2P protocols . . . regularly generating disproportionate burdens on the network, primarily on the upstream portion of the network, causing congestion that was affecting other users on the network.”
To combat the cause of the congestion, Comcast decided to establish thresholds for the managed protocols. Below that threshold, upload traffic is not managed.
The actual device that Comcast used for their traffic shaping is the Sandvine PTS 8210, which went into full deployment in 2007. The PTS 8210 is installed physically close to the CMTS it works with. The purpose of the device is to monitor the upstream traffic going by. Traffic does not ordinarily pass through the device. The flow from many customers is “mirrored” to the device, and it does not delay normal upload traffic.
The Sandvine PTS 1810
The PTS inspects the traffic as it goes by, looking in each packet header to determine what protocol is being used. It doesn’t bother with the content of the packets but it looks specifically for headers identifying the traffic as belonging to BitTorrent, Gnutella, Ares, eDonkey, or FastTrack. As it does this, it counts the number of packets using these protocols passing on to the upstream router.
Comcast determined that “the number of simultaneous unidirectional upload sessions of any particular P2P protocol at any given time serves as a useful proxy for determining the level of overall network congestion.” In other words, when a certain percentage of all traffic is on the upload side, congestion is soon to follow.
Their task was complicated by the fact that the different file sharing networks have different ratios of bidirectional and unidirectional traffic. BitTorrent, for example runs at about 20:1, which is consistent with its being a very distributed network used for sharing large files. (Torrents can consist of a file on a single user’s hard drive or the same file distributed across thousands of computers. This is very efficient, and it is the advantage of P2P networking.)
The PTS sits and watches, and when a certain number of unidirectional uploads for a protocol, say BitTorrent, rises above the preset thresholds, the PTS kicks in. In each package it identifies as part of that protocol, it inserts a “reset” flag. This is the equivalent of an error message in the network. Named “RST,” it tells the computers involved to reset the connection.
Sending out the reset message not only terminates the connection, it prevents the BitTorrent client from “seeding.” Seeders are PCs that have complete copies of the files involved. Thus Comcast’s network practice was not only interfering with the user’s traffic flow, but that of other users of BitTorrent.
Comcast claims that that P2P traffic now comprises about half of its network’s upload capacity. Even at this level, they say, less than 10% of the flow is affected by Comcast’s traffic management.
And they are going to stop doing it. In their consent to the FCC’s “Memorandum Opinion & Order,” Comcast will stop using reset packets to manage its high speed Internet Networks by the end of 2008.
So now Comcast has agreed to manage their network in a “protocol agnostic” manner. We’ll see how they plan to do that in the next article.
“We’re Not Blocking P2P Traffic” - How Comcast Plans to Become “Protocol-Agnostic” - Ordered by the FCC to cease blocking P2P protocols, Comcast has revealed how they plan to comply in a “protocol-agnostic” manner. This has been a fascinating look at the inner workings of a large ISP, but can we trust them to treat heavy Internet users fairly?
Trouble Ahead: Broken Down on the Information Superhighway - In the war against terrorism and media piracy the lines have been drawn - or have they? Highly visible is the conflict between the pipe owners and the content pushers, but are secret deals and treaties that affect the global use of the Internet, music, and movies being made out of the public eye? Read this to learn more about the “Anti-counterfeiting Trade Agreement” and who’s really behind ACTA.