Back in November, T-Mobile announced a new service for its mobile customers called Binge On, in which video streams from certain websites don’t count against customers’ data caps.1 The service is theoretically open to all video providers without charge, so long as T-Mobile can recognize and then “optimize” the provider’s video streams to a bitrate equivalent to 480p. At first glance, this doesn’t sound too harmful—customers can watch more video without worrying about their caps, most will consider 480p to be adequate quality (especially on a small phone screen), and the harms of treating individual video providers differently are diminished when T-Mobile offers the program to any provider for free.

However, as Marvin Ammori wrote in Slate, there is another “feature” of Binge On that has many customers complaining. Ammori pointed out that T-Mobile is applying its “optimization” to all video, not just the video of providers who have asked T-Mobile to be zero-rated. T-Mobile claims it does this to provide a better experience for its customers, saying that

“T-Mobile utilizes streaming video optimization technology throughout its network to help customers stretch their high-speed data while streaming video”

and that Binge On helps “deliver a DVD quality (typically 480p or better) video experience with minimal buffering while streaming.”

Testing T-Mobile’s Binge On Optimization

We were curious what exactly this optimization technology involved, so we decided to test it out for ourselves. We posted a video on one of our servers and tried accessing it via a T-Mobile LTE connection using various methods and under various conditions.

(If you want to skip the technical details of our testing and jump directly to the results, click here.)

All of the tests were done with an account that had Binge On enabled, in the same physical location and at roughly the same time of day, using the same phone. We ensured the phone had a good 4G LTE connection at all times. For each test, we measured the throughput between our server and the phone for approximately five minutes at five second intervals. The four tests we performed were:

• Streaming a video embedded in a webpage using HTML5 (“Streaming in Browser”),
• Downloading a video file to the phone’s SD card (“Direct Download”),
• Downloading a video file to the phone’s SD card, but with the filename and the HTTP response headers changed to indicate it was not a video file (“Direct Download, Non-Video File Extension”), and
• Downloading a large non-video file for comparison (“Direct Non-Video Download”).

Each test was done over an HTTP connection, which allowed T-Mobile’s network to observe the content of the connection and perform “optimization” (labeled “Binge On” in the graph below), and over an HTTPS connection, which prevented T-Mobile’s network from observing the content of the connection, thus representing behavior without any sort of Binge-On-related optimization (“Normal”).

Our findings are summarized in the graph below, where we plot the mean throughput for each test, as well as one standard deviation. The units are megabits per second.

Test Results: No Optimization, and Everything Gets Throttled

The first result of our test confirms that when Binge On is enabled, T-Mobile throttles all HTML5 video streams to around 1.5Mps, even when the phone is capable of downloading at higher speeds, and regardless of whether or not the video provider enrolled in Binge On. This is the case whether the video is being streamed or being downloaded—which means that T-Mobile is artificially reducing the download speeds of customers with Binge On enabled, even if they’re downloading the video to watch later. It also means that videos are being throttled even if they’re being watched or downloaded to another device via a tethered connection.

The second major finding in our tests is that T-Mobile is throttling video downloads even when the filename and HTTP headers (specifically the Content-Type) indicate the file is not a video file. We asked T-Mobile if this means they are looking deeper than TCP and HTTP headers, and identifying video streams by inspecting the content of their customers’ communications, and they told us that they have solutions to detect video-specific protocols/patterns that do not involve the examination of actual content.

Our last finding is that T-Mobile’s video “optimization” doesn’t actually alter or enhance the video stream for delivery to a mobile device over a mobile network in any way. 2 This means T-Mobile’s “optimization” consists entirely of throttling the video stream’s throughput down to 1.5Mbps. If the video is more than 480p and the server sending the video doesn’t have a way to reduce or adapt the bitrate of the video as it’s being streamed, the result is stuttering and uneven streaming—exactly the opposite of the experience T-Mobile claims their “optimization” will have.

Given the difference between what T-Mobile implies they do and what we found, we contacted them to get clarification. They confirmed that they don’t do any actual optimization of video streams other than reducing the bandwidth allocated to them (and relying on the provider to notice, and adapt the bitrate accordingly).

T-Mobile has claimed that this practice isn't really "throttling," but we disagree. It's clearly not "optimization," since T-Mobile doesn't alter the actual content of the video streams in any way. Even the term "downgrading" is inaccurate, because that would mean video streams are simply being given a lower priority than other traffic. If that were true, then in the absence of higher priority traffic, videos should stream at the same throughput as any other content. But that's not the case: our tests show that video streams are capped at around 1.5Mbps, even when the LTE connection and the rest of T-Mobile's network can support higher throughput between the customer and the server.

In other words, our results show that T-Mobile is throttling video streams, plain and simple.

Dear T-Mobile: Stop Futzing With Your Customer’s Traffic

Setting aside for the moment the question of when zero-rating constitutes a violation of net neutrality in and of itself (more on that later), it’s pretty obvious that throttling all traffic based on application type definitely violates the principles of net neutrality. It also obviously violates the FCC’s Open Internet Order, which says that ISPs

“…shall not impair or degrade lawful Internet traffic on the basis of Internet content, application, or service…subject to reasonable network management”,

especially since throttling traffic independently of congestion (as T-Mobile is doing) makes it clear that this throttling is not being used for reasonable network management.

What should T-Mobile do instead? One option would be to stop throttling the video of providers who haven’t signed up to be zero-rated in Binge On, regardless of the status of the T-Mobile customer. This would address the complaints of video providers, since only edge providers who actually chose to be throttled would have their videos throttled.

But the best option would be to make Binge On opt-in (instead of opt-out), with clear disclosure that opting in will throttle all video traffic. Many of T-Mobile’s customers don’t realize that Binge On has this unfortunate side effect―especially since T-Mobile has buried the fact that Binge On throttles all video in their fine print. If T-Mobile were to be clear with its customers that enabling Binge On meant all of their video would be throttled, and then ask them whether or not they wanted to opt in, then they could obtain meaningful customer consent.

As an aside, it’s also obvious that T-Mobile is capable of recognizing video streams from providers who aren’t enrolled in Binge On. Given that, we don’t understand why they require providers to enroll in Binge On in order to get their videos zero-rated. If T-Mobile truly wants to be neutral, then all throttled videos should be exempt from customer data caps.  

Of course, this entire argument operates on the assumption that data caps are necessary on mobile networks, since zero-rating only makes sense when there’s a cap for data to be exempt from. And even if you accept that data caps are necessary, whether or not zero-rating is a neutral practice is a completely separate question. Either way, however, we don’t think exemptions from data caps should necessarily be heralded as pro-customer moves—but these are topics for a separate blog post.

The FCC Should Investigate

In the meantime, if T-Mobile doesn’t change its behavior then it’s up to the FCC to follow up. After all, the net neutrality rules aren’t just words on a piece of paper—they’re regulations meant to protect Internet users from precisely this sort of abuse of power by ISPs. We believe the FCC should regulate lightly, but our research suggests this is a significant consumer harm that runs afoul of well-established open internet principles. The FCC can and should step in and hold T-Mobile accountable.

• 1. To be fair, T-Mobile’s plans don’t have traditional data caps—they just slow down to “2G” speeds once customers exceed a certain amount of data. With that said, a limit on data is still a limit on data—and we’ll call it a cap for the purposes of this post.
• 2. We determined this by comparing the hash of a downloaded video file to the hash of the same file on the server; in all of our tests, the hash was identical.