• New "ShaperProbe" tool detects ISP traffic shaping



    Two researchers at Georgia Tech can tell you exactly how American ISPs shape Internet traffic, and which ones do so. Bottom line: of the five largest Internet providers in the country, the three cable companies (Comcast, Time Warner, Cox) employ shaping while the telephone companies (AT&T, Verizon) do not—though that fact is less significant for the user experience than it might first sound.

    Partha Kanuparthy and Constantine Dovrolis wanted to measure Internet shaping, so they built a tool calledShaperProbe to do so. The tool relies on users from around the country running tests in which the user's computer transmits data at a constant bit rate, while ShaperProbe's 48 Linux-based server instances watch incoming traffic to see if that rate degrades in predictable ways over time. Using the M-Lab infrastructure, ShaperProbe has collected more than 1 million trial runs from 5,700 ISPs over the last two years (run your own test).

    The resulting paper (PDF) documents the first accurate attempt to “measure traffic shaping deployments on the Internet.”

    Emptying the bucket

    Traffic shaping hardware generally relies on the concept of a “token bucket.” Traffic management hardware will generate a digital token for each Internet user at a predetermined rate. These tokens fill a virtual token bucket; transmitting packets over the Internet removes tokens from the bucket. If the bucket empties, no more data can be transmitted until a new token is deposited.

    The practical result is that the user sits down to her computer with a full token bucket and can immediately blast data through her connection as fast as the connection can go. But after some interval of time, usually measured in seconds, this sort of full-throttle data transmission empties the token bucket and the user is now limited to transmitting at the token generation rate.

    ShaperProbe measures the length of time it takes for traffic shaping hardware to make a "level shift" to a slower speed limit on the line. Charting this data enough times, and from enough different users with different Internet access tiers, allowed Kanuparthy and Dovrolis to estimate the token bucket parameters—and thus to figure out exactly when and how traffic shaping kicks in for any particular ISP's speed tier.

    Traffic shaping isn't necessarily some devious attempt to give users less bandwidth than expected; it can also be used to let users transmit "bursty" files more quickly than their speed tier would otherwise allow, shaping them down to their chosen tier only after some set amount of time. This is how Comcast markets its "PowerBoost" technology, for instance.




    The use of shapers can also be dictated by the underlaying access tech. The researchers note that DSL providers can “dynamically change link capacity instead of shaping, while a cable provider is more likely to shape since DOCSIS provides fixed access capabilities." It can also be applied to the entire Internet link or more narrowly to specific kinds of traffic— which, as the researchers note, is "relevant to the ‘heated’ network neutrality debate."

    The research shows that Comcast, Road Runner (from Time Warner Cable), and Cox all use downstream shaping—but only Comcast and Cox also use upstream shaping. Neither AT&T nor Verizon shape in either direction.


    The research is separate from a recent million-dollar Google grant to other Georgia Tech researchers looking at ways to detect Internet censorship or throttling, but it was funded by a previous 2009 Google grant. It also runs on the Google-funded M-Lab hardware network. As Google noted in its most recent Georgia Tech grant, it hopes to create tools to that "users could determine whether their ISPs are providing the kind of service customers are paying for, and whether the data they send and receive over their network connections is being tampered with by governments and/or ISPs."

    Source: Ars Technica
    Comments 11 Comments
    1. darkstate01's Avatar
      darkstate01 -
      Over here in the u.k we are punished from 9am till 9pm. from 9am till 3pm you are allowed to d/l 4 GB then 3pm till 9pm you are allowed to d/l 1.5GB. once you have d/l either size you are then punished for 5 hrs at 1/4 the speed you are fully paying for. It's daylight robbery they can get away with this crap. If I pay for 10meg I want 10 meg 24/7. Any other Industry would be taken to court over this for not providing what they are selling. watching hd content online is close to impossible because of the caching of the infomation,otherwords constant buffering while watching.
    1. duke0102's Avatar
      duke0102 -
      Totally agree with you there darkstate01. If I went to mcdonalds and at times of the day there large coke/pepsi was half full you'd have trading standards busting the doors down like some sort of riot police in a terrorist situation but they don't give a monkeys about the phone side.
      I know BT have announced there higher packages are truly unlimited but I still get random crappy speeds that they put down to any old rubbish excuses.
    1. wtj's Avatar
      wtj -
      This article has a link to the ShaperProbe tool.
      It sent me an "Artemis" trojan/virus
      and changed the settings in :
      HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Security Center\

      Hey Editor,
      was this a rarity or don't you proof these articles?
    1. hopkins35's Avatar
      hopkins35 -
      Quote Originally Posted by darkstate01 View Post
      Over here in the u.k we are punished from 9am till 9pm
      With which provider though? I spent many unhappy years being bandwidth throttled by Tiscali (now talktalk) and BT before doing some research and taking the plunge with a (then) relatively unknown company called Andrews & Arnold (aaisp.net) who then served mostly business users. They're a bit more expensive although you can tailor your price using their unit system and overnight 1 unit = 1TB (perfect for newsgroup DLing) and you get no bandwidth throttiling, no internet censorship and when there's an issue with BT over your line they don't half kick some arse to get it fixed - a couple of years ago I had 3 openreach engineer visits in a week and wasn't charged a penny!! Check them out if you're fed up with shit service from your ISP
    1. mjmacky's Avatar
      mjmacky -
      Quote Originally Posted by wtj View Post
      This article has a link to the ShaperProbe tool.
      It sent me an "Artemis" trojan/virus
      and changed the settings in :
      HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Security Center\

      Hey Editor,
      was this a rarity or don't you proof these articles?
      You need to be more specific about which link, and did you download and run a program?
    1. wtj's Avatar
      wtj -
      mjmacky:

      There are two links to get YOUR OWN TEST

      "ShaperProbe"

      https://filesharingtalk.com/external/...aperprobe.html

      and

      "your own test"

      https://filesharingtalk.com/external/...ab-tools#tool5

      Follow them both and find out for yourself.
    1. duke0102's Avatar
      duke0102 -
      I downloaded the one from the "(run your own test)." link and that seems to work fine with no virus for me. Kaspersky and Win 7 64bit, ran it under Safe Run (similar to sandbox) and all cool so far.
    1. darkstate01's Avatar
      darkstate01 -
      Quote Originally Posted by hopkins35 View Post
      Quote Originally Posted by darkstate01 View Post
      Over here in the u.k we are punished from 9am till 9pm
      With which provider though? I spent many unhappy years being bandwidth throttled by Tiscali (now talktalk) and BT before doing some research and taking the plunge with a (then) relatively unknown company called Andrews & Arnold (aaisp.net) who then served mostly business users. They're a bit more expensive although you can tailor your price using their unit system and overnight 1 unit = 1TB (perfect for newsgroup DLing) and you get no bandwidth throttiling, no internet censorship and when there's an issue with BT over your line they don't half kick some arse to get it fixed - a couple of years ago I had 3 openreach engineer visits in a week and wasn't charged a penny!! Check them out if you're fed up with shit service from your ISP
      Sorry about that,The company was Initially NTL who were great,Then changed to Virgin broadband. http://shop.virginmedia.com/help/tra...anagement.html
    1. hopkins35's Avatar
      hopkins35 -
      Quote Originally Posted by darkstate01 View Post
      Sorry about that,The company was Initially NTL who were great,Then changed to Virgin broadband. http://shop.virginmedia.com/help/tra...anagement.html
      Virgin are notorious for it. Our town was one of the trial areas for their 50Mb fibre optic broadband and I was really tempted to sign up as I'm currently only on 7Mb but when I read their reviews and their fair usage policy I thought stuff it, I get my full bandwidth 24/7 at the moment. Tiscali used to drop me down to speeds less than dial up (literally 2KB/S) at some times of the day and never more than 100KB/S between 3pm and 11pm and this on a 7Mb line.....crazy I'm never going back to that if I can avoid it!
    1. darkstate01's Avatar
      darkstate01 -
      I just can't be arsed moving,I was on BT when Broadband first came out but they were useless. Someone will realize this is a huge con and start a movement,then I'll just on for the ride to get it stopped.
    1. hopkins35's Avatar
      hopkins35 -
      Quote Originally Posted by darkstate01 View Post
      Someone will realize this is a huge con and start a movement
      Let's hope it goes that way rather than towards net neutrality!! Switching ISPs is a hassle but if you do some research and dont mind spending a couple of quid extra a month you'll never have to switch again. My advice is smaller is definitely better when it comes to ISPs.....the other thing about that ISP that I mentioned is that they're based in this country, they have a support team of about 20 and English is their first language!!