Some technical questions about usenet

[maphongbax]

Hey guys,
So I have been using Usenet for a while now but there are something I still don't understand:

1. What makes a post incomplete in the first place? To transfer files, I am pretty sure they use TCP protocol, isn't TCP supposed to deliver file in a pristine condition?

2. How can usenet providers make profit if they charge such as low price? I downloaded 300 GB plus last month, and paid $11 to astraweb. Usually, other hosting company like godaddy.com charge a fortune for 100 GB per month, (they say their unlimited but that's a lie).

[Wwwildthing]

1. retention
2. volume

[UsenetGuy]

I can't answer the first question, since I am no expert. The second one I can answer though.

Companies such as Astraweb will have thousands of customers, many of them will use the service very little (sometimes even not at all) and just keep paying the money because it is not that much, it does not affect them. Meaning there will be plenty of bandwidth for the heavy users. Also I suppose astraweb would buy so much bandwidth and have contracts with various major internet backbones that they would get it fairly cheap.

[Sonnentier]

1) I'm pretty sure it's not the fault of TCP or the protocol used to upload, but while messages propagate from one server to another, it looks like some are left out. They might also be stored on a news server which then experiences hardware problems that affects these parts, or some articles are out of the retention's reach.

Feel free to add/ revise

[Rart]

I'm not positive, but usenet has a massive amount of content. Petabytes(maybe exa or higher) of data are on usenet being indexed everyday from one provider, so I'm guessing theres bound to be mishaps.

[ericab]

about 4.5 petabytes total, +/-

and 5+ terabytes added per day

[Beck38]

1. What makes a post incomplete in the first place? To transfer files, I am pretty sure they use TCP protocol, isn't TCP supposed to deliver file in a pristine condition?

Absolutely NOT. Virtually all incompletes (I call them 'SKIPS' when a part or parts of a RAR posting is simply missing in action, so to speak) is caused by the transmission from the poster to the usenet server fades a bit. Big gaps are caused by really big fades. The server-to-server protocol (say, Astraweb to Giganews) is pretty tight, but can fail as well.

2. How can usenet providers make profit if they charge such as low price? I downloaded 300 GB plus last month, and paid $11 to astraweb. Usually, other hosting company like godaddy.com charge a fortune for 100 GB per month, (they say their unlimited but that's a lie).

There are two basic costs with running a usenet server (I did for a couple years in the mid-90's):

Cost of hardware and on-going basics (space, co-location costs, electricity, etc). We all know the hardware costs are rapidly heading toward zero (just look at the cost of hard-drives; 1Gig in 1995 cost around $500. Today, 10cents.

Of course, one can say (truthfully) that the 'consumables' (space, electricity) is a lot more, but NOT 1000 times more.

The other big cost is, of course, the bits-space coming in. That as well has dropped like a rock, due to fiber. Research the cost of say, a basic chunk of bits (oh, today, say an OC48/2.5Gb/s) at any of the collocation facilities. I'm a bit 'out of date' on my pricing, having retired from the telecom industry some 7 years ago. But you can do the math on how may users such a line could support, at whatever speed those users are coming in at.

BTW, seven years ago such a line would run around $2500-4000/month, is now about 1/10th that. But my figures may be radically out of date. And, for every extremely high rate user on FIOS (say 20-100Mb/s) you have for your 'unlimited' service, there are thousands stuck at <5Mb/s. So it all works out.

The metrics are all there. Say you want to start up a server plant, from ground zero, not go to one of the major providers (like Usenet-News or others and resell), and figure that since usenet is pulling in some x gigabytes/day, how much space you'd need to provide x days of storage. Then crunch the numbers.

Or go to the one of the major suppliers (like Usenet-News) and see what they're selling to the re-sellers for.

Right now, I'd say unless you're independently wealthy, that doing a start-up is not worth it vrs buying from any of the major guys. The two independent biggies in the US, Astraweb and Giganews, were started a LONG time ago, so have a lot of the infrastructure bought and paid for, and things like the telecom links were 'plowed in' a long time ago as well. Just see where they are: Astra right in the heart of San Jose, Giganews started in Austin, TX, moved to Phoenix for several years, then moved to the 'loop' outside Washington, DC, in Virginia (known as 'telecom row') with the wealth of MCI/Sprint/ATT/Verizon fiber plowed there.

It's a good business, makes tons of money. Even at $10/month peruser. The costs of doing business (the servers and the storage) are continuing to drop like a rock. The amount of user bandwidth (to your home users) is not really expanding, in fact, it may be shrinking (look at what the king of bandwidth, Verizon, is doing right now to FIOS plants, selling a boatload of them off to other telecom's that can't SPELL FIOS).

And the cablecos continue to put 'caps' in place, as do other telecoms with DSL, of all things. They buy bandwidth at wholesale prices (or build it themselves), and sell at retail. OR squeeze that retail.

There are a lot of things in the mix. But 'right now', things look VERY VERY good for usenet providers.