Please explain difference in file sizes, headers, usage, etc.

[JabbaWockee]

This is the first time I've used a block account, and I having a hard time calculating my usage for a nzb listed as 100.41 MB. I downloaded this thing because it was a roundish number and I thought I'd be able to see how things are calculated. nzb is John Kelly Only One Reason. I have no idea what it btw. I haven't yet contacted the provider. Help me make sense of it all.

A nzb file: no pars, yEnc (if this matters?)

-Listed file size from nzb site/sabnzbd+:
100.41 MB

-Difference in quote from usenet-news after download:
0.1090 GB (100%)
0.0981 GB ( 90%) what's counted

-Windows properties folder size:
97.3 MB (no warnings in sabnzbd+ so i presume nothing was missing)

About usenet-news:
-10% of what you download isn't counted towards stats.
-1 GB PROBABLY equals 1,000,000,000 bytes. In other words 1GB != 1GiB. It's not mentioned.

You only download headers when you want to manually browse the newsgroups, correct?
Not when you're loading a nzb in sabnzbd+, for example?
Is there some sort of overhead when converting the newsgroup posts into binary or something? yEnc?

[zot]

Usenet bandwidth will always be higher than filesize because of 1.) transfer overhead, and 2.) encoding overhead.

First of all, file size and bandwidth are never going to be equal. Downloading a file -- using any method: FTP, HHTP, Bittorrent, etc -- will by necessity add some form of additional overhead in network transfer, as the packet contains more than just the payload.

Additionally, keep in mind that text-encoding is required when posting a file on NNTP -- that is converting a binary into text format so it can be posted on usenet as a series of individual "articles". Encoding adds overhead. Yenc is by far the most common for files posted on usenet, and this encoding's overhead is very small, under 5%, but older, less efficient encoding types, such as MIME, UUencode, base64, etc, can add up to 20% overhead to a file size.

To get an accurate count of actual bytes transferred when you downlaod a file, you can use a bandwidth meter, or, much easier, just log into the provider's site to check bandwidth stats (though these are often delayed by several hours).

The 10% extra that Usenet-News gives for free was a policy change made about 2 years ago to compensate users for their decision to cancel the free headers they used to provide. I never understood how Usenet-News could sell 2GB blocks with free unlimited headers -- as a full header load of big groups like a.b.boneless can easily add up to over 2GB alone. (Most ISPs and Dutch pay-providers only offer very limited headers, maybe 2 weeks or so; therefore I suspected a lot of users would get Usenet-News' $2/2GB blocks for the lifetime supply of headers alone. Those who thought they could leech headers forever from Usenet-News ended up getting shafted.

The odd thing is that copycat companies like Blocknews (a virtual clone of Usenet-News formerly-unique business model) also give an extra 10% -- even though Blocknews was started after Usenet-News corrected it's "mistake" of offering free headers.

Instead of downloading headers in your newsreader, one roundabout way to get free "headers" is to go to any usenet search engine like Binsearch.info and select "browse" and the site's webpage will display the (pre-grouped) headers (about 100 per page). (Since some clients like SabNZBd and Ninan don't do headers at all, users of these "newsreaders" will need to use an indexing site to browse headers.)

Just one last thing: If you download headers at all (useful for browsing single-part posts as in MP3 groups) be sure to use a client that supports header compression. Most newsreaders don't, but Newsbin and Usenet Explorer support compressed headers, which cuts bandwidth by nearly 90% over downloading uncompresed headers.

[JabbaWockee]

Thanks. I appreciate your post.
I don't plan on downloading headers with this block account. I do that occasionally on unlimited plans, when also I have a lot of nzbs saved up.
I prefer newsleecher over newsbin. Does that compress headers?

[c0ld]

newsbin was one of the first to accept astrawebs compressed headers iirc. newsleecher doesn't do it atm afaik, but there's a workaround;
http://filesharingtalk.com/vb3/f-general-newsgroups-79/t-newsleecher-supporting-compressed-headers-astraweb-xzver-359869

[zot]

No, Newsleecher does not support compressed headers, but they have been promising this most-requested feature for over a year. Newsbin Pro and Usenet Explorer are the only clients I know of that support compressed headers. If you tried Newsbin but didn't like it, have you tried Usenet Explorer?

An alternative to having a compression-supporting newsreader is to use Stunnel, a free proxy application (SSL wrapper) which works on servers which support SSL-ZLIB compression -- which Usenet-News does, as do other Highwinds server resellers. Stunnel works with most newsreaders.

Using headers can very helpful for browsing small, single-part content-specific groups whose material is generally not reported on NZB sites -- for example, a MP3 group of a type of music you like. Of course, it would be a huge waste of bandwidth to download headers for groups like alt.binaries.hdtv because the files are so large and the content is much quicker and easier to browse on NZB sites. A big file like a blu-ray rip, when posted on usenet, might constitute a hundred thousand individual headers and articles (with each 'rar' part-file itself containing possibly several hundred.)

I now use block accounts exclusively, and I even download headers on providers that charge for header bandwidth. The key is using compressed headers, which download very fast and eat up practically no bandwidth.

@c0ld

Thanks for the input. There are several different types of header compression. To make matters even more confusing, different servers use their own customized versions of a compression method. The software posted in that thread by Zrubavel only supports Astraweb's XZVER compression. Usenet-News (Highwinds) has never supported XZVER header compression, but does support SSL-ZLIB compression which can be utilized by the Newsleecher + Stunnel combination.

[JabbaWockee]

Would you say usenet-news, blocknews, and astraweb are the top 3 providers for block plans?

[dette]

Would you say usenet-news, blocknews, and astraweb are the top 3 providers for block plans?

I would yes

[solar_king]

Usenet Explorer supports Astraweb, Giganews and SSL compression plus the "standard" diablo server compression, the latter is used e.g. by UNS, Easynews, Newshosting, Eweka, etc.

NewsBin doesn't support the "standard" header compression, so if your server is UNS you won't get compressed headers with NewsBin.

NewsLeecher - doesn't support any including SSL compression, it uses SSLv2 which doesn't include compression option.

Not every server has SSL compression enabled, because it also compresses downloads, not only headers; most servers which have header compression enabled disable SSL compression to avoid double compression which is wasting CPU time.