Usenet how it works? download, upload, DMCA

[Shinok]

Hi,
When file is uploaded to usenet, will all servers connected to usenet download this file immediately, or only when file is requested by user?

I found some old usenet map, so if file is uploaded to usenet.net (it is connected to giganews), if usenet.net receive dmca complaints will be file deleted from server where it is reported only or also giganews too?
How about other servers "astraweb", "euroaceess dutch servers" ....?, they use their own email for report files.

[NNTP]

First... there are no files on usenet. File uploads will be split into chunks called articles, each one identified by a unique message id.

When an article is uploaded, the provider stores it and pushes it to its peers who will do the same. This way, an article uploaded to one provider usually is copied to every other provider within seconds (YMMV). Complaints or take down notices, on the other hand, are not distributed automatically but have to be sent individually to each provider. However, there are only about 6 providers, some of them owning several server farms. If and when the takedown happens depends on policy and local law. An article deleted on provider X might still be available on provider Y.

Forget the map - it's confusing and misleading. There's no central "usenet" and the "connections" are random and don't represent existing peerings.

[Shinok]

thanks for clarification

[piercerseth]

First... there are no files on usenet. File uploads will be split into chunks called articles, each one identified by a unique message id.

When an article is uploaded, the provider stores it and pushes it to its peers who will do the same. This way, an article uploaded to one provider usually is copied to every other provider within seconds (YMMV). Complaints or take down notices, on the other hand, are not distributed automatically but have to be sent individually to each provider. However, there are only about 6 providers, some of them owning several server farms. If and when the takedown happens depends on policy and local law. An article deleted on provider X might still be available on provider Y.

Forget the map - it's confusing and misleading. There's no central "usenet" and the "connections" are random and don't represent existing peerings.

Great summary. Also, providers can pick and choose which groups they decide to keep in their spool respective of the full feed. Looking at you Astra and the *town groups.

[DngrMs]

And 'peers' can create/service 'peers' that are invisible to the 'providers' so the existing network is already much wider than documented.

Once an article is propagated from source it's infinitely available irrespective of the take-down effort.

[Beck38]

The 'pushing to peers' is properly called 'propagation'; there haven't been any documented problems with that system for well over a decade, and it was pretty minor and was fixed quickly (it was a Giganews problem).

Virtually every server plant out there now has 'unlimited' retention, as far as the newsgroups they carry, and back to whenever they decided to stop 'rolling off' articles. For most of the majors (Giganews, Astraweb, Blocknews, et. al.) that means around August 2008 (plus or minus a month or few weeks).

The biggest problems with this amount of retention is that the two largest 'open' search engines out there (Binsearch and NZBIndex) don't actually go back anywhere near that far back, which means if you're searching for something that was posted a LONG time ago (and which may be available on many servers) won't show up.

There are a few ways around this, particularly newsreaders (Newsrover for one) who contract with servers (in Newsrovers case Giganews) that allow a search of their entire plant for what you may be searching for (one must however keep a 'subscription' current to use this capability). If you have a Giganews account (at any 'level', even the minimum), their 'Mimo' application can do the same (but obviously the lower levels that don't have the same retention level won't be able to d/l the thing from Giganews, which is how Giganews differentiates between those 'levels' of service). I think there are some other newsreaders out there that use the same search capabilities, with whatever plant's they contract with ('Superseach' and the like).