'Zero Day' Hasn't Budged, Probably won't for 'Forever'

[Beck38]

A few articles of late trumpeting '1000 Days Retention'....

http://www.slyck.com/story2170_The_March_to_1000_Days_of_Usenet_Newsgroup_Retention

But as I pointed out a while ago, the real 'figure' should be on which day (approximately) that a retention goes 'back to'.

The actual 'day' is a bit different for every server plant, and of course there are various 'fades' ('gaps') in the retention, but it generally goes back to around August, 2008, or thereabouts.

As long as a plant doesn't suffer a major, unrecoverable, loss (pretty near impossible with multiple plants on multiple continents), and even with usenet continuing huge leaps in traffic (while storage costs continue the downward spiral), we should be sitting here 3+ years from now with articles proclaiming '2000 days retention!'.

[zot]

the real 'figure' should be on which day (approximately) that a retention goes 'back to'.
In theory, that should be a fixed-in-stone date that never changes. But in reality it has changed somewhat. Highwinds recently "back-spooled" retention to Aug 2008. About a year ago, Readnews also back-spooled about two months onto its retention. And both providers have in the past seemed to have had retention erode - and then return - at the outer limits.

"Day-Zero" was August 7, 2008. That's the age of oldest binary post that any provider (in this case Giganews) has on its servers. August 15, 2008 is the next oldest date - originally Astraweb's alone but now shared by three separate NSPs (and maybe more on the way?)

Similar to Moore's Law, usenet retention (measured in days) had been roughly doubling every year for the last decade. But according to that formula, once retention surpassed a year, the yearly increase would need to become greater than a year's retention -- so the "point of no return" was reached in 2009, and from that moment on there would be infinitely expanding retention with no end in sight. So articles will never again need to be retired off the major's servers.

I'd like to make another prediction:

At some point in the future all this massive retention will become worthless. As removing all this material is as easy as shooting fish in a barrel, more and more files in usenet's back catalog will be DMCA'd until practically nothing is left.

Then it will be back to the old days, and we'll once again be required to grab stuff fast before it gets deleted from the servers. I'm sure the main reason this has not happened up until now is because the copyright cops have had much bigger dragons to slay. But as usenet continues to emerge from the shadows and become a mainstream file-sharing mechanism, it's head will eventually be on the chopping block some day.

[Hypatia]

s removing all this material is as easy as shooting fish in a barrel, more and more files in usenet's back catalog will be DMCA'd until practically nothing is left.

Then it will be back to the old days, and we'll once again be required to grab stuff fast before it gets deleted from the servers

i already posted in one topic that posters can avoid DMCA takedowns like 99% if they take certain simple precautions

BUT

Astraweb has made it simple and safe for us ATM and at the same time has spoiled all usenet community.

[heiska]

the real 'figure' should be on which day (approximately) that a retention goes 'back to'.
[snip]
I'd like to make another prediction:

At some point in the future all this massive retention will become worthless. As removing all this material is as easy as shooting fish in a barrel, more and more files in usenet's back catalog will be DMCA'd until practically nothing is left.

Then it will be back to the old days, and we'll once again be required to grab stuff fast before it gets deleted from the servers. I'm sure the main reason this has not happened up until now is because the copyright cops have had much bigger dragons to slay. But as usenet continues to emerge from the shadows and become a mainstream file-sharing mechanism, it's head will eventually be on the chopping block some day.

Yup, and the USPs could be forced to block uploaders of new infringing content (this has already happened with some USPs) after the clowns have DMCA'd all the good shit. After all, like ~70-80% of the content there is posted by like a few dozen of uploaders.

And no, AW is not safer than any other USP having multi-million dollar infrastructure physically located in the "land of the free".

[Beck38]

I'd like to make another prediction:

At some point in the future all this massive retention will become worthless. As removing all this material is as easy as shooting fish in a barrel, more and more files in usenet's back catalog will be DMCA'd until practically nothing is left.

You know, I've searched and searched for thing that have been 'DCMAed' and found.... nothing. Now, maybe it's because the things I search for (lossless/pureCD music, HD movies) are not on some 'hit list', but I would think not. Sure, there have been some 'high profile' take downs, but for every one of those, there are literally thousands (tens of thousands?) that haven't been. It shows me that the effort being put toward the current process is... next to nothing.

I'm sure that those files you keep and eye on for scope of retention aren't PD, so how long have they been up without being acted upon... those 1000 days. If I was someone paying someone to sent out takedown orders, I'd fire the folks who obviously aren't working worth anything.

So, except for that 1/1,000,000th of one percent, it's not hardly worth talking about, it's below the general 'noise' level.

[Hypatia]

I wouldnt say that "it's not hardly worth talking about" in terms of all console games, HBO tv series, many mainstream movies.

[nntpjunkie]

I don't think the Usenet is going anywhere any time soon and I don't think we will see any major noticeable clamping down on user generated content. The notion that binaries will get DMCA'd to death is not a very likely reality - looking into the future I think we are far more likely to see the NSPs of the Usenet take us into the next evolution of paid for and legal content consumption. If we consider for a moment that all the great building blocks for an extremely awesome content distribution system are already in place in terms of speed, storage, reliability and content availability, the reality of the next step would seem obvious once the old and busted copyright laws have been rewritten to account for the internet and the realization that buying plastic disc to watch a single movie over and over seems a dated concept. Once the MPAA and the RIAA realize that we no longer live in an 8-Track, VHS world then the Usenet will be the obvious go to to allow them to deliver massive amounts of content at a subscription price that millions would jump at the chance to pay. We already pay for access - Usenet has the makings of a awesome White Hat backend for stellar content.

[Beck38]

I wouldnt say that "it's not hardly worth talking about" in terms of all console games, HBO tv series, many mainstream movies.

Okay, since I don't do 'games', perhaps those publishers are much more aggressive. I'll grant you that.

But ever since I dropped my HBO (and Cinemax) subscription several (7+) years ago, I've been 'sucking up' many the series (first in SD, then nothing but HD the last 3+ years), and I'm not that 'quick' on getting them (I even go several weeks with Bill Mahr's program unless it 'makes news'). Never have had a problem, and there is usually at least a half dozen or more postings/entries on several usenet 'tracking' sites.

And since the weekly NFL preview program moved over to Showtime this last year, the same with it. Zero problems for years with HBO, and no problems with Showtime. Maybe there's some folks lurking out there to DMCA the things, but from what I see, again, pretty piss poor if they are targeting anything.

And, really, in the last 5 years of d/l'ing BR movies (both full/ISO and MKV recodes), I've yet to hit anything that, after getting the NZB, storing it off on my HD for weeks/months before deciding to grab it, no problems with anything. I tend to think, that with Netflix, Blockbuster, etc., that it's really a problem, except that those rental outfits are being slow to handle a lot of what's actually available on the consumer market (taking weeks/months to actual have a certain BD movie).

[Hypatia]

blurays are mostly not affected for some reason

as for HBO i guess its pure luck on your side. Just yesterday i tried 5 nbzs from HBO shows(true blood), via nzbmatrix - all dmcaed =(
dmcaed poorly( only first 1-2 blocks are removed) but not all of them can be repaired.

[zot]

Some software companies have been hitting usenet hard for years. Not so much the cheap shareware stuff - I'm talking about the professional/specialty software that costs thousands of dollars per license. Here's the situation: If you download it off torrents or ed2k, you're sure to get a nasty letter from your ISP, and if you download it off usenet (Highwinds in particular) every copy will likely give you an "article not found" error. It's funny that the same title that's been completely scrubbed off Highwinds servers can be found in abundance on file-hosting sites - and probably, I'm sure, other lesser-known usenet providers.

Now that I think about it, there have been a few titles that were missing from Binsearch (I checked several times) but present on other NZB search engines. Could be just a glitch ... or perhaps a takedown.

[Beck38]

Exactly what 'causes' the missing files, whether bad uploads, bad servers, bad downloads, or 'take-downs', is always going to be a question.

'Back in the Day' (i.e., before back checking uploaders like JBinUp), and still today with a huge amount of traffic coming off the various PowerPost types, unless the uploader follows 'good practice' that part of the equation is still a goodly percentage of problems, and perhaps that's why the DVD (as opposed to BR/HD stuff) seems to have a larger 'total' share of problems (those BR/HD files are HUGE, and using older posting s/w is problematic, I know from personal experience).

The second part, the servers, has gotten to the point of near zero, at least the best ones. And D/L'ing has gotten very fast and very good as well, leaving the 'take-downs' the last part, perhaps the 'fills' server could help. But it appears that it takes a fair amount of figuring to really know for sure if something actually did.

[mjmacky]

I'm still using Camel PP, has there even been any new posting software lately? Actually what's the newest one that you're using and is it still being developed? I remember when doing the research on all of the posters, the selection seemed to be pretty shite.

[zot]

@ mjmacky - if you might want to look through old discussion threads from a few months ago, I believe Beck has posted results of his rather in-depth research on usenet posting software.

[mjmacky]

I kinda remember those... didn't I even post in them? Anyways, I guess that answers the question of stagnancy.

Edit:
I just checked it and the recommendation was Jbinup... I just don't know about it

[Disme]

Now that I think about it, there have been a few titles that were missing from Binsearch (I checked several times) but present on other NZB search engines. Could be just a glitch ... or perhaps a takedown.

Have been experiencing the same phenomenon ... searching on Binsearch ... nothing ... searching for the same string on another indexer ... bingo ... it's there :blink:

[Beck38]

I kinda remember those... didn't I even post in them? Anyways, I guess that answers the question of stagnancy.

Edit:
I just checked it and the recommendation was Jbinup... I just don't know about it

Once they got it to work and stable with Astraweb (they use an older form of indexing due to their s/w plant), development kinda slowed down to a crawl, but all the stuff based on PowerPost (Camel Systems included) really only are tweaks around the edges.

JBinup is, I believe, the only 'block level checking' uploading s/w out there, It really is the only choice, particularly when dealing with large/verylarge files at high/veryhigh speeds. Some don't like the fact it's Java based, but like Sabnzbd,, works just find by me.

[mjmacky]

I thought Sabnzbd was python, am I confused about something? I think I'm going to try it out, when messing around with sabnzbd I realize one of my uploads is really messed up. It comes up as having all parts when searched on nzbindex but when I use Alt.binz or sabnzbd to grab it, so many parts missing.

So anyway if I'm peeved about jbinup I'll complain here or that other old thread.

[Hypatia]

Jbinup cant utilize my 50+mbit upload for some reason.. yet camelpost does the job right with only 12 connections
Are there some tricks?

i can get max 4.3MB with 20 connections.. and its lower than using camelsystem iwith 12 connections

and adding large releases to jbinup is a pain - kinda slow

[mjmacky]

and adding large releases to jbinup is a pain - kinda slow

When you say large releases, are you talking # of files or total filesize?

[Hypatia]

i meant adding lots of files.

but on the other hand camelsystem just doesnt let you add a large ammount of files
when u click "add" it just closes "add new upload" window lol

anyways ive tried easy usenet uplaoder, camelsystem and jbinup

EUU= 3.3MB max speed (12 was very slow, used 20)
JbunuP=4.3+ MB max(sometimes 4.5) (the same)
CAMEL=5.5+ MB max (only 12 connections)

got no idea why =)

[mjmacky]

I've got a slow upload, using CAMEL I get 300 kB/s using 2 connections to AW-EU
and I get 450 kB/s (MAXED out) using 2 connection to AW-US

I wish I could just FTP my stuff over to a box that will auto post it. My slow uploading breaks apart my releases on the auto indexers in a weird way sometimes.

[zot]

I wish I could just FTP my stuff over to a box that will auto post it. My slow uploading breaks apart my releases on the auto indexers in a weird way sometimes.

Assuming that you are posting under a unique username, search engines should compile and display it correctly. I've sometimes had the opposite problem when searching for a download: the site erroneously combines releases made weeks or months apart.

Giganews newest feature - a "cyberlocker" - might actually be a useful addition if it provided a mechanism so it could act as a usenet "seedbox" and serve as a kind of 'storage pen' between Giganews usenet servers and people's computers. That's my biggest complaint about uploading to usenet, and a reason why I don't do more: it's easy to get messed up and post corrupt/incomplete articles due to poor line conditions or whatever reason, as NNTP (which was never even designed for posting files) does not have good error-correction built-in. I'd also like the ability to upload to a remote FTP-like server which can check for completeness/errors, before finally posting to NNTP.

Another binary uploader that you might find worth trying out is Usenet Explorer. The free 14-day trial version has no time limit on posting.

[Beck38]

Jbinup cant utilize my 50+mbit upload for some reason.. yet camelpost does the job right with only 12 connections
Are there some tricks?

i can get max 4.3MB with 20 connections.. and its lower than using camelsystem iwith 12 connections

and adding large releases to jbinup is a pain - kinda slow

Nowhere within 2000miles of me is any provider, including what's left of FIOS, offers upload speeds to consumers that fast, so... :w00t: The few that offers ethernet service do, but you have to be sitting down clutching some nitro pills when they give you the quote.

But, I have my poorest machine doing duty uploading, 2.8Ghz Athlon64 single core, 2.5G ram, no problem whatsoever, barley notices when I 'add' a 50GB file (maybe 3 secs of 100% cpu), doesn't slow down the uploading it's doing at all. Maximum ram used when running, <300Mb, cpu at 2-5% avg. w/ posting speed at a bit under 5Mb/s (550MB/s).

Well, works for me. Like to 'try' it at 50Mb/s. Considering that until this last year the maximum available in my area (non-FIOS) was 10by2Mb/s, and FIOS topped out at 25Mb/s upload before they (Frontier Communications) started raising rates through the roof and then shutting things down.

[mjmacky]

OK, next time around I'm sticking with Camel PP, jbinup is completely fucking up this post. I don't know if it's an issue on the program side or Astraweb side but I've never had these problems while using CPP. This would also be the first time using SSL.
Using AW ssl-eu address. Using 4 of my 50 connections. UL speeds dropping well below 100 kB/s (when it started it was 350 - 400 kB/s).
Pausing midpost, switching server to non-ssl, non-eu server doesn't seem to be helping too much either. I was getting a bunch of these errors:

Error no. 1. occured in part REDACTED ! Details:
One Connection to Astraweb-3 blocked for 15 Seconds .
Error message: Error in method 'Senden': Connection reset by peer: socket write error

Error no. 1. occured in part REDACTED ! Details:
One Connection to Astraweb-1 blocked for 15 Seconds .
Error message: Error in method 'Senden': Connection reset by peer: socket write error

Error no. 1. occured in part REDACTED ! Details:
One Connection to Astraweb-3 blocked for 15 Seconds .
Error message: Error in method 'Senden': Connection reset by peer: socket write error

[Beck38]

Absolutely about as next to zero problems here as it can be, posting with JBinUp on Astraweb/US, I monitor post/propagation to Giga and Blocknews at a minimum, usually a third if I see any wobbliness at all (maybe once a month if that, one block error per on one of the two 'biggies'), right now I'm going on three months without a single error either posting or in propagation.... damn near flawless over 2TB+. Back in the DSL days, that would have been at least a year without a single 'wobble'.

Of course, SSL-US plant, 8 connections works fine for that (5Mb/s) speed. When d/l'ing, it takes a full 20 connections to 'flood out' the 22Mb/s connection, but SABnzbd+ is solid as a rock there on a 2core 3.2Ghz Athlon there.

I don't know why you'd be having so many problems with it. I'll presume you are using the latest build that's been around for forever at this point, 0.90 Beta 8 Build 738. I know they had tons of problems with JBinUp and Astraweb in the very beginning, due to the way Astra does their message ID's. But that was a LONG time ago now. It's as solid as granite here....

[mjmacky]

Absolutely about as next to zero problems here as it can be, posting with JBinUp on Astraweb/US, I monitor post/propagation to Giga and Blocknews at a minimum, usually a third if I see any wobbliness at all (maybe once a month if that, one block error per on one of the two 'biggies'), right now I'm going on three months without a single error either posting or in propagation.... damn near flawless over 2TB+. Back in the DSL days, that would have been at least a year without a single 'wobble'.

Of course, SSL-US plant, 8 connections works fine for that (5Mb/s) speed. When d/l'ing, it takes a full 20 connections to 'flood out' the 22Mb/s connection, but SABnzbd+ is solid as a rock there on a 2core 3.2Ghz Athlon there.

I don't know why you'd be having so many problems with it. I'll presume you are using the latest build that's been around for forever at this point, 0.90 Beta 8 Build 738. I know they had tons of problems with JBinUp and Astraweb in the very beginning, due to the way Astra does their message ID's. But that was a LONG time ago now. It's as solid as granite here....

Well I ended up having ~ 260 missing parts by the end of it. It took awhile to post as the speeds kept choking. I do have the latest build, but am I missing some settings, like check headers with server, etc.?

The other thing I notice is that jbinup works differently than CPP in that with CPP if you specify 4 connections, it'll use 4 connections for 4 threads. But with Jbinup, 4 connections on one thread is what you'll get. Maybe I just need to specify more connections, I'm trying to repost to fix errors now, what a fucking mess of a post.

[zot]

The other thing I notice is that jbinup works differently than CPP in that with CPP if you specify 4 connections, it'll use 4 connections for 4 threads. But with Jbinup, 4 connections on one thread is what you'll get. Maybe I just need to specify more connections, I'm trying to repost to fix errors now, what a fucking mess of a post.

I've noticed that on all the old binary usenet clients, each rar file gets one connection ... while on the newer clients, each article -sequentially- gets one connection regardless of which rar the article corresponds to. So as a result, the first rar is completed before the next rar starts.

I've discovered that under poor line conditions, the more connections I use, the more errors I end up with. So in those situations of rampant errors, I try to use only one connection, even though it takes much longer.

SSL can also create additional errors under poor line conditions (such as if the 'handshake' procedure gets interrupted) so I disable SSL.

[Hypatia]

Judging by my experience ( and i post A LOT) i'd say that posting via properly configured Jbinup is safer than posting via any other uploading software like camelsystem\powerpost 52 connections\ easy usenet uploader in terms of getitng incompletes after one's upload process is finished
i have no idea why..
yesterday i had some problems uploading via Astraweb and though camelsystem uploaded at almost full speed many parts got screwed up. Jbinup ,despite of having some strange speed flactuations sometimes dropping to almost zero during that period. managed to finish the same upload OK though it took AW's servers some time to properly process it.

[Beck38]

Well I ended up having ~ 260 missing parts by the end of it. It took awhile to post as the speeds kept choking. I do have the latest build, but am I missing some settings, like check headers with server, etc.?


JBinUp one has the option to check headers either with the server one is posting to (standard) or an alternate server, like if one is posting to Astra and doing 'checks' on Giga, something like that. In that configuration, one is really checking the headers PLUS propagation. I checked out that feature when I originally trying the program out, but decided that using the 'standard' of doing header checks on the posting server was 'good enough' as I routinely do manual propagation checks on multiple other providers 'just for grins' so to speak.

BTW, I run at around 90% duty cycle upstream, some 20GB/day on average, so it's not the huge jaw-dropping rates others have, I doubt I'd be able to collect and properly process much more than I already am, but JBinUp chugs along with nary a whimper. As far as Astra, I do note the occasional hiccup where the connections get 'reset' at the server (couple of times every 24 hours or so), but JBinUp has handles it just fine.

Hope you get better luck with the configuration and operation...

[mjmacky]

So should I just chalk it up as having Astraweb glitches? When I review it, the SSL connections gave me MANY more errors than the non SSL connections (same # of conn. = 4). This is the most botched job I have ever seen, and I had to repost a lot of parts that came back as missing. Worse than that, there are some parts being reported by NZBIndex and binsearch as OK, but when I DL them and par2 check, there are missing parts not being reported by the indexers. I don't think I could have had a worse experience with Jbinup.

Now since I'm not the one to just bitch and complain, before I completely ditch jbinup, could you glance over my settings and tell me if there's anything I screwed up here?

http://thumbnails39.imagebam.com/13068/0d08a5130672774.jpg (http://www.imagebam.com/image/0d08a5130672774) http://thumbnails12.imagebam.com/13068/79132f130672775.jpg (http://www.imagebam.com/image/79132f130672775) http://thumbnails30.imagebam.com/13068/992a38130672777.jpg (http://www.imagebam.com/image/992a38130672777)
http://thumbnails28.imagebam.com/13068/820381130672779.jpg (http://www.imagebam.com/image/820381130672779) http://thumbnails27.imagebam.com/13068/ecf77c130672781.jpg (http://www.imagebam.com/image/ecf77c130672781)

[Hypatia]

Header check is possible with this server - ON


Use server for header check - ON

PS i wonder why do u prefer uploading via SSl server.. it cant "save" you or anything.. well. except for from ISP shaping=)


PPS ive just had "header checking" fixing my upload. for some reason te last scene got uploaded not completely as it happens with astraweb sometimes and JbinUP automaticall did header checks and reuploaded the parts that were missing.. it took some time, but now my upload is 100% complete, i love it=)

[zot]

@Hypatia

Since you tested several binary posting clients, and you also use Usenet Explorer, what's your opinion of Usenet Explorer's posting ability compared to Jbinup and other posting clients?

From Usenet Explorer's site:



Free poster

The program contains free text/binary poster, also related functionality like newsgroup list related
is free. The reason for including it as a free tool is existing widely used poster 'powerpost' is
terrible by introducing options that may puzzle text newsreaders by forming invalid subjects and
arbitrary placing references which isn't used by binary downloaders anyway and has backlog of all
kinds of problems.

The poster can post par2 sets, nzb and sfv files, for posting par2 it seamlessly uses par2 command
line client.

Other features are reposting segments/attachments, retrying posts reposting only failed segments,
post task persistency between sessions, simultaneously posting several posts each to its own server
(but still not suitable for sending spam), remembering prior posts etc.

See the post menu and edit menu->properties->posts for related settings.



It seems there are no specific "header check" settings to adjust, but I assume this may be set to a built-in default value. I'm not sure exactly what all of UE's settings do; the documentation is vague and there appears not to be a lot of discussion about Usenet Explorer.



So should I just chalk it up as having Astraweb glitches? When I review it, the SSL connections gave me MANY more errors than the non SSL connections

It's not just Astraweb. Also HW and RN - and probably all others. SSL can be very demanding of having a rock-solid connection - or it craps out. At least that's been my experience. That leads to a dilemma I often find myself in when on a shaky public Wi-FI connection. Use SSL - and risk posting errors. Don't use SSL - and risk having my password harvested (the lesser evil since I use a nearly-depleted block for posting).


there are some parts being reported by NZBIndex and binsearch as OK, but when I DL them and par2 check, there are missing parts not being reported by the indexers.

Could it possibly be that the binary attachment is missing from the post but everything else is there?

I wonder if a mal-formed post will get recognized by some servers but not by others?

[mjmacky]

I didn't come across anything before regarding Astraweb and header checking with Jbinup, so thanks for the clue about that. I will give up on SSL, I was just trying it out (and only for part of the post). It's not like I was using SSL with CPP before, so no big difference there if I ignore it.

Next time around with Jbinup I'll see if my post gets raped or not. I'd also like to hear personal feedback on UE.

[Hypatia]

what's your opinion of Usenet Explorer's posting ability compared to Jbinup and other posting clients?

ive never used it.

I prefer to use software that is designed solely to posting stuff.

but now that you've mentioned it i might give it a try someday =)

On a side note UE is an awsome newsreader=) a bit confusing at first but..=)

PS ive just tried posting via UE and my speed is slow.. maybe im doing something wrong dunno)

[zot]

In Usenet Explorer, it helps to create another account just for posting, adjusting the number of connections to optimize the upload rather than the download speed, disable SSL, etc. (I upload with a lot fewer connections than I download with)

UE is the only newsreader I know of that was designed for posting binary file sets (although it indeed lacks some of the 'tweaking' settings found on other, dedicated posting clients). All the other newsreaders are really only capable of posting single-file attachments, so for those users a dedicated poster would be essential for uploading big files.

[mjmacky]

JBinup... Everything seemed to go more smoothly this time, and the header check works out just fine. There is a missing part that it's not recognizing, but it's on a repair volume so I just don't care. However, I was dealing with 1 split file rather than a bunch of split files, so it's hard to gauge how much my experience has improved, I'll keep testing it.

[Hypatia]

zot i tried everything i could. i still cant get more than 2.5-2.8MB uploading via UE ..

[zot]

Hypatia - I can only dream of getting a 2.5 megabit upload speed.

With any posting client, there's obviously inherent upload speed limits. For instance, the files can only be yenc-encoded as fast as the computer (and the software code's efficiency) will allow.

UE is said to download faster now for users with ultra-high speed connections since it now processes articles completely in memory rather than writing them to disk like it did before. Perhaps the same sort of 'streamlining' revamp might be needed on the upload side of the software. Or maybe it's the 'handshaking' with that particular server that's causing the bottleneck.

I think that's a problem the developer might want to know about, as it's possible UE may never even have been tested on such high upload speeds.

[Beck38]

For those beating on JBinUP, trying to get it to work or work better, last evening (my time, around 13+ hours ago from the date on this message), I glanced at my upload machine before hitting the sheets and instantly noticed that Mr. Astraweb was seriously trying to veer off the rails (either that or the something in the path, of course), where JBinUp was reporting tons of connection errors, timeouts, you name it, to the point where the rar parts (fairly large, though) had about a half dozen of them 'awaiting verification'.

I gritted my teeth and watched for a bit; it took a good half-hour plus but eventually all the rar parts that hadn't been verified did get the 'magic checkmark', and the errors reported by JBinUp settled down; and in checking both Astraweb and the propagation out to other servers, all was good an hour later.

Pretty good. That kind of 'veering' would have nailed the PowerPost types, and created what I call a 'fade' (huge parts simply not received, and therefore not propagated).

[Hypatia]

Beck38
i had exactly the same problem on Astraweb ,i guess, at approximately the same time period while uploading 24Gb release.
speed was very inconsistent, few first episodes got screwed up severely, but when i got back from work everything was posted OK and the ones that got screwed up too.
Ive decided to increase header check interval to 240 sec just to give enough time to propagate..


That kind of 'veering' would have nailed the PowerPost types,
yeah, i had this problem with camelpost.
at some point i even considered getting myself $5 giganews account just for uploading( you can upload unlimited ammounts and no incoming traffic at all, i tried it once.)

[Beck38]

Since GN doesn't have block accounts (so long ago I can't remember when they dropped having them), I still have a very low $3/month account they don't sell anymore (greedy b*sta*ds), but I guess they have let me keep it. Unfortunately, if I ever pull it up (I did so a couple of times after switching to Astra just to get some things that were too corrupt elsewhere, but that was well over a year ago), I really keep it simply to 'check' propagation.

I guess I could use it for posting, but I have a very old block account with Astra that can't be traced to me whatsoever (bought with untraceable Visa gift card some 8+ years ago) and my traffic goes through a randomized VPN, so...

Anyway, back in the day (2 years ago) when my speed was slower than dirt (768K DSL that actually topped out around 600Kb/s) such 'fades' really gritted my teeth. When my DSL provider took a hike, I switched to the only competition, and both the VPN and a much better uploader was necessary. I did around 500GB of testing, with the various PowerPost types, and fairly quickly figured out that they were 'not up to the task' unless I wanted to continue to track these 'fades' but at a much faster (6-7x) rate.

Yikes. No. Every once in awhile I'll 'see' a skip, very very small. But right now, I'm going about 12 weeks without anything, during which time I've transferred better than 2TB. Bloody Good.

[mjmacky]

It would appear that my timing seemed to be the biggest issue when testing Jbinup initially with Astraweb. The more I test it, the happier I am with it. I'd have to say my initial problems were MOSTLY with AW (as well as SSL).

[Beck38]

Tonight (okay, say around 0300GMT) I started my weekly upload 'routine', to find that AstrawebUS was quite literally dragging. So much so I did a Speedtest to San Jose, CA., which told me that the entire path was just peachy.

Hmmm. So, I let it grind away a bit (a half-hour or so) and things settled down (or, I should say settled UP speed wise). Things happen, and I first think about my ISP (cable), and the path, but Speedtest quickly dispelled that, Of course, anything can happen at all during the 'night' as long as things don't completely come apart (it did once some 6 months ago, both my posting machine and another did a 'reboot', of course wasn't a power thing as everything is on a nice large UPS....)

But at least it came right back up to speed tonight, and hopefully it'll stay there. I know folks here tend to go ding-dongs when things get wacky, and to a bit so do I. That's why JBinUp has been a bit of a godsend, locking down that upload path. I'm kinda surprised that another type of block checking uploader hasn't come out of the woodwork, particularly with the jaw-dropping speeds that are available around the world (particularly in both Europe/Britain and the Far East/Japan/Korea).

[mjmacky]

Well I wouldn't call it a godsend. I haven't quite figured out if it is a propagation issue yet, but I'm still getting missing parts on the download end (when I'm checking), even though header checking and auto indexers are telling me it's OK. When I try to repost those parts manually, it's kind of hard to force jbinup to get started. I have to force stop and start over and over again before it does anything.

[Beck38]

When I did my initial testing, the one 'wobbliness' that JBinUp exhibited (at least for me) was in trying to post to servers that were very far away from me, either my hop count (tracert) or by ms delay (ping). but as I needed to use a VPN to encrypt/hide my traffic, I simply choose a port on the VPN network as close as possible to the server I wanted to post to.

That is actually reverse from what is 'normally' suggested by the VPN folks, they want your port to be as close as possible to you; but I quickly found out that for usenet, the reverse worked perfectly. As Astraweb/US is, in San Jose/Santa Clara CA, a stones throw away (and some 6 hops) from the VPN plant in SanFran, my tracert to Astraweb is 8 hops total, with a ping delay averaging <50ms. So, it's quite literally as if it's 'right next door', as if I still lived back in Alameda County (on the north east side of the bay).

Those were the 'good old days', late 90's, making an almost obscene amount of money (apartment and home prices were also 'obscene', but had full-duplex T1's in every unit!), and gas prices were 'already' at $4 a gallon (! GD California!). That and when the dot.com's started to fail left and right, one didn't know if the company one was doing contract work for would exist the next day or week.... Fun times!

Anyway, I tried Giganews (server in Virginia/DC Suburbs) and a couple others, but it seemed as soon as that hop count/delay ramped up, JBinUp would get 'squirrely'. Even tried Astraweb/EU (Amsterdam) but the trans-Atlantic hop was too much for it. So, you might take a look at your actual connection, and see if it's got a really tight 'lock' with the posting server from your location.

[zot]

It would be nice if most text newsreaders had Jbinup's post-checking features. It's frustrating to write out a long post, only to have it disappear into the ether when posted to the NNTP server. :)

I assume that the reason why VPNs would be recommended to be physically located close to the person using them is to reduce the length of unencrypted traffic. I've noticed though, that encrypted traffic seems to have a harder time being transferred through the internet, maybe because encryption tends to clog up deep-packet inspection servers, or whatever the case, but encrypted traffic is often slower. I hate it that some DPI-using ISPS lump all encrypted traffic in the 'p2p' category and severely throttle it.


In the late '90s crude oil dipped to under $10/barrel, so something must have been seriously screwed up in California if gasoline was $4/gal -- it was under $1 in other states (and much of that was taxes).

[Beck38]

I assume that the reason why VPNs would be recommended to be physically located close to the person using them is to reduce the length of unencrypted traffic.

Actually not; the traffic is encrypted either from the originating PC or from the encryption box; I use both methodologies, as the encryption box is extremely stable but a bit slow, maxing out around 5+Mb/s but covering my entire network at once (and my upstream speed limit) but doing it on a machine level (multi-core) easily swamps my receive level (22Mb/x).



In the late '90s crude oil dipped to under $10/barrel, so something must have been seriously screwed up in California if gasoline was $4/gal -- it was under $1 in other states (and much of that was taxes).

You have to remember that California is, when it comes to energy, another country (NOT the US). The gasoline/oil formulations are very different (smog), and require highly specialized refineries. This is why the crooks at Enron targeted it when doing their 'trades' and such; that and a goodly percentage of the electricity comes from the Pacific NW (Columbia River, Federal Bonneville Power Administration) so they had to 'buy off' the Republican FERC Federal Regulators, which was proven long ago by the Public Utility District that serves me today (phone taps) when we had a Democratic state AG.

We have a Republican AG today who refuses to press charges (although the evidence was collected many years ago), although some of that material was used to convict and send to Costa Rica the Enron crooks. This is just like around the country where folks voted these guys in on a 'jobs' platform (or in this case a 'law and order' platform) only to find out that they had a different agenda to pursue once they actually got elected (other states, spend all their time blocking abortion, going after teacher unions and such), but here it's not going after the real criminals, because they're 'politically connected'.

The current 'spike' in gas prices today is a minor 'bump' in prices in California, especially around the SF area. That's why when you see stories on the national news the pain is in rural mid-west areas, not really in Cal. even though the commute is usually terrible. The same thing with home prices, although we're supposed to be in a 'double-dip' in that, the prices were so artificially high to begin with that only minor reductions have taken pace during this recession.

I count myself very lucky in that I sold my home in 2003 when I took very early retirement, bought a duplex (rental rates have continued to go up during the recession), and I still can (eventually) retire to the coast if I choose in a couple more years.

Of course, we could have a nice earthquake, which would throw a wrench into that plan (!)