many downloads incomplete - astranews

[mysticbertie]

many of the 1080p movies i download are missing many blocks and require many par files to repair them, some cant even be repaired with the provided par files, sometimes if i re-download that specific rar file ie no 57 58 59, that sometime helps but sometimes fails too.

is this down to astranews, these files were only posted a few weeks ago so not a retention issue.

should i try another news server and whats recommended around $10 a month

:dry:

[cola]

Astraweb tends to be a lil finicky. I would try setting up the EU server as your backup, or US if EU is your primary. This fixes 99% of incompletes. I also have a small block acct that picks up the rest.

[mysticbertie]

lol i said astra news duuh

i had the sever set for the general server so now its the eu one, how do i set up the us one as back up and should i be using the ssl secure server?

[cola]

what software are you using?

[mesaman]

I'm seeing 100% correlation for US and EU, in other words what's missing on US is also missing on EU. If the post is in that age range, a few weeks old to several months old, it's best to use another provider until Astraweb can fix this mess.

[cola]

Could it be DMCA takedowns? I know giganews has that issue, maybe Astraweb is having to comply more.

[Malcontent]

A few Astraweb users over at NZBMatrix report the same issues. At least one user has contacted Astraweb about the problem and reports that Astraweb tech. support acknowledge that there is a problem and that their working on fixing it.

User=origitat


I emailed Astraweb tech support and they told me that they were aware of the issues and were trying to address them. Anyone else get info from tech support?

Here is the ticket response I got:

Hi Matt,

We are currently working on these issues and we hope to fix it as soon as possible. At the meantime you may try again or perhaps switching between US and EU servers or working on different ports to see if it helps.

Sorry for any inconvenience caused.


Thank you and have a nice day!

Regards,
Kylie

[cola]

Weird that they suggest switching ports could alleviate that problem.

[Malcontent]

http://helpdesk.astraweb.com/index.php?_m=news&_a=viewnews&newsid=53


Incompletes & Article Not Available

We are looking into and fixing the issue with article completion for articles at and around the 110-130 day old mark. This issue is unrelated to the Network Instability (http://helpdesk.astraweb.com/index.php?_m=news&_a=viewnews&newsid=52) issue posted earlier today.
Completion will return gradually as the articles are recovered.
We regret and apologise for this incident, and our engineers are working with urgency to rectify the problem.

[sandman_1]

Use all 3 servers they give out and spread your connections across all of them equally. I do this in Newsbin Pro and don't have any issues completing articles. Also your AV could screw up things too so make sure your news client has full access.

[ThoR52]

Same problem as the OP here. I've tried both EU and US servers, and it doesn't change anything, i'm still missing parts while the post appears to be complete.

[mysticbertie]

Use all 3 servers they give out and spread your connections across all of them equally. I do this in Newsbin Pro and don't have any issues completing articles. Also your AV could screw up things too so make sure your news client has full access.

I DONT KNOW HOW THIS IS DONE IM USING NEWSLEECHER

oops caps off sorreeeee

im using eset security so it sets up everything automatically

[sandman_1]

I DONT KNOW HOW THIS IS DONE IM USING NEWSLEECHER



You can read can't you? Amazing what you can find going to their website and actually searching it.

Found this in two seconds, http://www.newsleecher.com/forum/viewtopic.php?t=17974



• Click on the Manager page tab
• To add a new server, click on the Add button.


Post a link to one of the files you can't get/complete and I will see if I have the same issue with it.

[mysticbertie]

You can read can't you? Amazing what you can find going to their website and actually searching it.

Found this in two seconds, http://www.newsleecher.com/forum/viewtopic.php?t=17974



• Click on the Manager page tab
• To add a new server, click on the Add button.


Post a link to one of the files you can't get/complete and I will see if I have the same issue with it.

sarcastic báwbág i was away searching it, if you have issues helping people then please refrain from doing so:lol:

i had to delete this as it was incomplete but i cant remember what specific files were missing blocks.

http://www.binsearch.info/?b=am.2010.1080p.BluRay.x264-REFiNED+refined-made.in.dagenham-1080p&g=alt.binaries.movies&p=meck90btn%40hotmail.com+%28Gladiator%29&max=250

[sandman_1]

sarcastic báwbág i was away searching it, if you have issues helping people then please refrain from doing so:lol:


I guess it was a bit harsh. However I was able to find the information on their site quite easy. I will try that file at the link you posted and see how it works out.

Noun
bawbag (plural bawbags)
(vulgar, slang, pejorative) scrotum.

Hmm, how colorful.

ok downloaded it. It had parts missing in the rars and even pars but Newsbin Pro was able to repair it. Binsearch said all parts available so definitely parts missing on Astra's servers.

[mysticbertie]

sarcastic báwbág i was away searching it, if you have issues helping people then please refrain from doing so:lol:


I guess it was a bit harsh. However I was able to find the information on their site quite easy. I will try that file at the link you posted and see how it works out.

Noun
bawbag (plural bawbags)
(vulgar, slang, pejorative) scrotum.

Hmm, how colorful.

ok downloaded it. It had parts missing in the rars and even pars but Newsbin Pro was able to repair it. Binsearch said all parts available so definitely parts missing on Astra's servers.

my you can be helpful when you try lol :P

the term bawbag is used widely in these parts, appropiate when someone is harsh :naughty:

thanks for checking, just confirms the problem lies with astraweb, i have added the servers eu ist us 2nd and general 3rd, i will monitor it over the coming week and see if its any better. :yup:

oh and thanks;)

[mjmacky]

For specifics, try this:
http://filesharingtalk.com/nzbs/tv/x264720/30323-The-Killing-S01E01-E02-PROPER-720p-HDTV-x264-ORENJI
or this:
http://filesharingtalk.com/nzbs/tv/webdl720/30629-The-Killing-S01E02-The-Cage-720p-WEB-DL-DD5.1-H.264-TB

It definitely seems related to time period

This is the first time I've ever been unable to grab something using Astraweb

[Beck38]

Did a little checking, as far as one can 'out on the edges' so to speak.

First, the upload was 100%, as it was to Astraweb/eu, and the propagation was 100% out to the peers. Okay, the non-Astra peers, as both the Astra/EU and US are missing more that a fair enough parts to deep six the thing, even though the Pars should have been acceptable at ~10% as well.

But I did notice that the posting was to two newsgroups (a.b.tv, a.b.hdtv), and the posting was via some program called 'newsmangler 0.02', which did raise some eyebrows a bit, particularly as the '.02' hasn't been worked on since (from what I can find out) some 10+ months ago.

But then again, I don't trust anything but JBinup anymore. And that only as far as checking the outgoing headers from Astraweb...!

[mjmacky]

Did a little checking, as far as one can 'out on the edges' so to speak.

First, the upload was 100%, as it was to Astraweb/eu, and the propagation was 100% out to the peers. Okay, the non-Astra peers, as both the Astra/EU and US are missing more that a fair enough parts to deep six the thing, even though the Pars should have been acceptable at ~10% as well.

But I did notice that the posting was to two newsgroups (a.b.tv, a.b.hdtv), and the posting was via some program called 'newsmangler 0.02', which did raise some eyebrows a bit, particularly as the '.02' hasn't been worked on since (from what I can find out) some 10+ months ago.

But then again, I don't trust anything but JBinup anymore. And that only as far as checking the outgoing headers from Astraweb...!

Did you see the ones I mentioned, the collections being hosted here?

I remember saying I was happy with jbinup, but everything I've posted since I started using it always ends up missing some parts and the header check never catches it. I've tried reposting, but then everything gets kinda funked up. So jbinup has left me a bit wanting. I've just stopped posting altogether cause I've had it with incomplete uploads and slow upload and connection dropping. I had already uninstalled CPP and don't feel like setting it up again, so I'm just kinda stuck.

[Beck38]

Yes, that's what I did the checking on, those two files you listed.

I've had next to zero problems with JBinup over the past 10+ months, over 4.2TB transferred through Astraweb/US, with only minor wobbles (fairly quickly auto-fixed), with about as zero problems with propagation (a few weeks back Giganews was the most major off-track problems I've seen all year, and it 'eventually' fixed itself over about a week).

The posting to two newsgroups is what really caught my eye, doing so has really not been necessary for close to ten years now, since the server plants no longer allocate space by newsgroup.

If JBinup is set up to check the server you're posting to (it can be set up to check propagation to another server), and reports things are 100% but upon checking headers you find either the posting server or the propagation is skipping (random parts unaccounted for) or fading (huge chunks of parts over a fair amount of time), then something is going off the tracks somewhere at the posting server, since it has reported to JBinup that things are '100%'.

I'd be looking for a different server. Maybe I just lucked out with Astraweb/US. 'Internet hop wise' it's just very close to me (although some 500+ miles away), all the other server plants in the US are on the opposite (east) coast and are 10-15 hops (and large delays at 3000+ miles) away.

But while posting I watch these things like a hawk... Then again, it's more of an 'industrial process' to me and not an occasional thing. Like I said, I'd be looking for another posting server (knock on wood that Astra?US keeping running good!)

[mjmacky]

But while posting I watch these things like a hawk... Then again, it's more of an 'industrial process' to me and not an occasional thing. Like I said, I'd be looking for another posting server (knock on wood that Astra?US keeping running good!)

Yeah I've tried all 3 AW variants, with no resounding success. I also can't really watch them that closely, as they can take up to 10 hours to finish 1 post. I wish I could FTP these things somewhere (with better upload) that will then get posted to usenet. There's no equivalent to a seedbox (temporary) on usenet.

[Beck38]

It takes me upwards of 3 days to post a 'collection'... 10 hours would be a breeze. Following the posing via a standard newsreader is easy, double-checking via Binsearch and others I can do from a smart(er) phone that has good web access.

If you have a hosting company near to you (I have one some 10 miles distant) you can set up a remote controlled server that has access to super-fast internet at very low rates (>$10/month; but you have to be really serious to do that. I'm still thinking about fiber, which is in my area, but I'd have to move and right now it isn't worth it.

Meanwhile, as long as things seem to be working okay, I guess I'm just lucky.

[Hypatia]

i wish i could help with ur problem.. but once i switched from camel powerpost to jbinup i havent had problems while uploading stuff via AW.
I set checking to 5 min interval.
Uploaded about 400Gb in the last 30 days. All seem to be OK.

Also you should take into account that those index sites they basically download headers, then delete them after a certain period of time has passed
When AW had problems with headers most of my uploads were shown as incomplete on almost all index sites that have a header feed from AW. yet the files were there complete. supersearch still shows some of my uplaods incomplete when in reality theya re all there.

[mjmacky]

Well mine have all shown to be par2 repairable, but it's just kind of annoying that they all have to be repaired. How many connections are each of you using in jbinup?

[Hypatia]

i set 20 connections. For some reason i need much more connections to achieve normal speed when uploading unlike when downloading

SP i really wish someone made alternative to jbinup not using Java.. i dont like java software for various reasons

[mjmacky]

Wow, maybe I should use more, I limited mine to 2 to reduce number of errors

[Beck38]

I did a whole lotta testing at first... But found that I easily swamped Astraweb/US at 8 connections with my current pipe (5Mb/s). I could probably cut that down to 4 but found that things ran 'smoother' at 8 (to my eyes). But then again, I use a VPN so that enters the 'mix' as well, and the VPN/router and circuit, with my ISP throughput, seems to find that 'sweet' point at that 8.

Unless and until I get a better 'pipe', then I'll have to look at some h/w upgrades (VPN router needs to get faster), and that is all bucks, and since the fiber folks have called a halt to any further expansion toward my current location (2-3 miles down the road), it'll be another year at least before that (and unless the cableco drops their rates or ramps up their speeds, which may happen according to DSLReports).

[Crack0saurus]

This thread doesn't surprise me.

Lately i've had lots of incomplete files (most times too much for repair) in a.b.hdtv.french with french 720p/1080p bluray rips between 100 and 300 days.
Thought it was dmca-like demands at first but several days later tried to download again and it was much better.

Still have some issues with some posts from around 110 days but i guess i'll ty again later. Just glad to see it's a technical issue and not compliance with some BS organization.

[Beck38]

I looked back at my records, and despite posting virtually every hour/day during that time frame, saw absolutely nothing wrong at the time.

In a couple hours I'll have a window to d/l a couple things that I posted, and know that they were 100% both on Astra and on the other servers I monitor. If they are loosing things on their drive plant, and both plants at the same time. then there's real questions. I already verified that some things that were posted to Astra (not by me), are missing huge chunks, but were propagated just fine out to other servers, which is weird to begin with.

Well, this didn't take long, but...

Okay, VERY interesting.... A Posting I know to be 100% on both the US and EU server, during the period we are talking about (around the first week of April, 2011), are both semi-hosed.

What is particularly weird, is that both files have errors is pretty much EXACTLY the same spots. Now, since I know that both were 100% when posted, the possibility that both server plants, separated by some 8000 miles, would go off the rails in pretty much the same way.... means a couple of things (maybe).

Astraweb could be doing what I suggested, cross-checking the two plants against each other at whatever intervals. And one of those plants in experiencing drive errors, and the cross-checking routine isn't checking the files (par or whatever) before 'updating' the other plant. So error's 'proliferate', which is exactly what one DOESN'T want happening. The possibility that both plants would suffer pretty much the same disc error's, affecting the same files, is astronomical.

Either that, or someone they are peering with is overwriting GOOD files with BAD, and their s/w is allowing that to happen (VERY BAD). I'll bet that's what happened, and if they don't fix their s/w, it'll happen again. Or it already has, except it's a small error or folks simply haven't noticed it yet.

[loubarlow]

Well it's not getting any better is it? Nigh on a week and still I can barely download anything over 100 days old.

[Beck38]

Not that I can see. The 'barrier' at the top (most recent) end is around 83 Days (9 Apr 11), things newer than that are okay (as far as I can tell).

From 83 days to around 300 (2 Sept 10), pretty much hosed. The missing parts do die off in a slow curve, going from the really bad in this April to just a bit towards that 300 day mark. Since I just completed that testing, I don't know if the plant is 'improving' at all, will have to wait and re-do things a bit in the days to come.

Of all that I can accurately test beyond the 300 days looks okay, but like I said there is a 'curve' of missing parts from this April going back, and by the time one gets to around 200 or so it's pretty minimal, IF the poster put up a decent amount of pars. But, I tested my own postings, which I know were as 100% as possible (JBinUp and header tests on multiple servers both Astra and Giga) when posted. Like I said, if it was worse I don't know, we'll see in a few days if things start improving, and from which 'end' (newest or oldest).

[zot]

I never came across this problem at all. Maybe because I had been using Binload for everything 400 days and less (and getting 100% completion) and saving my AW block for the older posts - which only had an occasional incomplete.

The last time users were reporting that Astraweb had a serious completion problem was around Christmas 2009. (I was too busy to investigate it then, and now this one pops up at another major (US) holiday) Judging from the last meltdown and its eventual resolution, I'm guessing that it might take AW a few weeks to back-fill all the missing parts, we shall see.

[Beck38]

That last 'major' meltdown I remember well, and to my mind they really didn't recover completely from that; I believe it was related to some plant upgrades that went awry. Others have also commentated elsewhere that they see skips and such on 'known good at one time' postings going back 250+ days, so one can hope that they don't simply fix the 'minimum' range and leave it at that, which has been the result in the past.

Again, I find it of interest that both servers would go off the tracks in virtually identical ways; one hopes that that is at the top of their priority as to investigation.

In doing some re-tests today, no difference in number/severity of skips, in the 'hit zone' of 100 days; the 'roll-off' seems to have changed, but it's within the 'margin of error', no real change.

Doesn't look like any work is being done.

[Beck38]

4 JUL Report - No changes, no fills, no corrections noted

[Hypatia]

you should send them email.They should be bombarded with emails of the same type: FIX UR SHITE ASAP FFS! :D
of course all the necessary data to back up this statement(you have problems, guys) should be there as well

[Beck38]

They have been, and have a 'note' (wasn't this previously listed?) up saying they are 'working on it'.

http://helpdesk.astraweb.com/index.php?_m=news&_a=viewnews&newsid=53

"Incompletes & Article Not Available
Posted By: Steve On: 26 Jun 2011 5:17 PM
Details Incompletes & Article Not Available

We are looking into and fixing the issue with article completion for articles at and around the 110-130 day old mark. This issue is unrelated to the Network Instability issue posted earlier today.
Completion will return gradually as the articles are recovered.
We regret and apologise for this incident, and our engineers are working with urgency to rectify the problem."

I'll continue to check known original 'completes' at that 100 day mark, ones further back around 250 days, and those around 75 days ('gap creep') and for both any additional gaps or any 're-fills', if any. So far, none noted.

As both plants have pretty much the exact same gaps, they can't use one to 'cure' the other, and that being said, the core of the problems (I think) is one of their peers. Identifying which one, then fixing the actual data, is going to be a big project.
Unless this happened really fast, all at once, means that their watchfulness was a bit too minimal (I ran a usenet server plant for close to three years in the late 90's).

Good luck.

[Malcontent]

http://forums.newsbin.com/viewtopic.php?f=20&t=28479#p175011

by stevef
(http://forums.newsbin.com/memberlist.php?mode=viewprofile&u=3718&sid=1713b3bff2963ad6057901fb06bfa98b)Official Astraweb Representative



The wheels are in motion. We had some recovery early last week, then a few days of non-recovery later last week as replacement hardware was ordered. The long weekend doesn't help.
We expect the hardware and other things to be in place in a few days, and the recovery will continue then. I'm not intimate with the recovery process, but I'll post more if and when I get more details.

The post it near the bottom of the page.

[Beck38]

That's interesting that they are going down the h/w failure route, but most of the plants these days are built using some pretty specialized h/w. I did have some (somewhere) some interesting pictures of some SATA Rack-Bays and such...

So, I guess I'll wait until the end of the week (TH/Fri) before running any more tests.

[loubarlow]

Has anyone figured out a formula to discovering what age of articles are affected? Is it between a certain timescale or *all* articles of a certain age?

[sandman_1]

by stevef


We are well aware of the problems and we are fixing it. The estimate for the window of incompletes may have been a little off, but it was an estimate. What is important is that we are addressing the incompletes, regardless of whether it is 1 day or 100 day window.
Although the barrage of complaints is well deserved, it does not help to speed up the recovery.


I would disagree with stevef. IDT he has heard of the term, "The squeaky wheel gets the grease." I think if anything, the more complaints, the faster they are going to fix it. If they start pissing off tons of their customers where they might jump ship, you bet that will light a fire under their ass to fix the problem quickly. And I think Quade needs to sthu sometimes. If they are advertising a certain percent completion and people aren't getting it, then they have every right to complain.

[Beck38]

Undeniably, even though they knew of the problem(s) several days before the 4th holiday, obviously that slowed down the response if they needed to do a fair (or more) amount of work in that area (and getting a vendor to go faster is a big IF under those circumstances), then if they can get things started to turn around by the end of the week is pretty fast.

I don't think they could do it much faster; the only thing is if they had found out the problems earlier. They need a better system rather than relying on their 'customers' as to things going 'off the tracks'.

[loubarlow]

I downloaded 5 items yesterday and although a couple needed significant repairing, all were successful, including a couple around that magic 150 days mark, so maybe things are slowly improving.

[Beck38]

Probably not; most of the RAR's that had serious skips and such could have been repaired IF (a big IF) they had enough pars to do so; it's one of my particular 'bitches' that with the (generally) improved propagation and retention of most major providers these days, particularly compared to 'years past', that it has lulled a lot of posters to 'ramp down' the percentage of pars they include in posted archives.

'Back in the Day' (10+ years ago) that percentage was at least 20%. Despite having much faster connections, and much larger archives, many posters have pulled down to 5% or lower. That has proven to be an 'accident waiting to happen'. With the majority of 'skips' I've looked at (including my own postings), if 10% had been supplied, virtually all would be easily 'fixed'.

What's interesting to me (as well), is the insistence of many posters to include multiple groups in posting (like a.b.hdtv along with a.b.blufay or the like). Again, 'back in the day/, most providers allocated space on a 'per newsgroup' basis, and that resulted in having to do this multi-group posting routine. No providers plant s/w does that today that I know of, and so posting to one group is sufficient, it helps not one wit to do multiples (in fact it may be detrimental). Just my 2 cents.

From the 'word' we are getting, it will probably be at least the end of the week before things improve. I have my records and a 'baseline' of errors to go back and look at, to see if/when things start to do so.

[Hypatia]

What he said, guys!
im damn glad we have you here. =)

PS to be honest,10% par2 takes some time and not every poster has a powerful PC...
i myself have switched from 10% to 6%
the % doesnt matter when it is 700Mb that needed to be PAR2ed ,.. but if its something considerably larger)2gb+) than it becomes nuisance.

[Beck38]

Update 7/9 02:30 UTC: Did some short tests on known good/bad files, no change from earlier in the week.

So either they haven't done any upgrades, or haven't started to repair the original damage.

[zot]

As for the ideal percentage of pars, I think it probably makes little difference in the long run. This is because, I strongly suspect, most NSPs probably aren't going to get up and take action until the level of incompletes on the servers exceeds the (currently about 5%) PAR-repairability limit - and their customers start leaving in droves.

That's my theory on why Highwinds' completion on older files (which I both tested and complained about fairly extensively) has been a constant 98% to 99% for many months; I finally came to the conclusion that it would need to get worse before it has any chance of getting better.

It's even possible that Highwinds might have more incompletes than Astraweb, but because Highwinds' missing articles are spread out over a much greater range (+/-500 days vs. +/-100 days for Astraweb) that means most of Highwinds' damaged files can be par-fixed, and so -- unlike Astraweb -- Highwinds' problem never rises to a level that corporate management would consider critical.

[Beck38]

A good portion of it is, we have gotten, over the past three years (oh, since the retention ramp-ups of mid-2008), had it pretty good. More than pretty. That's why I made the comment that (posting) folks were getting a little bit complacent in ramping down the amount of pars a bit too much on the lean side (<5%) especially when the transmission pipes (your ISP) has (generally) gotten much faster over the years. Post 10%, how much time/effort extra does it take, on both 50GB postings and especially on 1-2GB ones.

Just about any 'examples' of Astra skipping files would easily be repaired in place IF they had 10% pars. So I think you're just about correct, we've all been spoiled by how well things have, overall notwithstanding the occasional veering off the tracks that we experience. I just think back to my first introduction to usenet circa 1987 or so, on a 2400baud modem. Rock bottom Stone Age.

How's things going, Mr. Flintstone?

[Hypatia]

It's even possible that Highwinds might have more incompletes than Astraweb,

Astraweb had had the best completion+speed+price right till the moment they had some hardware failure.
Highwinds was nowhere near unlike readnews(yet slow speeds on "old" and inconsistent speed on new)
As for now its hard to say what is the best choice..
because mark my words- this is the day they will be talking about for years - "Giganews Strikes Back" lol

supernews+ blocknews account== very neat, indeed. considering their speed< stability, and now retention+price. what is not there(dmca) blocknews will cover.
i would say it beats usenetnow\ngroups+something. though that is if you dont have more than 50mbit

[mysticbertie]

i have a crappy internet download speed anyway so which news server should i try as im fed up with astra news, i cant be bothered writing to them i would rather just jump ship.

had more files incomplete and wont repair.

rubbish

[zot]

If you don't want to (or don't know how to) set up a fill-server using Blocknews.net or Usenet-News.net to pick up Astraweb's missing parts, then these would be my choices in a (similar-monthly-priced/similar retention) Astraweb alternative:

Supernews.com:[/*] outstanding completion, but a lot of copyright takedowns.

UsenetNow.net or Ngroups.net:[/*] good completion, fewer copyright takedowns

You can also use a trial Binload account (400 day retention) as an Astraweb backup.

[mysticbertie]

If you don't want to (or don't know how to) set up a fill-server using Blocknews.net or Usenet-News.net to pick up Astraweb's missing parts, then these would be my choices in a (similar-monthly-priced/similar retention) Astraweb alternative:

Supernews.com:[/*] outstanding completion, but a lot of copyright takedowns.

UsenetNow.net or Ngroups.net:[/*] good completion, fewer copyright takedowns

You can also use a trial Binload account (400 day retention) as an Astraweb backup.

i dont know how to set up a fill-server i use newsleecher, is it easy?

usenetnow is only $12 a month so i might just go with that.

cheers matey.

[zot]

i dont know how to set up a fill-server i use newsleecher, is it easy?

usenetnow is only $12 a month so i might just go with that.

cheers matey.
Yes, Newsleecher supports fill servers. In the window where you put in the server information, check off "use this server for fills only" or set a server priority list, something like that. I still have block accounts I bought over 5 years ago -- the download gigs never expire until used.

Ngroups costs less than UsenetNow, uses the exact same backend servers. But I think UsenetNow also has a $10/month special floating around somewhere.




posting to one group is sufficient, it helps not one wit to do multiples (in fact it may be detrimental). Just my 2 cents.


I've always posted music in multiple groups: the specific genre groups as well as a more 'blanket' group like alt.binaries.sounds.mp3.complete-cd. This is because I believe that a lot of people might browse specific groups looking to sample music from new artists they've perhaps never even heard of before. And obviously the more (relevant) groups something is posted in, the more likely people are to find it via browsing headers/filenames. (just my own 2¢)



'Back in the Day' (10+ years ago) that percentage was at least 20%

I remember that a dozen years ago, completion ranged from bad to horrible - even just the next day after something was posted. Having 15% or 20% pars made sense then. But today, going from 5% pars to 15% pars would mean losing over 100 days of retention. Personally, I'd rather have those extra 100 retention days, since incompletes can be completely avoided simply by piggy-backing block accounts. But since the vast majority of Usenet users appear not to know this "secret" of using fill-servers - then having a higher par percentage would no doubt save a lot of people from undue anguish and misery.

My biggest complaint is with the people who post things -- most notably MP3s and full albums -- without ANY pars, and this example of ignorance is sadly all too common.

[mysticbertie]

Yes, Newsleecher supports fill servers. In the window where you put in the server information, check off "use this server for fills only" or set a server priority list, something like that. I still have block accounts I bought over 5 years ago -- the download gigs never expire until used.

Ngroups costs less than UsenetNow, uses the exact same backend servers. But I think UsenetNow also has a $10/month special floating around somewhere.

well it sounds easy enough setting up the fill server, either way i need to sign up for something new so it may as well be a new news server, ill see if i can find the usenetnow 10$ special if not ngroups $11 deal.

thanks for your help ;)

i found the deal for $10.39 so signed up for that.

[Beck38]

I've always posted music in multiple groups: the specific genre groups as well as a more 'blanket' group like alt.binaries.sounds.mp3.complete-cd. This is because I believe that a lot of people might browse specific groups looking to sample music from new artists they've perhaps never even heard of before. And obviously the more (relevant) groups something is posted in, the more likely people are to find it via browsing headers/filenames. (just my own 2¢)

Uh, when this latest problem surfaced, there was a flurry of 'headers.. who cares about headers' type messages, so who exactly is following any specific group at all any more? People use whatever search engine is in front of them, either on a site like this, nzbmatrix, newzbin, or one of the myriad of usenet search engines. Of course, yes, that means that folks won't 'trip' over something, but then again, I would think with music it's just like films, folks rely upon their favorite site or something to give them a heads up, then go out to the search engines to see if it's 'available'.

As to pars, I like to think with the cost of the server plants 'keeping up' with what's being posted, and the 'zero day' being July/August 2008, that in the next couple of years, usenet will get ever closer to what P2P is as far as having just about anything one could want. I hope that the 'work' I do will never go away, period. And posting a far number of pars will help that process no matter what kind of heavy crashes occur (hopefully).

[managedhybrid]

Only one download so far today has been corrupted on me. I have been with Astraweb through months now (7+ months) I cannot complain for the price I am more than happy.

[Beck38]

When this 'day 100' problem first surfaced, I immediately went to posting I had done (and verified on several servers including Astra at the time I posted) and d/l 3-4 of them w/o major problems - yes, there were a fair number of 'skiips' and such, but as all of those rar archives were 10% par'ed, they only needed a slice of those to fully repair. Where folks got into trouble was with files that were around 2-3% or so: not enough pars to repair. :(

[hdjunky]

i found the deal for $10.39 so signed up for that.

This is the link I found. http://usenetnow.net/ngr.html. I think they also have a 6 month special but I could not find it quickly. Been using the service for a long time and never had any problems. Blocknews is another o
Way to go if you wanna keep astraweb just put it in your reader as backup.

[Beck38]

Well, as the date/time stamp on this message, no change I can see in Astraweb performance regarding skipping around 100 days.

Don't think anything has been done yet.

[Beck38]

Another day goes by, no change either in the 'target zone' on what is now ~150 days, or anything from about 100-300 as well.

I think if they get past another weekend (and they haven't posted any 'updates' on the situation on their website), things are going to get more than a bit bad with their 'customers'.

[Hypatia]

We have been recovering data by group. The most popular groups are being recovered first, then a full sweep of less popular groups will be recovered.
If you want, pm me the nzb or group and I can see what the status of your group is at.

oh well.. i wonder how long will it take to cover at least 50% of lost data

[Beck38]

If so (and that quote isn't to be found on the Astraweb site, btw), I'm monitoring the three top usenet groups (top as in the amount of traffic in GB as listed by several tracking sites), plus a smattering of others that were listed at the top of this thread when things were first noticed 'going off the tracks'.

The only notice put up on the site so far as to any problems, was exactly one month ago. No updates, no nothing, since. So I opened another ticket asking 'what's up?'.

Like I said, it's been a month since the last 'official' pronouncement. The skips, according to folks here and elsewhere, starting showing up probably two weeks before that. The fact that both server plants, US and EU, were affected pretty much exactly the same, is to me the biggest clue as to what happened, either one plant ran off the rails and then propagated the failure to the other (an original h/w failure made worse by a s/w that failed to do proper checking before propagating it to the other server), or a upstream peer that nailed both servers simultaneously.

But I'm sure by the end of the days, folks simply want it fixed. From what I see, not anytime soon, and pretty much becoming never. Sad.

Got a reply to my message on their site. 'Will be fixed real soon now'. ha ha

[Beck38]

As of some 5 hours ago (from the date/time of this message) still nothing changed.

Folks are getting antsy (uh, me?), and there are several folks/threads elsewhere, where folks are reporting problems where there doesn't seem to be any (mine and other confirmations of such), where users are making simple mistakes and then blaming it on Astraweb.

Again, they better fix things soon, like right now, or it'll snowball.

[cola]

Do you have an NZB that you know won't work on Astraweb? I'd like to test around a bit.

[Beck38]

Lots been listed here, just look at the first few posts.

The last 'news' posting by them was on 26June, saying that the gap was centered around 110-130 days, which would have put it around the end of February. Since that notice was just shy of 30 days ago, the gap as moved (time marches on), and center should be around 160 days back now.

I use the 'mergekit' HD postings as a way to see what's going on, since I d/l'ed them all (through Astraweb/US) at the time them were posted; therefore I know they were 100% at that time. Generally the 'mergekit.C' posts are the smallest, and any skipping is a dead giveaway as to problems with their plant.

However, tonight I notice that Astraweb/US is not adding any files, either ones that are posted at them (they are being propagated out to other servers, through) or ones coming from their peers. That tells me that 'maybe' they may be working on the plant (finally). FYI Astraweb/EU is not being affected. That tells me the 'front end' of the plant is still up and running, but the 'back end' may be shut down for 'maintenance'. One can hope, we'll see by tomorrow some time.

[cola]

I was hoping something for something more specific. I tested a handful around 160 day mark and couldn't find anything. I did find 2 that were had incompletes around the 70 day mark. EU server didn't seem effected. Interesting part is that using different ports gets different incompletes. Setting the EU server as the fill server seems to almost get around the problems on the US farm.

[PastTense]

" or a upstream peer that nailed both servers simultaneously. "

I don't understand this at all. The theory is that you have multiple peers. Each post is going to be offered to you by multiple peers--you take it from the one which offers it to you first. So if there is an upstream peer which isn't offering posts, you just get it from someone else. And a major site like Astraweb should be very well connected.

So could someone explain this?

[Beck38]

Depends on how your s/w is set up. It used to be you defined a 'trusted' peer, and got 99% of your stuff from there, and only any others if it was something 'new' and unseen.

Still no recovery as of today, I'm seeing more and more folks dumping them and going with whoever.

[zot]

I don't understand this at all. .... So could someone explain this?
Just speculation.

I can explain what I think the problem might be:

The major companies like Highwinds and Astraweb appear to use some kind of automated software that back-fills missing articles between the US and EU servers. I've noticed that in many cases the US and EU server will be missing the exact same articles on older files -- files that are 100% complete at other companies' backends. Steve at Astraweb has stated in the past that they back-fill missing articles across the Atlantic between server farms, but I've never seen him post the details of how this is done. (trade secret perhaps?)

But anyway, it must be some sort of automated process, because there are far too many articles on a server to cross-check/back-fill manually (and let's assume that they don't have a huge staff).

Because both of Astraweb's server farms were suddenly afflicted in the exact same 100-day post range, I strongly suspect that the problem was a result of a malfunction of this automated cross-checking/back-filling software that Astraweb presumably uses.

Highwinds appears to be set up in a similar way: all older (500+ day) files are incomplete, and the US and UE servers both have (mostly) identical missing segments.

This is why it's so important to set up a block account (that uses another backend) as a fill-server. In the old days (3+ years ago) it seemed that server farms were completely independent, and using a provider's EU server as a backup to its US server would usually deliver 100% completion on old (non-DMCA'd) files. But today, that's no longer true: whatever is missing on one server will typically be missing on the other.

[cola]

[QUOTE=PastTense;3597319]
In the old days (3+ years ago) it seemed that server farms were completely independent, and using a provider's EU server as a backup to its US server would usually deliver 100% completion on old (non-DMCA'd) files. But today, that's no longer true: whatever is missing on one server will typically be missing on the other.

You'd be surprised, looking at one file around 70 days ago, I got these results using the NZB completion checker:

This was the US server on port 119
http://i.imgur.com/ILcfq.png

Port 23
http://i.imgur.com/TRVTM.png

Port 1818
http://i.imgur.com/2FCRG.png

Port 8080
http://i.imgur.com/itKt1.png



EU Server, only port 119 had an incomplete
http://i.imgur.com/bMaVS.png

[zot]

Thanks, Cola. I'm completely stumped on why the port number can make any difference at all -- assuming that your result is consistent and repeatable across multiple trial runs.

Since we know the same backend is used regardless of the port setting, that would of course suggest that the article exists on the server, but is somehow not being reported correctly.

Have you tried checking the completion using SuperNZB? It uses a different method, which appears to be more accurate, faster, and overall superior in my opinion.

http://filesharingtalk.com/threads/435072-SuperNZB-client-mini-review

[cola]

I wasnt that surprised after Malcontent on the first page was told by astraweb to try different ports. I was a bit skeptical, but shit. I've seen it happen on one other nzb. Still took about a dozen NZBs to find 2 with missing parts, but I think par2 still would have fixed it. I might try some more tomorrow with SuperNZB.

[zot]

I didn't put too much weight in that at the time, since tech support guys can often just shoot back the same standard responses (try SSL, try different ports, etc.) to a lot of different problems, since ports are so often blocked or throttled by ISPs.

But this is something quite unique. I suppose on a newsreader that allows multiple fill-servers, a person could set up a separate server field for every port number, and get near-perfect completion using that string of multiple port numbers.

Seems like the completion issue is more of a software bug than anything else.

[Beck38]

Seems like the completion issue is more of a software bug than anything else.

I think that's pretty obvious. I'm 'fighting' at least three separate problems caused by lax programmers, on things as diverse as Mr. Astraweb, to my computer network, to my media system.

Add to that my 'smartphone', where the vendors doing 'apps' want to be paid not just twice (the paid service the app connects to, and the end user/me) but by the phone manufacturer, and now the phone carrier (four 'streams' of profit) If the manufacturer refuses to 'pony up', they change the code slightly to make sure the app you bought a year or two ago to stop working. If the carrier refuses to pay their kickback, the app simply stops working in it's tracks.

Everybody is getting too greedy. What's going on with Astraweb is particularly troubling, as I know that the s/w they are running is 'home grown', not anything from a s/w development company. So, who they're paying to maintain that s/w is a good question.
For the thing to feed differently according to the port one connects to is beyond bizarre. Major problems.

For several years now, I've been telling folks the story of the last time I spent time living in Silicon Valley (1998-2001); all over the place programmers were getting laid off, and replaced under the H1B visa program with Indians who worked for 1/4th the price (which is illegal, but who was cracking down on it? Nobody). The companies would put up these folks in 1 bedroom apartments, 4-5 per, with a pocket of bus passes to get back and forth to 'work'. Then the dot-com crash. Care to guess what contributed to that?

Now this morning my malware program needed updating, that I just spent a couple days on two weeks ago cleaning up a problem. Now it has the same problem. Don't these folks TEST their product out before releasing it? Huh? Nope. 'My code don't stink' is the typical refrain. Yes it does, now I'll spend another couple days cleaning up after their current mistakes.

[cola]

I finally pulled the plug on astra and jumped on the supernews bandwagon. Retention is pretty much the same, price is slightly better but with a more reliable feed.

[Jackbauer]

I use to have problems all the time with astraweb. Then I got sabnzbd. I tend to download all around the retention sphere, and in the last 6 months have only had 1 failed download (some lazies don't upload any pars...) yet have downloaded over 1.5 terabytes.

Strange that I've never even run into the issue they confirmed... I have sickbeard auto downloading new tv shows yet never have had one of those fail either...

My advice is try a new client like sabnzbd before giving up. And lowering connections can help too

Sent from my Droid using Tapatalk

[cola]

Clients aren't going to fix server side issues.

[Beck38]

in the last 6 months have only had 1 failed download (some lazies don't upload any pars...) yet have downloaded over 1.5 terabytes.


many folks here do that in less than a week.

Still no change whatsoever on either US or EU servers as of the date/time on this message. As of the coming Monday it will be a full month since Astraweb acknowledged the problems with a 'news' item, and they were advised of the problem by many users at least two weeks previous.

I think that if you can, and it works for you, it's way past time to jump ship, or have a nice large bucket of fill-bits from another provider (like Blocknews).

[Hypatia]

to jump ship

to jump where exactly?

Highwinds? Nope. Hopeless

Supernews? DMCA , one needs a fill account too

Readnews(resellers? Slow speed

it is still the same 2 choices

supernews+ blocknews
astraweb+blocknews

well maybe readnews(ngroups\usenetnow) if you dont have 50+mbit plan
downloading files past xxx days is a pain on a fast connection

[zot]

Also, depending on what you download, it's quite possible that a Blocknews fill account would last much longer on Astraweb right now than on Supernews.

re: jumping ship. ....

Sometimes the only thing that will force a company to correct a problem is if enough people leave. That's obviously not the case here with Astraweb, but I strongly suspect that a mass-exodus might be the ONLY thing likely to get Highwinds to fix its incompletes that exist month after month. That was indeed what it took for Highwinds to finally get serious about Easynews' meltdown 2 years ago, after several months of runarounds and excuses.

It would certainly be helpful if Astraweb at least gave a weekly update, and posted general advice on AW's "system" page. It should not be necessary for every single person to (separately) email support to get answers for a common problem.

And for some reason Steve will post on the Newsbin forum info that never goes up on the Astraweb site.

[cola]

Fill servers are going to be a reality now that DMCAs are being used more frequently. Theres no way around that, unless you're using a dutch provider. I don't know of any dutch providers that have 1000+ days retention though. Its not as though block accounts are that expensive. What would worry me is if blocknews starts having the same DMCA holes as the other big providers.

http://www.supernews.com/super-special/

[Beck38]

It would certainly be helpful if Astraweb at least gave a weekly update, and posted general advice on AW's "system" page. It should not be necessary for every single person to (separately) email support to get answers for a common problem.


Which I have told them to do twice now, with no real response. Why more info is on a 3rd party (Newsbin) forum rather than their own is really stupid.

And probably yes, a Blocknew 'filler' would probably last longer with Astraweb than Supernews, but it all depends on what you are grabbing. I used up a significant chunk of a 100GB Blocknews 'block' in the last couple of weeks grabbing some things (successfully) I wanted, but it amounts to a hidden 'tax' on using Astraweb.

But I've run lots of studies on using anything other than Astraweb, and found every time that because of the fact I'm on the west coast of the US, and Astraweb is both very close to me both physically, hop wise, and delay wise, that all the other server plants in the US are clustered generally around Virginia/DC or on the east coast, they are all problematic and would require more additional hoop jumping for me to use on a continual basis.

So I'm stuck. Folks in Europe/Britain don't really understand the size of the US, it's the equivalent of someone in London told to use a server out of Moscow.

[mesaman]

Giganews has a server in Los Angeles.

[cola]

that all the other server plants in the US are clustered generally around Virginia/DC or on the east coast, they are all problematic and would require more additional hoop jumping for me to use on a continual basis.

I'm surprised that doesn't effect your posting here.

[Hypatia]

Theres no way around that, unless you're using a dutch provider.
since when astraweb has gone dutch ? :D
they dont have dmca problems. they have technical problems lol

[cola]

They're starting to get some, but not as many.

[Hypatia]

id like to see examples =)
so far everything i encountered that is DMCed on giganews\supernews (HBO, console games) is available on AW
Except for the stuff posted during this gap due to technical failure or whatever.

[cola]

Thats part of the problem, since Astraweb has to many problems its hard to know which is due to which.

[Hypatia]

and here i thought i would get something..*sigh*

So..as i suspected its pure speculation

anyways, its not a problem at all.

Its relatively easy to distinguish dmca-ed material from simple incompletes.
Consistency matters, yes. the way dmca is done.

[cola]

Don't worry, I'm tested a bit with astra, giga, and an NL block account. Tested a popular nzb with over 1k grabs of a recent movie, both astra and giganews are missing parts, all near the end of the RARs. All parts available on an NL server. So while I don't have astraweb on record admitting to a DMCA takedown, the evidence is pretty suggestive.

[Hypatia]

So..when i asked you to give me examples. you said it would be problematic...

now you claim you tested this "popular" nzb. maybe you should get your story straight first?

is it really so hard to give me a link to this 'popular nzb"? =)
its like they say seeing is believing, you know.

whats with the people these days i wonder..

[cola]

What I meant by problematic is that with the quality of astraweb's feed so bad right now, it's a bit hard to distinguish between a DMCA takedown and just the feed quality at times. Not impossible. But if I were to claim that this NZB was DMCA'ed by Astraweb, you could easily say that its just incomplete because Astraweb's feed quality is so degraded.


80781

80782

[mesaman]

Doesn't fit any pattern of DMCA, it's random pieces.

On Astraweb:

.r36 - missing 52 and 58
.r44 - missing 32,46,60,64
.r45 - missing 32,65
.r46 - missing 27,41,60
.r47 - missing 47,49.
.r48 - missing 40

On Giganews it depends on the server. A few articles are consistently missing on the L.A. server:

.r04 - missing 17
.r17 - missing 59
.r35 - missing 17
.r39 - missing 30
.r41 - missing 27
.r46 - missing 23
.r49 - missing 1
sample - missing 19

[Hypatia]

But if I were to claim that this NZB was DMCA'ed by Astraweb

It is clearly not DMCAed/
I dunno where did u get the idea that this particular upload was DMCed.

Not even close. Nothing fits
And here i thought you knew as you claimed, the difference, between DMCAed articles and just incompletes.

just some random , small completion issue quite possibly caused during the upload process or later.

[cola]

What port, since that seems to make a difference with AW.

Also, post an nzb thats clearly DMCA'ed so we can all look.

[Hypatia]

here is one (http://nzbindex.nl/release/27812798/40245-FULL-a.b.teeveeEFNet-True.Blood.S01E10.DVDRip.XviD-REWARD-0154-true.blood.s01e10.dvdrip.xvid-reward.nfo.nzb) of the HBO tv series episodes that was DMCAed on giganews\supernews(they dmca HBO series, console games on a regular basis)

and here is (http://img222.imagevenue.com/img.php?image=136652891_54545545_122_804lo.gif) the picture that shows the pattern

1-2 blocks from every file are removed.(worst case- everything is removed, but i saw just several of the kind)

if posters make 10% par2 then in most cases its not a problem. if its only 5% par2then not everything can be repaired.

[Beck38]

Cute... but more pars does defeat it. I wonder if they 'ramp up' the deletions if they see that it does have more pars...

Meanwhile, still no improvement, Monday it will be one month since their 'official' announcement on their site of problems.

For whatever reason, in my testing, the absolute worst part of their plant is centered around 30Mar11, absolutely dreadful.

I though I could see things were actually getting worse a couple days ago, but it was well within the 'wobble' factor of my ISP connection and the error rate came back to within were it was weeks ago.

[Hypatia]

I wonder if they 'ramp up' the deletions if they see that it does have more pars...


they wont lift a finger unless they get specific details of the upload that needs to be removed in a dmca take down request

Those only 1-2 blocks removed- you might consider them as a sign of dmca mafia's watchdogs stupidity and lack of knowledge about how usenet works. or they are just lazy. =====> (http://www.giganews.com/legal/dmca.html)

Message-IDs for all articles the DMCA Notice is requesting Giganews take down. unless there is an easy way to show all message-ID of all articles at once.. i dunno about that.

[cola]

When I tested on supernews, I didn't get the same result as you did:

80851



Using Astraweb:

80852

[Hypatia]

through away whatever you are using to check atricles availability =)

NZb download checker (http://www.softpedia.com/get/Internet/Other-Internet-Related/NZB-Download-Checker.shtml) is the way to go.

You can try to actually download this nzb and you will see incompletes.

here is the same picture from supernews for this NZb
http://img235.imagevenue.com/img.php?image=439678801_4545_122_443lo.gif (supernews eu)

here is Sabnzbd report
-
85 articles were missing (supernews eu, not counting a bit of not downloaded(not needed) par2 articles)

astrawebs' stats
Files found: 51
Message-ID's required: 2894
Message-ID's found: 2894
Message-ID's missing from the NZB: 0 (0%)
The files were posted 268 day(s) ago.


Parts checked: 2894
Undownloadable parts: 0
All OK!


supenews tats

Files found: 51
Message-ID's required: 2894
Message-ID's found: 2894
Message-ID's missing from the NZB: 0 (0%)
The files were posted 268 day(s) ago.


Parts checked: 2894
Undownloadable parts: 86
2,97% of the data can't be downloaded!

[Beck38]

A bit ago (last month) I brought up here:

http://filesharingtalk.com/threads/434785-NZB-Checker-%28Download-Posting-etc%29

that what was needed was a real, windows-based, multi-provider, nzb checker. Still searching (latest to go through the grind and fail miserably is "NZB Completion Checker"). You bring up the "NZB Download Checker" (I'll assume v2.0.391), yet another program that 'says' it runs on windows, but not on my machines (probably requires some add-on that I don't have and don't want cluttering up things).

So, so far, it doesn't exist. I guess that the relative high-completion rate, ever since the bumpiness of the 'retention wars' of late 2008, that it hasn't reared it's ugly head. Until now. A decent tool MAY have given some 'early warning' as to problems. We'll never know now, though.

Come Monday (the one month anniversary of Astraweb acknowledging the problems) I'm going to send a blistering message to this chumps. Then I'll probably join the horde of others that have jumped ship.

[cola]

What version of windows are you running? Runs ok on Win7 x64.

[Beck38]

Windows 7 has been BANNED from my network, for basically the same reasons that it has from the 4 largest corporate networks within 50 miles of me. I tolerate Vista (64 is on high probation, 32 runs just about perfectly after well-documented early years of ... c**p ) because I have to run it because of various s/w that won't run on anything else (including W7).

I hold out hope for W8. But I'm an eternal optimist. But if you've take a look at the 'Win8 Phones' just reaching the market right now, any optimism might be squashed before it takes hold (really BAD).

[Hypatia]

windows-based, multi-provider
i dont want windows-based nzb checking software.

id prefer something that could run on linux as well.=)