many downloads incomplete - astranews

Undeniably, even though they knew of the problem(s) several days before the 4th holiday, obviously that slowed down the response if they needed to do a fair (or more) amount of work in that area (and getting a vendor to go faster is a big IF under those circumstances), then if they can get things started to turn around by the end of the week is pretty fast.

I don't think they could do it much faster; the only thing is if they had found out the problems earlier. They need a better system rather than relying on their 'customers' as to things going 'off the tracks'.

I downloaded 5 items yesterday and although a couple needed significant repairing, all were successful, including a couple around that magic 150 days mark, so maybe things are slowly improving.

Probably not; most of the RAR's that had serious skips and such could have been repaired IF (a big IF) they had enough pars to do so; it's one of my particular 'bitches' that with the (generally) improved propagation and retention of most major providers these days, particularly compared to 'years past', that it has lulled a lot of posters to 'ramp down' the percentage of pars they include in posted archives.

'Back in the Day' (10+ years ago) that percentage was at least 20%. Despite having much faster connections, and much larger archives, many posters have pulled down to 5% or lower. That has proven to be an 'accident waiting to happen'. With the majority of 'skips' I've looked at (including my own postings), if 10% had been supplied, virtually all would be easily 'fixed'.

What's interesting to me (as well), is the insistence of many posters to include multiple groups in posting (like a.b.hdtv along with a.b.blufay or the like). Again, 'back in the day/, most providers allocated space on a 'per newsgroup' basis, and that resulted in having to do this multi-group posting routine. No providers plant s/w does that today that I know of, and so posting to one group is sufficient, it helps not one wit to do multiples (in fact it may be detrimental). Just my 2 cents.

From the 'word' we are getting, it will probably be at least the end of the week before things improve. I have my records and a 'baseline' of errors to go back and look at, to see if/when things start to do so.

What he said, guys!
im damn glad we have you here. =)

PS to be honest,10% par2 takes some time and not every poster has a powerful PC...
i myself have switched from 10% to 6%
the % doesnt matter when it is 700Mb that needed to be PAR2ed ,.. but if its something considerably larger)2gb+) than it becomes nuisance.

Update 7/9 02:30 UTC: Did some short tests on known good/bad files, no change from earlier in the week.

So either they haven't done any upgrades, or haven't started to repair the original damage.

As for the ideal percentage of pars, I think it probably makes little difference in the long run. This is because, I strongly suspect, most NSPs probably aren't going to get up and take action until the level of incompletes on the servers exceeds the (currently about 5%) PAR-repairability limit - and their customers start leaving in droves.

That's my theory on why Highwinds' completion on older files (which I both tested and complained about fairly extensively) has been a constant 98% to 99% for many months; I finally came to the conclusion that it would need to get worse before it has any chance of getting better.

It's even possible that Highwinds might have more incompletes than Astraweb, but because Highwinds' missing articles are spread out over a much greater range (+/-500 days vs. +/-100 days for Astraweb) that means most of Highwinds' damaged files can be par-fixed, and so -- unlike Astraweb -- Highwinds' problem never rises to a level that corporate management would consider critical.

A good portion of it is, we have gotten, over the past three years (oh, since the retention ramp-ups of mid-2008), had it pretty good. More than pretty. That's why I made the comment that (posting) folks were getting a little bit complacent in ramping down the amount of pars a bit too much on the lean side (<5%) especially when the transmission pipes (your ISP) has (generally) gotten much faster over the years. Post 10%, how much time/effort extra does it take, on both 50GB postings and especially on 1-2GB ones.

Just about any 'examples' of Astra skipping files would easily be repaired in place IF they had 10% pars. So I think you're just about correct, we've all been spoiled by how well things have, overall notwithstanding the occasional veering off the tracks that we experience. I just think back to my first introduction to usenet circa 1987 or so, on a 2400baud modem. Rock bottom Stone Age.

How's things going, Mr. Flintstone?

Quote:

---

It's even possible that Highwinds might have more incompletes than Astraweb,

---

Astraweb had had the best completion+speed+price right till the moment they had some hardware failure.
Highwinds was nowhere near unlike readnews(yet slow speeds on "old" and inconsistent speed on new)
As for now its hard to say what is the best choice..
because mark my words- this is the day they will be talking about for years - "Giganews Strikes Back" lol

supernews+ blocknews account== very neat, indeed. considering their speed< stability, and now retention+price. what is not there(dmca) blocknews will cover.
i would say it beats usenetnow\ngroups+something. though that is if you dont have more than 50mbit

i have a crappy internet download speed anyway so which news server should i try as im fed up with astra news, i cant be bothered writing to them i would rather just jump ship.

had more files incomplete and wont repair.

rubbish

If you don't want to (or don't know how to) set up a fill-server using Blocknews.net or Usenet-News.net to pick up Astraweb's missing parts, then these would be my choices in a (similar-monthly-priced/similar retention) Astraweb alternative:
[*]Supernews.com:[/*] outstanding completion, but a lot of copyright takedowns.
[*]UsenetNow.net or Ngroups.net:[/*] good completion, fewer copyright takedowns

You can also use a trial Binload account (400 day retention) as an Astraweb backup.

Quote:

---

Originally Posted by zot

If you don't want to (or don't know how to) set up a fill-server using Blocknews.net or Usenet-News.net to pick up Astraweb's missing parts, then these would be my choices in a (similar-monthly-priced/similar retention) Astraweb alternative:
[*]Supernews.com:[/*] outstanding completion, but a lot of copyright takedowns.
[*]UsenetNow.net or Ngroups.net:[/*] good completion, fewer copyright takedowns

You can also use a trial Binload account (400 day retention) as an Astraweb backup.

---

i dont know how to set up a fill-server i use newsleecher, is it easy?

usenetnow is only $12 a month so i might just go with that.

cheers matey.

Quote:

---

Originally Posted by mysticbertie

i dont know how to set up a fill-server i use newsleecher, is it easy?

usenetnow is only $12 a month so i might just go with that.

cheers matey.

---

Yes, Newsleecher supports fill servers. In the window where you put in the server information, check off "use this server for fills only" or set a server priority list, something like that. I still have block accounts I bought over 5 years ago -- the download gigs never expire until used.

Ngroups costs less than UsenetNow, uses the exact same backend servers. But I think UsenetNow also has a $10/month special floating around somewhere.

Quote:

---

Originally Posted by Beck38

posting to one group is sufficient, it helps not one wit to do multiples (in fact it may be detrimental). Just my 2 cents.

---

I've always posted music in multiple groups: the specific genre groups as well as a more 'blanket' group like alt.binaries.sounds.mp3.complete-cd. This is because I believe that a lot of people might browse specific groups looking to sample music from new artists they've perhaps never even heard of before. And obviously the more (relevant) groups something is posted in, the more likely people are to find it via browsing headers/filenames. (just my own 2¢)

Quote:

---

Originally Posted by Beck38

'Back in the Day' (10+ years ago) that percentage was at least 20%

---

I remember that a dozen years ago, completion ranged from bad to horrible - even just the next day after something was posted. Having 15% or 20% pars made sense then. But today, going from 5% pars to 15% pars would mean losing over 100 days of retention. Personally, I'd rather have those extra 100 retention days, since incompletes can be completely avoided simply by piggy-backing block accounts. But since the vast majority of Usenet users appear not to know this "secret" of using fill-servers - then having a higher par percentage would no doubt save a lot of people from undue anguish and misery.

My biggest complaint is with the people who post things -- most notably MP3s and full albums -- without ANY pars, and this example of ignorance is sadly all too common.

Quote:

---

Originally Posted by zot

Yes, Newsleecher supports fill servers. In the window where you put in the server information, check off "use this server for fills only" or set a server priority list, something like that. I still have block accounts I bought over 5 years ago -- the download gigs never expire until used.

Ngroups costs less than UsenetNow, uses the exact same backend servers. But I think UsenetNow also has a $10/month special floating around somewhere.

---

well it sounds easy enough setting up the fill server, either way i need to sign up for something new so it may as well be a new news server, ill see if i can find the usenetnow 10$ special if not ngroups $11 deal.

thanks for your help ;)

i found the deal for $10.39 so signed up for that.

Quote:

---

Originally Posted by zot

I've always posted music in multiple groups: the specific genre groups as well as a more 'blanket' group like alt.binaries.sounds.mp3.complete-cd. This is because I believe that a lot of people might browse specific groups looking to sample music from new artists they've perhaps never even heard of before. And obviously the more (relevant) groups something is posted in, the more likely people are to find it via browsing headers/filenames. (just my own 2¢)

---

Uh, when this latest problem surfaced, there was a flurry of 'headers.. who cares about headers' type messages, so who exactly is following any specific group at all any more? People use whatever search engine is in front of them, either on a site like this, nzbmatrix, newzbin, or one of the myriad of usenet search engines. Of course, yes, that means that folks won't 'trip' over something, but then again, I would think with music it's just like films, folks rely upon their favorite site or something to give them a heads up, then go out to the search engines to see if it's 'available'.

As to pars, I like to think with the cost of the server plants 'keeping up' with what's being posted, and the 'zero day' being July/August 2008, that in the next couple of years, usenet will get ever closer to what P2P is as far as having just about anything one could want. I hope that the 'work' I do will never go away, period. And posting a far number of pars will help that process no matter what kind of heavy crashes occur (hopefully).

Only one download so far today has been corrupted on me. I have been with Astraweb through months now (7+ months) I cannot complain for the price I am more than happy.

When this 'day 100' problem first surfaced, I immediately went to posting I had done (and verified on several servers including Astra at the time I posted) and d/l 3-4 of them w/o major problems - yes, there were a fair number of 'skiips' and such, but as all of those rar archives were 10% par'ed, they only needed a slice of those to fully repair. Where folks got into trouble was with files that were around 2-3% or so: not enough pars to repair. :(

Quote:

---

Originally Posted by mysticbertie

i found the deal for $10.39 so signed up for that.

---

This is the link I found. http://usenetnow.net/ngr.html. I think they also have a 6 month special but I could not find it quickly. Been using the service for a long time and never had any problems. Blocknews is another o
Way to go if you wanna keep astraweb just put it in your reader as backup.

Well, as the date/time stamp on this message, no change I can see in Astraweb performance regarding skipping around 100 days.

Don't think anything has been done yet.

Another day goes by, no change either in the 'target zone' on what is now ~150 days, or anything from about 100-300 as well.

I think if they get past another weekend (and they haven't posted any 'updates' on the situation on their website), things are going to get more than a bit bad with their 'customers'.

Quote:

---

We have been recovering data by group. The most popular groups are being recovered first, then a full sweep of less popular groups will be recovered.
If you want, pm me the nzb or group and I can see what the status of your group is at.

---

oh well.. i wonder how long will it take to cover at least 50% of lost data

If so (and that quote isn't to be found on the Astraweb site, btw), I'm monitoring the three top usenet groups (top as in the amount of traffic in GB as listed by several tracking sites), plus a smattering of others that were listed at the top of this thread when things were first noticed 'going off the tracks'.

The only notice put up on the site so far as to any problems, was exactly one month ago. No updates, no nothing, since. So I opened another ticket asking 'what's up?'.

Like I said, it's been a month since the last 'official' pronouncement. The skips, according to folks here and elsewhere, starting showing up probably two weeks before that. The fact that both server plants, US and EU, were affected pretty much exactly the same, is to me the biggest clue as to what happened, either one plant ran off the rails and then propagated the failure to the other (an original h/w failure made worse by a s/w that failed to do proper checking before propagating it to the other server), or a upstream peer that nailed both servers simultaneously.

But I'm sure by the end of the days, folks simply want it fixed. From what I see, not anytime soon, and pretty much becoming never. Sad.

Got a reply to my message on their site. 'Will be fixed real soon now'. ha ha

As of some 5 hours ago (from the date/time of this message) still nothing changed.

Folks are getting antsy (uh, me?), and there are several folks/threads elsewhere, where folks are reporting problems where there doesn't seem to be any (mine and other confirmations of such), where users are making simple mistakes and then blaming it on Astraweb.

Again, they better fix things soon, like right now, or it'll snowball.

Do you have an NZB that you know won't work on Astraweb? I'd like to test around a bit.

Lots been listed here, just look at the first few posts.

The last 'news' posting by them was on 26June, saying that the gap was centered around 110-130 days, which would have put it around the end of February. Since that notice was just shy of 30 days ago, the gap as moved (time marches on), and center should be around 160 days back now.

I use the 'mergekit' HD postings as a way to see what's going on, since I d/l'ed them all (through Astraweb/US) at the time them were posted; therefore I know they were 100% at that time. Generally the 'mergekit.C' posts are the smallest, and any skipping is a dead giveaway as to problems with their plant.

However, tonight I notice that Astraweb/US is not adding any files, either ones that are posted at them (they are being propagated out to other servers, through) or ones coming from their peers. That tells me that 'maybe' they may be working on the plant (finally). FYI Astraweb/EU is not being affected. That tells me the 'front end' of the plant is still up and running, but the 'back end' may be shut down for 'maintenance'. One can hope, we'll see by tomorrow some time.

I was hoping something for something more specific. I tested a handful around 160 day mark and couldn't find anything. I did find 2 that were had incompletes around the 70 day mark. EU server didn't seem effected. Interesting part is that using different ports gets different incompletes. Setting the EU server as the fill server seems to almost get around the problems on the US farm.

" or a upstream peer that nailed both servers simultaneously. "

I don't understand this at all. The theory is that you have multiple peers. Each post is going to be offered to you by multiple peers--you take it from the one which offers it to you first. So if there is an upstream peer which isn't offering posts, you just get it from someone else. And a major site like Astraweb should be very well connected.

So could someone explain this?

Depends on how your s/w is set up. It used to be you defined a 'trusted' peer, and got 99% of your stuff from there, and only any others if it was something 'new' and unseen.

Still no recovery as of today, I'm seeing more and more folks dumping them and going with whoever.

Quote:

---

Originally Posted by PastTense

I don't understand this at all. .... So could someone explain this?

---

Just speculation.

I can explain what I think the problem might be:

The major companies like Highwinds and Astraweb appear to use some kind of automated software that back-fills missing articles between the US and EU servers. I've noticed that in many cases the US and EU server will be missing the exact same articles on older files -- files that are 100% complete at other companies' backends. Steve at Astraweb has stated in the past that they back-fill missing articles across the Atlantic between server farms, but I've never seen him post the details of how this is done. (trade secret perhaps?)

But anyway, it must be some sort of automated process, because there are far too many articles on a server to cross-check/back-fill manually (and let's assume that they don't have a huge staff).

Because both of Astraweb's server farms were suddenly afflicted in the exact same 100-day post range, I strongly suspect that the problem was a result of a malfunction of this automated cross-checking/back-filling software that Astraweb presumably uses.

Highwinds appears to be set up in a similar way: all older (500+ day) files are incomplete, and the US and UE servers both have (mostly) identical missing segments.

This is why it's so important to set up a block account (that uses another backend) as a fill-server. In the old days (3+ years ago) it seemed that server farms were completely independent, and using a provider's EU server as a backup to its US server would usually deliver 100% completion on old (non-DMCA'd) files. But today, that's no longer true: whatever is missing on one server will typically be missing on the other.

[QUOTE=zot;3597434]

Quote:

---

Originally Posted by PastTense

In the old days (3+ years ago) it seemed that server farms were completely independent, and using a provider's EU server as a backup to its US server would usually deliver 100% completion on old (non-DMCA'd) files. But today, that's no longer true: whatever is missing on one server will typically be missing on the other.

---

You'd be surprised, looking at one file around 70 days ago, I got these results using the NZB completion checker:

This was the US server on port 119
http://i.imgur.com/ILcfq.png

Port 23
http://i.imgur.com/TRVTM.png

Port 1818
http://i.imgur.com/2FCRG.png

Port 8080
http://i.imgur.com/itKt1.png



EU Server, only port 119 had an incomplete
http://i.imgur.com/bMaVS.png

Thanks, Cola. I'm completely stumped on why the port number can make any difference at all -- assuming that your result is consistent and repeatable across multiple trial runs.

Since we know the same backend is used regardless of the port setting, that would of course suggest that the article exists on the server, but is somehow not being reported correctly.

Have you tried checking the completion using SuperNZB? It uses a different method, which appears to be more accurate, faster, and overall superior in my opinion.

http://filesharingtalk.com/threads/4...nt-mini-review

I wasnt that surprised after Malcontent on the first page was told by astraweb to try different ports. I was a bit skeptical, but shit. I've seen it happen on one other nzb. Still took about a dozen NZBs to find 2 with missing parts, but I think par2 still would have fixed it. I might try some more tomorrow with SuperNZB.

I didn't put too much weight in that at the time, since tech support guys can often just shoot back the same standard responses (try SSL, try different ports, etc.) to a lot of different problems, since ports are so often blocked or throttled by ISPs.

But this is something quite unique. I suppose on a newsreader that allows multiple fill-servers, a person could set up a separate server field for every port number, and get near-perfect completion using that string of multiple port numbers.

Seems like the completion issue is more of a software bug than anything else.

Quote:

---

Originally Posted by zot

Seems like the completion issue is more of a software bug than anything else.

---

I think that's pretty obvious. I'm 'fighting' at least three separate problems caused by lax programmers, on things as diverse as Mr. Astraweb, to my computer network, to my media system.

Add to that my 'smartphone', where the vendors doing 'apps' want to be paid not just twice (the paid service the app connects to, and the end user/me) but by the phone manufacturer, and now the phone carrier (four 'streams' of profit) If the manufacturer refuses to 'pony up', they change the code slightly to make sure the app you bought a year or two ago to stop working. If the carrier refuses to pay their kickback, the app simply stops working in it's tracks.

Everybody is getting too greedy. What's going on with Astraweb is particularly troubling, as I know that the s/w they are running is 'home grown', not anything from a s/w development company. So, who they're paying to maintain that s/w is a good question.
For the thing to feed differently according to the port one connects to is beyond bizarre. Major problems.

For several years now, I've been telling folks the story of the last time I spent time living in Silicon Valley (1998-2001); all over the place programmers were getting laid off, and replaced under the H1B visa program with Indians who worked for 1/4th the price (which is illegal, but who was cracking down on it? Nobody). The companies would put up these folks in 1 bedroom apartments, 4-5 per, with a pocket of bus passes to get back and forth to 'work'. Then the dot-com crash. Care to guess what contributed to that?

Now this morning my malware program needed updating, that I just spent a couple days on two weeks ago cleaning up a problem. Now it has the same problem. Don't these folks TEST their product out before releasing it? Huh? Nope. 'My code don't stink' is the typical refrain. Yes it does, now I'll spend another couple days cleaning up after their current mistakes.

I finally pulled the plug on astra and jumped on the supernews bandwagon. Retention is pretty much the same, price is slightly better but with a more reliable feed.

I use to have problems all the time with astraweb. Then I got sabnzbd. I tend to download all around the retention sphere, and in the last 6 months have only had 1 failed download (some lazies don't upload any pars...) yet have downloaded over 1.5 terabytes.

Strange that I've never even run into the issue they confirmed... I have sickbeard auto downloading new tv shows yet never have had one of those fail either...

My advice is try a new client like sabnzbd before giving up. And lowering connections can help too

Sent from my Droid using Tapatalk

Clients aren't going to fix server side issues.

Quote:

---

Originally Posted by Jackbauer

in the last 6 months have only had 1 failed download (some lazies don't upload any pars...) yet have downloaded over 1.5 terabytes.

---

many folks here do that in less than a week.

Still no change whatsoever on either US or EU servers as of the date/time on this message. As of the coming Monday it will be a full month since Astraweb acknowledged the problems with a 'news' item, and they were advised of the problem by many users at least two weeks previous.

I think that if you can, and it works for you, it's way past time to jump ship, or have a nice large bucket of fill-bits from another provider (like Blocknews).

Quote:

---

to jump ship

---

to jump where exactly?

Highwinds? Nope. Hopeless

Supernews? DMCA , one needs a fill account too

Readnews(resellers? Slow speed

it is still the same 2 choices

supernews+ blocknews
astraweb+blocknews

well maybe readnews(ngroups\usenetnow) if you dont have 50+mbit plan
downloading files past xxx days is a pain on a fast connection

Also, depending on what you download, it's quite possible that a Blocknews fill account would last much longer on Astraweb right now than on Supernews.

re: jumping ship. ....

Sometimes the only thing that will force a company to correct a problem is if enough people leave. That's obviously not the case here with Astraweb, but I strongly suspect that a mass-exodus might be the ONLY thing likely to get Highwinds to fix its incompletes that exist month after month. That was indeed what it took for Highwinds to finally get serious about Easynews' meltdown 2 years ago, after several months of runarounds and excuses.

It would certainly be helpful if Astraweb at least gave a weekly update, and posted general advice on AW's "system" page. It should not be necessary for every single person to (separately) email support to get answers for a common problem.

And for some reason Steve will post on the Newsbin forum info that never goes up on the Astraweb site.

Fill servers are going to be a reality now that DMCAs are being used more frequently. Theres no way around that, unless you're using a dutch provider. I don't know of any dutch providers that have 1000+ days retention though. Its not as though block accounts are that expensive. What would worry me is if blocknews starts having the same DMCA holes as the other big providers.

http://www.supernews.com/super-special/