Corruption Finding & Fixing

[ilw]

Hey
I've written a few programs for kazaa so i thought i'd post an update on what i've done:


MP3 Checkers

These two programs fulfill different requirements, MP3 checker is designed to check all your existing MP3's and remove any screechers from your existing collection and the autoscanner is designed to stop any more screechers getting into your collection (and thereby helping to remove them from the fasttrack network). Neither of them are perfect and will allow some to slip through and (although i haven't seen any) they may give false alarms.

MP3 Checker is a GUI based program which can scan MP3's for the tell-tale signs of RIAA added screeching. It can scan around 800 MP3s at a time and can do a full scan in under a minute.

MP3 Autoscanner, is a command line program designed to work with the KLExtension's file organiser so that MP3's are scanned and if bad are quarantined as soon as they are downloaded.

Other Anti Corruption programs


My other anti corruption programs are based on the same principle as the Intelligent Corruption Handling available in better maintained p2p networks such as the edonkey network. The idea is to calculate CRC (Cyclic Redundancy Check) information for chunks of a file, this means that each chunk has been given a unique identifier based on the data contained in it. Any errors or changes to this data gives a different CRC value and so chunks can be easily checked for corruption. The way I hoped this would turn out was that lots of people (especially those who often contribute verifieds) would convert to KSig mod and use it to generate the modified quicklinks which contain the CRC hashes. Then anyone downloading one of these verified files can CRC check their finished files and if there is any corruption then Kchecker will see it and redownload that part of the file(with 2% resolution).

Afaik if a verified file you downloaded is corrupt then you have 3 choices

1) To ask someone who has a good version of the file to crc hash it with one of my programs
2) If you know where in the file the error is (eg if its a video file the errors are visible and rough file position can be estimated) then you can use DrSpud's KCF and use that to redownload the bad parts of the file.
3) Junk the file and redownload it all.

The programs

KChecker: KChecker is my main program, it generates crc information for files, compares crc hashes to determine corrupted chunks and generates a dat file so that kazaa will redownload the bad chunks. It is capable of checking finished and ongoing downloads (though it only checks the first good chunk of ongoing downloads). If Ksig mod has been used on a crc quicklink then this crc information is stored and is automatically checked by KChecker.

KSig Mod: This is basically just a modification of Hasnain's KSig program it is modified to also calculate the crc information that Kchecker uses and also integrates this information in backwards compatible quicklinks. (NB its based on the previous release of ksig, I haven't got round to updating to hasnain's latest release)

KPadder: KPadder is just a small program to allow you to change the size of a file, this is useful if you lost a file due to a corrupted dat appendix (eg after a power cut). See the kchecker page for more details on how you would go about recovering a deleted file.


More info, rough guides and downloads from my websites http://www.ilwsoftware.tk/ OR (TEMPORARILY NOT UPDATED http://www.freewebs.com/ian_l_williams/

I only recently did the websites so theres lots of spelling errors, grammatical mistakes and generally the written content is pretty crap, i'll sort it out soon so you don't have to post any typos you see If you see any broken links though please tell me.

[sharedholder]

Good work,and nice sites too!Thank you.

[MUSLEMAN]

i'm posting that sucker every chance i get its great, good job boss
wats up sh

[Livy]

ilw what about having a look here http://www.klboard.ath.cx/index.php?showtopic=70807

perhaps u could work together, or add some command switches to mp3checker so u can add it to check files after they download using kl extensions organiser

[Muse]

Nice work ILW

I like the concept of the autoscanner. I works great but not with all messed up files

I have found a screecher that slips through. It's one of the Timberlake fakes:

File:CD Quality LILY.mp3
Length:3952037 Bytes, 3859KB
UUHash:=vzxPsuiX/kvjk1qdtJrMc5iReag=

Maybe you use this example to improve your autoscanner?

Another problem is that not all fakes are screechers.

Maybe we should fight these polluters by their own means? If you download a fake, rename the file to "Don't download this fake" and keep sharing it. These renamed files will show up in the list of identical search results and anyone can recognize a fake before you download it.

[ilw]

Thanks for the feedback & i saw the other post about this, but decided to reply here. Btw thanks sharedholder
I tried the file, but i must have got a more corrupted version than you cos it found it on my computer. I've increased the amount of scanning so that it may take a tiny bit longer but is more likely to spot a fake, hopefully it should now spot it. (Do you by any chance still have the file so you can test it?)
I've also changed it so that it renames the files to
This_is_a_FAKE_filename
I don't think i can fix the bug where File Organiser repeatedly scans the file because if i change the file extension then the file won't show in searches and so there would've been no point of changing the filename to show its a fake and this was an idea that i kind of liked. Even though the RIAA can easily start saying using this to discredit valid files, they would still have to be sharing the file in order for them to say its a fake and if they're sharing it then chances are it is fake (their high bandwidths mean they can spread fakes very quickly)
NB I've now made it clear on the website that you shouldn't have the download directory as your 'move to' directory.

[Muse]

OK ILW.

Is this changed version available for donwload? I'd like to try it.

Glad you like my idea about sharing the fakes with "This is a fake"filename.
Hope we can persuade more users to adapt this method.

[ilw]

Yeah, I updated the geocities website (in my sig)
I'll change the name of the executable so its more obvious which version you have.

[nikita69]

yes, great work ilw and i created some tips about them in my tips topic.

[quentarez]

I absolutely love the mp3 autoscanner and checker!

I was wondering if you could possibly try to do something for me? The autoscanner, each time a file is complete, scans it, but it also pops up a console window, which is, to me (im a nut about this kinda stuff), very aggravating.

I kinda bypassed it by making a shortcut to the autoscanner exe, and telling the shortcut to run minimized, but I was wondering if maybe you could have it run the exe as a background process instead of a console app like that? I don't even know if this is possible, but I thought I might as well suggest it anyway!

Thanks again for these awesome progs!!!!