More reliable PAR2 routine - inspired by Par-N-Rar

[afdad]

Since the repair routine of ABZ fails every now and then, I looked around for a better alternative. And I found it. It's a freeware program call 'Par-N-Rar' that features some very interesting options:



And the program was able to repair/extract downloads where ABZ failed, i.e., Par-N-Rar identified the number of missing blocks correctly.

The program and the source code is available at:

http://www.milow.net/site/projects/parnrar.html

[hrothgar1]

I was looking into the same issue, afdad.  As you can see ( http://www.quickpar.org.uk/forum/viewtopic.php4?t=627 ) -- and as pointed out by rdl ( http://www.altbinz.com/forum/viewtopic.php?t=330 ), Quickpar and par2commandline ("par2.exe" as used by ABZ) differ slightly in data detection and file scanning.  Quickpar is slightly tweaked to find good blocks at the end of corrupted files.

Since ABZ is closed source, I couldn't dig around, but I did take a peek into par2commandline and this new par-n-rar.  Unfortunately, par-n-rar just uses a modified version of par2commandline (take a look at the source).  However, the par-n-rar guy did slightly modify par2commandline:  I diffed the .4 release of par2commandline and par-n-rar's bundled par2commandline source.  Unfortunately, it just looks like he (mostly) just modified the cout and cin output/input calls.  The file par2repairer.cpp and its file-scanning function ScanDataFile(...) as well as FileCheckSummer are unchanged as far as the above-referenced "end of file" data block detection.

Unfortunately again -- since Quickpar is closed source, I couldn't find out these couple Quickpar file-scanning "tweaks" -- or I'd just recompile a fixed version of par2commandline myself.

Could you recheck your findings that this program works where par2.exe (par2commandline) doesn't?  I just tried it with a sample test that I earlier had trouble with -- and par-n-rar was still a block short (like par2.exe, but unlike quickpar or ABZ).

[hrothgar1]

Also...  This place is as good as any:  does anyone know what par2 library is used by ABZ (internally)?  Since it obviously has these "end of file" tweaks (as in Quickpar), I'm curious what they look like...

[afdad]

Interesting observations, hrothgar1!

I should be more precise of what I meant by Par-N-Rar succeeded where ABZ failed: As you noted, hrothgar1, Par-N-Rar basically uses the par2commandline routines, and, hence, it also misses some good blocks every now and then. What I like about it, though, is that the program indicates the correct number of blocks that is needed for its repair function to work smoothly.

As far as I can tell, that's where the problem with ABZ lies: Its checking algorithm (like Quickpar?) to identify missing blocks is more advanced than the actual repair routine. Therefore, occasionally ABZ downloads too few blocks for the repair to work. Now, my point is that if ABZ used the same (although inferior) checking routine as par2cmdline uses, there shouldn't be any failed par processes anymore.

[hrothgar1]

Good point, afdad.  As you said, there's also the option of "downgrading" ABZ's internal par2 routine -- which is why I'm curious what par2 library is being used (the only other available library "libpar2" is based almost entirely off of par2cmdline -- and a quick check just showed that they handle file-scanning the same way -- unlike ABZ and Quickpar).   Of course, "the only available library" == "what I've found through a little googling."  Using something like libpar2 inside ABZ could be a solution (since it's basically the same thing as par2cmdline).

My concern is that the difficulty of fixing this lies in how rdl integrated par2 functionality into ABZ itself.  If the library he used is radically different from libpar2, it might just be easier to use a better commandline par2 repairer, instead of modifying ABZ itself.  

My hope is that if he has a "better" par2 library, then creating a "better" commandline repairer could be straightforward.

Again, I just discovered ABZ last night, and I hadn't done any programming with par2's until this morning, so if I need be set straight, someone please do!

Anyway, above all -- much respect for rdl for making such a quality program!

[Rdl]

There are no available par2 routines for Delphi, so I had to write them myself. Unfortunatly I stoped messing with par2 cause I wanted quick implementation of par2 and ussing par2cmd did save lots of times. I'll try fixing the problem one way or another. Can anyone point to me exactly what file(and nzb) is missing blocks in par2cmd and not in abz and quickpar

[hrothgar1]

I don't have a server to upload it to right now, but the current, uhh...  *cough* *cough* episode of the Gilmore Girls (it wasn't for me, honest!) has this problem.  It's the eighth from the top on TvNZB right now.  For "...part24.rar" you get "11 of 14" good blocks in par2cmdline and "12 of 14" good blocks in ABZ internally and Quickpar.

(I'll see about uploading the actual nzb file as soon as possible -- though it should still be in the front of TvNZB.)

[hrothgar1]

And btw -- much respect for writing your own par2 functions!

(Forgot I could just link the nzb. here's the above-mentioned bad nzb: http://www.tvnzb.com/direct/nzb/8216 )

[Rdl]

I think I fixed par2.exe. Update will be available in next daily build

[hrothgar1]

woot! thanks!

[afdad]

Wow! Excellent. I'm looking forward to that. I will test it. And complain if it still won't work.  

Rdl, I'm deeply impressed with your extremely quick solution to the problem.

[Rdl]

Daily build with fix is already available for testing.

[afdad]

YES!!! It's working like a dream! I downloaded a DVD where I previously had problems with ABZ repairing it. Now it worked flawlessly. And, best of all, it only needed the same small number of par2 files as Quickpar needed - not as many as would have been required by par2commandline! So I assume that now ABZ has implemented the same tweaks as Quickpar. Excellent work!

I love you, Rdl!  :wink:

BTW, as a bonus, the par2 process is now much faster than in previous builds! Previously, all files were checked again before the actual repairing started. Now the repair process starts immediately (ABZ is already checking every file as it is downloaded).

[Rdl]

No, Ijust fixed the problem with par2cmdline and recompiled it. Par2cmdline(par2.exe) still need to check the files before repairing.

[afdad]

Oh. It seems I was too carried away by the great news.    I based my assessment of faster repairs on the log file.

Talking about speeding up the process: Since ABZ checks all files immediately as they are downloaded, is there not a way to skip the additional check before the repair starts? I guess this might involve integrating the repair routine into ABZ also. And I can't tell how difficult and time consuming that would be.