Delayed queue handling - repairing and unraring

[p1nky]

Hi rdl & all,

this seems to be related to this post also https://www.altbinz.net/forum/bugs/decoding-thread-hangs/

I have noticed this for quite a while and if queues are reasonably short or download connection is rather slow it doesn't happen or can't be noticed, however if the download queue is large, the download speed is high, and maybe also if the destination drive can't always keep up with the download speed, then it comes to this issue, and then it is quite annoying.

My scenario is that I have a D:\ drive (an SSD) where abz downloads to and where it repairs, and then it unrars to the E:\ drive (HDD).
The queue is sufficient to fit on the E:\ drive, but not on the D:\ drive, so I am dependant on it unraring and deleting it from the D:\ drive while downloading.

The scenario I had today was a 900 GB queue (with 900+ GB free on E:\) and 350 GB free on the D:\ drive.
Now when I woke up it turned out abz had went to pause mode, because the D:\ drive was full, apparently because a delay of more than 350 GB had built up during repairing and unraring.

The root cause is that abz for some reason at some point just starts sitting there and doing nothing for several minutes. It doesn't repair and it doesn't unrar for quite some time. Then after quite a long time something triggers the repair and unrar process after all - but just for a few downloads, befoe it goes to "idle mode" again and again waits a few minutes.

Have a look at my queue:


It just sits there and does nothing at the time I made the screenshot, there are dozens of downloads that are still red and not checked, dozens that have been repaired and not unrared, and then also there are dozens that have been downloaded (with par2 file) but that don't even show up yet in the par2 queue.

It would be great if you found some fix for that. As said it only happens when you have a huge queue, but when you do it is super annoying, because you can't rely on abz downloading your queue, but instead you can expect it to fill up your download drive and stop at some point.

If it can't be fixed then at least some workaround would be great, with abz auto-unpausing if HDD space gets sufficient again.
If you check my screenshot then by the time I woke up it had at least unrared 100+ GB already, but of course once it had gone to pause mode due to insufficient space it stayed there, even if space became available again at a later time.

btw by the time I made the screenshot I had manually unpaused abz so that is the reason why it's not in pause mode in the screenshot (but it was in pause mode when I woke up, and free space on the download drive was already 100+ GB then).

[p1nky]

One more note about this (and as I can't modify the original post anymore): the reason for the problem definitely is not that the E:\ drive, that stuff gets unrared to, is *that* slow. It for sure is slower than the D:\ drive and for sure it can't always keep 100% up with unraring, but for sure it is not that slow.
Also you can hear it when it's working and when abz is having this problem and is doing nothing in the queue then 99% of the time the E:\ drive is doing nothing and also idling. So it's not that abz is waiting for the E:\ drive to catch up. The E:\ drive has caught up with unraring loooooooooong before abz continues with processing the par2 queue.

[Rdl]

Workaround is easy fix, but I'm not interested in that (it will be added once I figure out what was the problem).
Par2/unrar thread can't get stuck for a while and do nothing and then come back. It must be doing something (if that makes sense at that time or not is not the focus for now)
I'm more interested in par2 sets (and collections for that set) that are bellow visible ones in this screenshot cause processing those files make par2 thread appear as doing nothing.
So, at the point in time of taking a screenshot, I would be interested in first par2 set that is bellow 1 block missing and not a single file bad (that would indicate to me which files in par2/unrar queue are currently processed)
Then, what is the collection that files came from, was there multiple par2 sets from that collection.

[p1nky]

ok just so I get this right, you want to have a look at the one set that is not yet visible at the time it seems stuck, is that right?
because in the screenshot there were dozens of "red" ones below, they were visible when scrolling down, i just couldn't fit them in the screenshot.

I will have another huge queue tonight, so when it gets stuck again I will have look at what collection is *not* yet showing in the par2 tab and then when it continues it should appear then and that is the one we will have a closer look at, correct?

[tvholic]

Just wondering if the system might be running low on CPU cycles or (more likely) system memory.  That could cause application delays, especially if your swap file is on your full D: drive.

[p1nky]

Just wondering if the system might be running low on CPU cycles or (more likely) system memory.  That could cause application delays, especially if your swap file is on your full D: drive.

Unlikely I'd say (this is while abz is currently working hard downloading):

[p1nky]

So the bad news is that it looks like it didn't run into any problems last night. I'll keep checking and report back tomorrow.

[p1nky]

So here we go again this night:


The last one at the bottom stuck at counting blocks 110/1250 for 5 minutes at the time I made the screenshot.
Also at this time the whole queue has been downloaded and there are 10 more releases that have already been downloaded, but don't show up in the par2 queue yet.
The size of those 10 releases is about 17 GB, so supposing abz is doing something on those not yet visible downloads (counting blocks in the background or something?) then for sure something must go terribly wrong here, as I am quite sue my SSD and CPU would have been able to process these 17 GB  in significantly less than 5 minutes.

As I have been uploading this and making this post certainly 10 more minutes have gone by and now abz is at 484/1250 blocks checked at the same release.
The counter isn't going up slowly but getting stuck again and again and then jumps ahead some blocks.
I really wonder what abz is doing in the background here.

EDIT: but in these 10 minutes while blocks went from 110 to 484 on the last release three more of the above downloads (the next one in queue) were repaired (but none more were unrared) and also of the 10 "missing" (not yet shown in par2 queue) downloads no more have shown up yet.

[p1nky]

One more thing I just noticed:
I am doing nothing else than let abz work on the download drive (the SSD) and task manager is showing me reads from the SSD at a rate of pretty constantly around 30 MB/s.
So it seems the issue might come from abz processing something very slowly. Again I doubt the SSD is that slow. When I copy files from this SSD to another one I get read speeds of around 500 MB/s.
Around 30 MB/s however is the download speed I usually have - only that at this time it's not downloading anymore, and it's not writing the downloaded stuff to the SSD, but reading something that has already been downloaded for a while.
Could it be that it somehow paces the reading from the SSD to the download speed but then keeps that rate when there is some delay, so it can't catch up anymore?

Here are 2 more pix, the first one is from when abz apparently down "nothing":


The second one then is when it is unraring something, you can see the read speed suddenly jumping up significantly:

[p1nky]

Watching this for quite a few minutes now and it is pretty amazing for me that the read speed from the SSD for minutes almost exactly matches the 30 MB/s rate and that the read speed diagram looks almost exactly like the download speed diagram in abz usually looks.
Can this be a coincidence?

[p1nky]

Here my final post, which goes in a different direction:

Everything has been processed now, but before it finished I added another download, which also finished.
Also it showed up in par2 queue after a while (while other stuff was still processing).

Now: ALL other stuff has been processed, it has been repaired and unrared, there is nothing more to do with the other downloads.
Also there are no more other downloads in the queue (well there are but ALL of the others are paused, so no unpaused download in the queue).
Also: All the blocks have been read from the last download.
BUT: now it's doing nothing, it's not saying "checking files" or "repairing" or "unraring"... it's just sitting there and doing nothing.
What could it possibly be doing in the background now!?

While I am writing this (after about 5 minutes being stuck at all blocks checked and doing nothing) it now started with "checking", "repairing" and "unraring".
But what did it do for 5 minutes?

See the last one here, all but 1 blocks read (and the one isn't there but needs to be gained by repairing), everything else processed, and nothing more in the queue (and also it wasn't reading anything from the HDD in those 5 minutes as there was no activity on the HDD at all):

[spamwerbung]

Check how many par2 files you have left to download. Your download is one block short (2360 out of 2361), so Alt.Binz can't continue to do anything useful unless you provide one additional block.

[Rdl]

I've already contacted the user privatly and he's been running debug version for 10-15 days now. So far, he hasn't been able to reproduce the problem.