alt.binz takes forever to close since new queue management

[therealjoeblow]

I have a large queue, the queue file takes ~800mb on harddrive, and alt.binz uses about 1.7gb or ram when fully loaded (please don't start with the debate on whether large queues are good or bad, it's been done before and the net result is a lot of wasted effort - it's purely a user preference, some use large queues, some don't).

The problem is that since alt.binz was changed in v33.3 to use a different queue structure, it takes *forever* to close down now.

In previous versions, if I closed it and watched task manager, it would use 50% CPU for about 30-60 seconds and the memory useage would stay constant.  Then it would start giving back memory in 200-400mb chunks and be done closing in about 1 minute.  That worked *just fine* for me, I had no issue with it at all.

Now it's taking over 30 minutes to close!  The memory is returned to the system in kb chunks instead of hundreds of megabytes.

If this new queue management code is optimized for small queues, could I respectfully ask that you please put the old code back in, and give users an option in the setup screen to select which method they want to use?

It's nearly unusable now, and has been working fine in this respect in the past.

Thanks
The REAL Joe

[Rdl]

I have a large queue, the queue file takes ~800mb on harddrive, and alt.binz uses about 1.7gb or ram when fully loaded (please don't start with the debate on whether large queues are good or bad, it's been done before and the net result is a lot of wasted effort - it's purely a user preference, some use large queues, some don't).

The problem is that since alt.binz was changed in v33.3 to use a different queue structure, it takes *forever* to close down now.

In previous versions, if I closed it and watched task manager, it would use 50% CPU for about 30-60 seconds and the memory useage would stay constant.  Then it would start giving back memory in 200-400mb chunks and be done closing in about 1 minute.  That worked *just fine* for me, I had no issue with it at all.

Now it's taking over 30 minutes to close!  The memory is returned to the system in kb chunks instead of hundreds of megabytes.

If this new queue management code is optimized for small queues, could I respectfully ask that you please put the old code back in, and give users an option in the setup screen to select which method they want to use?

It's nearly unusable now, and has been working fine in this respect in the past.

Thanks
The REAL Joe

Huh. That's news to me. New queue structure was implemented exactly for large queues.

[Rdl]

Do you have auto save queue on xx minutes?
How long does it take for altbinz to load that queue?
How many collections are in queue?
How many files are there in queue dir?

[therealjoeblow]

Do you have auto save queue on xx minutes?

Yes, every 60 seconds

How long does it take for altbinz to load that queue?

~25 seconds

How many collections are in queue?

857

How many files are there in queue dir?

859

Cheers
The REAL Joe

[Rdl]

And EVERY time you try to exit it takes 30min?
How long does it take to save queue every 60sec?

[therealjoeblow]

And EVERY time you try to exit it takes 30min?
How long does it take to save queue every 60sec?

Sorry, must have been something pooched earlier - I tested it over the past 2 days, and while running it looks like it's only taking ~400mb now instead of the 1.7gb before, and now it's closing in 30~60 sec.

Thanks for replying anyway
The REAL Joe

[therealjoeblow]

OK, so after considerably more testing, I can verify it takes a *long* time to close with the new queue.  It's not >30 mins like I reported earlier, there was something wrong with my system then; but it is between 10 and 15 minutes regularly.  Ive verified a dozen times over the last several days with reboots in between to be sure.

With the old queue. memory would be released in 200-400mb chunks when I watched it in taskmanager, and it would close in 30 or so seconds.  Now memory releases in 2-4mb chunks and it takes forever.

Can you make queue management the old way an option RDL?  Or is that no longer possible?

Cheers,
The REAL Joe

[Hecks]

Unfortunately, nobody else seems able to reproduce your problem. We've tested, and so far the results are all in seconds rather than minutes for very large queues of the type you've described. There's also no logical reason why the new approach should take so long, since considerably less data is saved to disk.

Have you tried on another machine?

[Rdl]

OK, so after considerably more testing, I can verify it takes a *long* time to close with the new queue.  It's not >30 mins like I reported earlier, there was something wrong with my system then; but it is between 10 and 15 minutes regularly.  Ive verified a dozen times over the last several days with reboots in between to be sure.

With the old queue. memory would be released in 200-400mb chunks when I watched it in taskmanager, and it would close in 30 or so seconds.  Now memory releases in 2-4mb chunks and it takes forever.

Can you make queue management the old way an option RDL?  Or is that no longer possible?

Cheers,
The REAL Joe

I'm 99% sure something is wrong with your computer. Do you have write back enabled on HD which stores queue?
I could leave both queue save/load routins, but then I would have to maintain both with future changes and I'm not really willing to do that.
As Hecks said, once you reported problem, we've tested with queues larger then yours and in worst case scenario (when all data needs to be saved) it took around ~40 seconds to write it (~30sec to load it) on a regular 3 years old HD and around 10 sec on SSD.
That's the worst case scenario that should never happen. Ussualy with moderate queue changes it should take sub 1sec.