Caching

[rb2k]

Does alt.binz already implement caching (in RAM)?
those usenet downloads can be a bitch for (especially laptop) HDDs on fast connections

Could also be usefull for overall CPU load (the many IO requests would be "punished" by the OS scheduler)

[Rdl]

Depends. We already had a problem with high cpu load due to logging to file, but that was fixed. Do you have problems with high cpu usage compared to other programs?

[rb2k]

Depends. We already had a problem with high cpu load due to logging to file, but that was fixed. Do you have problems with high cpu usage compared to other programs?

Hmmm... yeah
Taskmanager goes and stays at about 70% while just downloading (no par/rar threads running)

My CPU shouldn't be the problem (1730 mhz Centrino). But I am using Alt.Binz on a notebook so I/O Requests take a little bit longer than on average desktop machines because of the slow 2,5" 5400 rpm drive.

I guess creating a "big" object on the heap as a temporary storage and writing only every 50 MB or so would help reduce writing to disk
(ok, it would boost RAM usage, but I can live with that... that's what RAM is there for )

[Rdl]

At what speed are you downloading? Did you try tweaking OS to allocate more RAM for file caching?

[rb2k]

usually at about 1,5 mb/s+ (or from university at 4 mb/s +)
atm it dropped to 1 mb/s and is jumping somewhere above 50% with a RAM usage at about 10-20 mb
(which is not that much concerning the amount of disc I/O)

I didn't do any OS Tweaking yet, but I don't think it's the job of the OS to do the caching
What would be your windows-tweaking suggestions to keep down the CPU usage?

[Rdl]

1mbit/s is nothing. My download computer is cel850 with 5400 download drive and with my 768kbit connection cpu usage is around 5-6%.
Could you please for test purpose setup to 1 connection and try to download something with large parts size (bigger files/smaller part sizes) and monitor cpu usage while downloading. While downloading part there is no i/o involved, but when part finishes it is writen to disk. When all parts of one file are finished, then disk intensive operation begins.

Could you try to disable log to file. Also try to tweak this WinXP option ->Control Panel->System->Performance->Settings->Advanced->Memory usage->System Cache

[rb2k]

1mbit/s is nothing. My download computer is cel850 with 5400 download drive and with my 768kbit connection cpu usage is around 5-6%.
Could you please for test purpose setup to 1 connection and try to download something with large parts size (bigger files/smaller part sizes) and monitor cpu usage while downloading. While downloading part there is no i/o involved, but when part finishes it is writen to disk. When all parts of one file are finished, then disk intensive operation begins.

Could you try to disable log to file. Also try to tweak this WinXP option ->Control Panel->System->Performance->Settings->Advanced->Memory usage->System Cache

first of all: sorry, megabyte... not bit

without logging I got down to <50% (jumping 30-50%)

trying the systemcache setting now


edit: nothing really changed... still at approx 50% at 1,6 MB/s

[Rdl]

What is your expirience with other programs? Are they using more or less cpu time? Any way you could appear on #altbinz @EFnet so I could compile few test versions for you?

[rb2k]

What is your expirience with other programs? Are they using more or less cpu time? Any way you could appear on #altbinz @EFnet so I could compile few test versions for you?

As far as I remembered they used less CPU Time

I am on IRC atm

(btw: #alt.binz not #altbinz)

[Rdl]

Ok, could you compare new test version with some of the other programs

[sandler]

Just had the same idea with the caching and wanted to ask what's the status of this suggestion?

To be more specific, I was just wondering, why not keeping files written to the temp directory in memory (at least up to for example a certain max cache size)? I've seen this feature in SABnzbd and it reduces HDD access to a minimum. Not that I'm having performance issues with alt.binz, I just think it's another great idea for a fabulous program .

[proto]

Hi all,

great program, but it seems to be missing one VERY important feature that other readers support: being able to specifying the TEMP folder, thereby allowing you to assign the TEMP folder to a ram disk, which is SO MUCH better once you get used to using it, 90% less HD i/o, etc.

I have a 350MB ram disk set up in other readers, and thats usually big enough to contain all the individual msg files that make up a yenc 'chunk', and makes it very fast to decode it to its final location on HD.

please add the ability to specify the TEMP directory.

thanks!

[Rdl]

It's already added in one of the daily builds

[proto]

It's already added in one of the daily builds

Brilliant!

This prog is so great, just about every conceivable option has been thought of and added (download only needed pars = brilliant!).  I was able to work around my issue by copying the entire altbinz program folder to my ram disk (and setting the download folder to a folder on my HD), and run it from there, runs like greased lightning, and saves wear and tear on my HD.

once again, thanks for a great program!

p.

[spod]

In the early days of eDonkey and BitTorrent, there was no read/write disk cashing, and I burned out 2 hard drives from all the time I spent online. When I started downloading from Newsgroups, I thought it would be different. I was wrong.

I discovered that Grabit writes to the disk constantly (and rhythmicly noisily) and using it on a 256kbit connection, the constant 'strumming' destroyed my 3rd hard drive after only a few months. (It was an external; I already learned my lesson from using P2P)

Bittorrent apps have come a long way in implementing disk cashing - do doubt due to all the complaints about destroyed hard drives. Here is a screenshot of Utorrent showing it's cache configuration features:



Would it be possible to implement something similar in Alt.Binz? (Of course it would be much simpler, since there is no upload component going to multiple addresses, as with P2P.) It would be nice to be able to set a disk cache size, depending on factors like download speed and the PC's RAM memory.

Our hard drives would be thankful.