Author Topic: downloading using zlib compression  (Read 3289 times)

Offline MrbLOB9000

  • Contributor
  • ***
  • Posts: 43
downloading using zlib compression
« on: March 27, 2009, 11:50:05 pm »
my NG host supports zlib and I can use other apps to make it download using zlib but it'd be cool if altbinz would do it directly.

if it already does it, I can't find the option, can anyone point me to it?

Offline Hecks

  • Contributor
  • ***
  • Posts: 2011
  • naughty cop
Re: downloading using zlib compression
« Reply #1 on: March 28, 2009, 12:12:49 am »
Which NSP? Which other apps?

Offline MrbLOB9000

  • Contributor
  • ***
  • Posts: 43
Re: downloading using zlib compression
« Reply #2 on: March 28, 2009, 12:28:48 am »
newshosting and Stunnel.  just started monkeying with it and of course I can't get it to work. :(

http://www.newshosting.com/en/newshosting-premium-usenet-provider-support-lightning-accelerator.php

edit: woops forgot to uncheck ssl in altbinz, stunnel takes care of SSL so it was like double SSL and freaking it out.  So it's working with Stunnel now, but still would be cool to have it all in one app.
« Last Edit: March 28, 2009, 12:30:58 am by MrbLOB9000 »

Offline Rdl

  • Administrator
  • *****
  • Posts: 4050
Re: downloading using zlib compression
« Reply #3 on: March 28, 2009, 12:46:56 am »
newshosting and Stunnel.  just started monkeying with it and of course I can't get it to work. :(

http://www.newshosting.com/en/newshosting-premium-usenet-provider-support-lightning-accelerator.php

edit: woops forgot to uncheck ssl in altbinz, stunnel takes care of SSL so it was like double SSL and freaking it out.  So it's working with Stunnel now, but still would be cool to have it all in one app.
Nothing to gain using zlib except when downloading headers

Offline MrbLOB9000

  • Contributor
  • ***
  • Posts: 43
Re: downloading using zlib compression
« Reply #4 on: March 28, 2009, 12:48:35 am »
aren't posts stored in text and depending on how the uploader encoded them couldn't it help normal downloading as well?

Offline Hecks

  • Contributor
  • ***
  • Posts: 2011
  • naughty cop
Re: downloading using zlib compression
« Reply #5 on: March 28, 2009, 01:16:00 am »
Read JG's post about it here:

http://forums.newsbin.com/viewtopic.php?p=144084#144084

He's the Diablo guy, after all. Includes an explanation of why new XZVER command was introduced for compressed headers, and implemented now by Astraweb. ;)

Also, you can answer your own question by doing a test for us.  Download the same file twice (or more) with Altbinz+SSL-stunnel, and Altbinz-SSL+stunnel and post the download times.
« Last Edit: March 28, 2009, 01:26:25 am by Hecks »

Offline DM8Mydog

  • Contributor
  • ***
  • Posts: 48
Re: downloading using zlib compression
« Reply #6 on: March 28, 2009, 02:14:47 am »
don't confuse clear text with yEnc
it's true clear text can be compressed a LOT simply because clear text has frequent repetitive words/phrases (ie anywhere bettween 10-50% of original text's size)
BUT yEnc text data aren't clear text

They're data, usually compressed data (.rars or .zips in the case of usenet binaries, even if the .rar compression level is set to 'store' it's contents are compressed (ie .avi or .wmv IS compressed data) ... in practice what you would gain by compressing yEnc data is probably a smaller yEnc overhead, if at all. ( I kinda doubt it'd really have any practical reduction )

So, compressing already compressed data wont have any effect, other than wasting cpu cycles.

Headers would really benefit from compression, because they are clear text, and since they have a lot of repetitive chunks in them they'd compress very well.
« Last Edit: March 28, 2009, 02:17:59 am by DM8Mydog »

Offline Hecks

  • Contributor
  • ***
  • Posts: 2011
  • naughty cop
Re: downloading using zlib compression
« Reply #7 on: March 28, 2009, 02:33:48 am »
Let me take a stab at putting it another way: yEnc encoding involves a quite simple shifting of byte values to turn the (already compressed) data files into ASCII representations of those files  The relationship between the new values is basically the same as the relationship between the original values - so the text is no more compressible than the data - and presumably that would follow through in the SSL-encoded data too.  Or so I guess from my imperfect understanding of yEnc, zlib and SSL. ;)

It should be easy enough to test this out at your end.  If you see a big speed boost, then we're all wrong. :P
« Last Edit: March 28, 2009, 02:45:00 am by Hecks »