Alt.Binz forum
New Alt.Binz versions => Requests => Open requests => Topic started by: rygel16 on March 27, 2010, 07:45:35 pm
-
Since the password protected posts in Usenet get more and more (after unraring you end up with another archive and instructions to visit some site and fill surveys/click banners to get the password) I wonder if there's anything there can be done against it? For a while I helped myself with downloading series from upper "teevee" only, but recently this pw upper sh!thead started posting under the same id.
An idea:
With download sets > 700 MB alt.binz tries to download the first part in the RAR set first, decodes it. Then looks in the RAR file (not extracting) if there's another RAR set (around the same size as the download). If yes, pause the download with status "Possible password" and the user can unpause the download to continue, or cancel the download and search something else.
Edit: Here's a short screenvideo demonstrating how to identify pw protected uploads manually. Would love to see this process automated.
http://www.xup.in/dl,67033403/Video_2010-03-31_230003.wmv/
-
A big +1 on that. Very frustrating to find that what downloaded the previous night is useless.
-
+1
-
yep, definetly would be a nice feature
-
Since the password protected posts in Usenet get more and more (after unraring you end up with another archive and instructions to visit some site and fill surveys/click banners to get the password) I wonder if there's anything there can be done against it? For a while I helped myself with downloading series from upper "teevee" only, but recently this pw upper sh!thead started posting under the same id.
An idea:
With download sets > 700 MB alt.binz tries to download the first part in the RAR set first, decodes it. Then looks in the RAR file (not extracting) if there's another RAR set (around the same size as the download). If yes, pause the download with status "Possible password" and the user can unpause the download to continue, or cancel the download and search something else.
to make sense the pw check would need at least two levels:
1.) trying to look into the rar set ur downloading, if thats not posible -> output something like: "The collection you are downloading is pw protected! Please enter the PW!"and pause collection after that.
Otherwise:
2.) scaning the contents of the collection for rars; if found output something like: Possible pw protection found! Content of collection includes rarfiles.
even though i think having alt.binz check for pw protection would be a nice thing, it looks like to much work just to have a better solution to a minor problem
-
If we are using the UnRaR.dll I believe an event is raised in the event that a password is required. If we are using the Delphi component wrapper for UnRar.dll by Philippe Wechsler then the following event is thrown.
OnPasswordRequired(Sender: TObject; const HeaderPassword: Boolean; const FileName: WideString;out NewPassword: Ansistring; out Cancel: Boolean);
All this is great but I think we are still hosed. For this event to get tossed enough files have to be available to successfully attempt to extract the file. If we get one file and it only contains part of a larger file then we will see the "Required File missing" before we see the "Password Required". In this case we will still see false collections which are passworded getting downloaded; if we only check the first file downloaded. If we try and check the first 2, 3, 4, or 5 then that assumes they are in order and the files in the collection were stored as such.
I like Davidq666's idea of looking into the rar file. If we could get a listing of the files contained in the archive and see that little *filename then we may be golden.
ce
-
Nicely explained. Also small file could be listed in last archive :(
-
+1 much needed
If also that the first file to be downloaded can be suer set for the smallest file first. Just had to do this with breaking bad all at 15mb chose 23 last before .rar file which was 6mb. Quicker to check with smaller file if semi automated (via altbinz) or manualy done.
-
+1
Maybe you could just check about some files having special names like "click here for password.url", having like a blacklist of word combination where the user is prompted to cancel (or not) the current download.
-
+1
The best way to check these files, is indeed the see in the first rar, what the file extensions are. At times i also do it that way, but it ends up with way more work, especially when you have several tv episodes in serie ( like on monday ) in your batch.
To many of those protected rar's are showing up these days. Even worse is the fact that they take over the identical filename, and poster name. As a result, when you pick one of the last files uploaded, you end up with the "spam" rar file.
-
Same poster, same name? Just take the oldest one. This is what i do and i don't get bothered by PWed rars.
Anyway, you got my +1 on that request, might be useful for many ppl
-
+1!!
-
Sounds good. +1
-
+1 I would love this feature!
-
I don't think it would be that helpful because they now pack rars or other archives in the rar's you initially download and protect those with passwords. and that you can only see after extracting the archive inside the archive, but to do so you need the whole collection.
-
I hope it's OK to post this here (couldn't find any rules against it):
I just recently found a Tool called nzbSCAN (http://www.jongma.org/nzbscan/ (http://www.jongma.org/nzbscan/)) that does what you are asking for here.
If you just want to check if a download possibly has a password I'd recommend setting "scanpresent=0" in the ini-file. Anyway you have to setup some things in the ini first (like newsserver, username, password etc).
Just beware: This saves your password unencrypted (as of version 0.5) in the ini-file!!!
But as far as I can tell it doesn't steal your passwords (I checked the traffic with Wireshark).
It's a nice little tool although it sometimes says "possibly password-protected" when there's just another rar-file inside (without a password).
-
As others have pointed out, this is pretty useless in the most common situation where passworded RARs are packed in unpassworded ones, since these can only be checked after everything is dl'ed.
Although Alt.Binz might have an option to autopause these or require user confirmation to dl them, it could get annoying e.g. with subs or similar that often have another legit RAR inside. Generally it's better just to be sensible about where you source your NZBs, and learn from experience. It's not hard to recognize the dodgy ones. ;)
Edit: ah, I see the OP suggests comparing size of RAR within RAR. Note that most recent passworded crap has been in .ace format (WinAce). For straighforwardly passworded files you can of course check the output of 'unrar l foo.rar' for an asterisk next to the filename.
-
As others have pointed out, this is pretty useless in the most common situation where passworded RARs are packed in unpassworded ones, since these can only be checked after everything is dl'ed.
Not quite. The tool I mentioned above is able to detect password-protected RARs in unpassworded ones (I tested it myself). While it may not be 100% accurate, it is possible (and without downloading everything).
You can also determine it by hand if you just download the first part (even the first article is enough), unpack it and use the "keep broken files" option in WinRar. But it's really not that hard to tell even before the download (just stick to the oldest post for example).
Note that most recent passworded crap has been in .ace format (WinAce).
Hmm, I haven't come across those. But as I said just download the first part and if it contains an ace-file just skip the rest. ;)
-
Well, it's able to detect if a password was set for any file when it was added to the archive (i.e. the file is encrypted). So if a RAR is added and a password set for the RAR itself, then yes. But if the contents of that RAR were encrypted but the RAR itself wasn't, then no.
I have no idea what your tool does.
-
Would be nice, but although initially annoying I've found the best way for me, particularly with regular TV shows is to download the earliest version of it thats released, even if it's only a few hours earlier, it rarely fails me. Anything else it's a case of download the first rar and open it, if it contains a rar, ace, zip or any other archive it's a case of cancel and delete. I see it now as more of a minor nuisance than anything, but then I have a connection that allows me to view it as that, I can understand someone on a slower connection getting frustrated though.
-
nzbindex has this supported on their website searches. Guess there is no way to implement via this program? Or have I just not seen any yet?
-
@gazix
I think you need to read the thread again. Incidentally, Binsearch also does the same - passworded files where detectable have a [ * ] prepended to their subject in the search tab; and Mysterbin has a 'No pass' search option.
-
+1
Sorry if this is considered grave-digging since its been over 120 days since anyone posted here but it'd be awesome if a new release were able to check the first .rar file after downloading it for a "Click Here for Password File" and if it finds one, cancel the rest of the download not to waste bandwidth and space. Thanks guys and keep up the great work ;D
-
Morning,
Is there any way for Alt.Binz to check during (at the start of!) downloading to see if the content contained within the file set is encrypted or reqires a password ?
Too many times lately, file sets are unpacking to one fake file that claims requirement of a password.
Any way to save us the grief of multiple downloads ?
-
Or if was possible to auto check after download open the first rar part .r00 If any file named password inside aborts the download for the rest of the parts. Or pause that file continues download with the next file
-
+1
No clue how this could work in detail but it would be a great feature.
-
Hi,
As far as i remember, this has already been requested, i'm too lazy to search for you guys but it must be somewhere.
About those paswword encrypted downloads, here is a little hint :
When you search some content, you'll sometimes find multiple versions of the same post.
Download the oldest one, there is a chance that some guys downloaded it, encrypted and reupload it after. The oldest is usually (always?) not password protected.
Second hint : some tabs like NZBsearch and NZBclub has "not passworded" search option, check it !
-
Some good tips there, especially the grab oldest, that i find is usually the case.
There are a lot of these repacked passworded crap sets around, where the only files are one large fake content RAR file, and a text file saying download password here, with a link to some useless data mining site that will never give you anything.
If i hit one bad set, as soon as i unrar it asks to overwrite existing rar file, i cancel and delete the set, dont even bother going any further, after that i grab the next set on restricted speed and test open the first rar file, if it's another inside i trash the download, and try the next until i find a genuine one.
For me, it is not the ISP bandwidth or speed/time that annoys me with these, but that some of them are coming off my pre-paid "pay by download" astraweb account, so it is costing me for every fake piece of crap i download.
I sometimes wonder if a black list of posters would be handy
-
I just realized that SABnzbd on my NAS supports the "no password" functionality very effectively. So there must be a way to implement this in Alt.Binz, I guess. :)
Thanks for the idea to download the oldest fileset. ;D
Second hint : some tabs like NZBsearch and NZBclub has "not passworded" search option, check it !
-
I merged these threads to keep the discussion on track.
The hardest issue currently is that a main source of such spam is single-volume RARs *without passwords* packed within multi-volume RAR sets, with the first volume (usually '*.rar') deliberately removed by the spammer before posting.
This means that the whole set needs to be downloaded and PAR2 repaired to recreate the missing .rar before you can know what the single-volume archive contains ... because that's where the file header info is, and only there (the RAR spec does not include separate central directory file records at the end like ZIP does). Not even sabnzbd can make up missing bytes :P
And what it usually contains is a file that *does* need a password to extract, but you can't know that in advance .... these spammers are devious.
On the easier options, a solution is on the way, I gather.
Edit: another thing to note is that these spammers usually include a file like 'CLICK HERE FOR THE PASSWORD.html' at the end of the final RAR volume (e.g. '*.r26') so a strategy might be to download the last volume first and check its contents, but that's obviously a PITA.
-
That said older and yet some .rar archive voles that are the type *.r00 , *r.01 to *.rar opening the first *.r00 after it has downloaded will show the contents and able to navigate through any sub folders it has. After opening if parse contents any file with *password* then stop the download continue to next download.
Or even stop the download when check that if any of the enclosed files are password protected stop the download continue with next download. Alternative idea if unable to auto parse file for names within first downloaded file *.r00 when download rar set archives like this
-
Just to be clear, the *.rar is always the first in the set that includes *.r00, *.r01 etc, regardless of the upload order. File headers are spread across volumes, but always serially - files packed in early volumes will not have file headers in later volumes. So what you see in the .r00 depends entirely on how large the packed files are.
For reasons explained, this does not help at all if the archive only contains *one* file spread across all the volumes which is itself a RAR that contains passworded files. All that the file headers in the different volumes will tell you is that they contain a single RAR that isn't itself encrypted. What we're interested in is the contents of *that* RAR, and that info can only be in the first *.rar volume, where its contents actually start ... if that's missing, or if the packed RAR is compressed, you're out of luck.
If you want to see what I mean, try downloading this NZB (http://www.binsearch.info/?b=the.walking.dead.s03e15.hdtv.x264-asap&g=alt.binaries.multimedia&p=NoOneHere+%3Cno%40one.com%3E&max=250) and try inspecting each volume with this PHP class (https://github.com/zeebinz/rarinfo) or the like, or a hex editor if you know how to read the RAR specification (https://raw.github.com/cataphract/php-rar/master/technote.txt). Compare the contents of the .rar that Alt.Binz needs to create by PAR2 repair *after downloading all the volumes* (see the small PAR2 to confirm that the .rar is needed) with the .r00 that's actually included in the NZB.
And no, opening a volume in WinRAR or similar doesn't give an accurate view of what's actually in each volume, as those proggies inspect all the volumes in the same directory and collate their file headers.
-
Yes that one doesn't have any other files likely in the sub .rar file only accessible when the download is complete. I also download the same name but with padlock to see difference from BS I note that one has two files one a sub rar the other a text. No mention in either archives of the word password, which puts to bed that that wasn't possible anyway. These should be auto checked and erased from the servers after upload when found to be password protected files. Problem there is what of those that belong to groups that do password protect their files for whatever reason they think they should have password protection. Maybe it should be no files ever allowed to be uploaded that contain any passworded archives at any level of the archive !
-
Yep, as you point out NZBIndex does mark that same BS example as password protected. The last volume, .r26, actually contains a 'CLICK HERE FOR PASSWORD.html' right at the end, I guess they scanned for that, which is easy enough when you're only indexing ... or they applied some other fuzzy guesswork, like if the PAR2 contains .rar but the collection doesn't, then it's probably suspect.
Alt.Binz could do the same: check NZB & PAR2 for .rar if it contains .r00; if missing, check last .r** for any suspect files, etc. But I suspect in the long run it would just be a game of whack-a-mole with the spammers. Far better to identify trusted sources for NZBs and stick with those, methinks.
Edit: another passworded version of that example on NZBIndex contains a Readme.txt right at the start of the first volume, *.part01.rar in this case, with a 'please download password from here' message. That's an easy one to detect, since the word 'password' appears in plain text within the first 200 bytes of the volume.
-
Finally some movement on this.
Few detect cases are implemented: Downloaded rar(.rar .partxxx.rar .001) is password protected, rar has files inside that have word 'password' in the name, rar has small files inside that have word 'password' in the content.
Few more methods will be added later.
Option is located in Setup->Misc #3->Rar password checker mode
You have options for disabled, Warning only and Warning+Pause collection. Default is disabled.
-
Love the password detection features added. But i have a question, if you set it to Warn + pause, will it pause collections you have added the password for ? or only collections with no password added for them.
Thanks :)
-
If password is set for collection, no checking/warning/pausing applies.
-
Ahh awesome, thanks RDL, i didn't want to set it to pause until i found out :)
-
Just quickly scanned over the 3 pages for this thread, so sorry if just rehashing what someone had already brought up.
Was wondering if could have an option to pause (no warning) and change the colour of the pause icon to x instead of blue? I guess was hoping to have the same thing with no enough par files. Meaning no error message at all, and just pause and change to a different colour pause icon.
Also, in terms of the password searching feature, is there a way to check for a file with the name password within the archives without downloading the whole thing first? I notice that sometimes there is a single pw protected rar file within the rar set itself. Along with a file (usually html) say something like click here for password....
Thanks
-
Oops, one more ask. Would be possible to just delete the files off the computer (or when delete from the paused queue in alt binz) for the not enough pars and/or password protected files? Obviously either isn't really useful since can't really do anything with them.
-
@gazix The problem is that the .html with the links to scam password sites is typically right at the end of the archive, in the last segment of the final .rar in the set. I guess Alt.Binz could try downloading that one first, but this would probably confuse a lot of users. Remember also the whole category of spam posts that can't be scanned properly because of rars deliberately removed by posters to force a whole download and par2 repair. Any compression applied also complicates things.
Having spent rather too much of my life implementing different ways for detecting spam, I can say with confidence: the only guaranteed way to have a trouble-free life on usenet is to make sure you get your NZBs from *trusted sources only*. If you're getting unexpected passworded files to the extent that you need some automated checking like this, it's a 100% sure sign that you're going about things the wrong way and it's time to rethink your whole approach.
Just my $0.02 :)
-
Ahh, thanks for the heads up. Shows what I know, figured that everything would be have to be listed within the first few files.... I guess downloading the first and last file wouldn't be any solution :)
-
To warn by a colour on download queue and par2 queue for those archives found to need a password to extract the contents. No pop-up to be generated for any warning. Warning could be placed on ui grab bar (top bar of alt.binz ui window) after Alt.Binz version## where it would also be seen. For the tray as does for each download when completed that does remain until clicked away
For downloads (such as episodes) that are downloaded from one group. So that all archives with passwords are paused and allow all other archives in the a group to continue downloading
-
@gazix The problem is that the .html with the links to scam password sites is typically right at the end of the archive, in the last segment of the final .rar in the set. I guess Alt.Binz could try downloading that one first, but this would probably confuse a lot of users. Remember also the whole category of spam posts that can't be scanned properly because of rars deliberately removed by posters to force a whole download and par2 repair. Any compression applied also complicates things.
Having spent rather too much of my life implementing different ways for detecting spam, I can say with confidence: the only guaranteed way to have a trouble-free life on usenet is to make sure you get your NZBs from *trusted sources only*. If you're getting unexpected passworded files to the extent that you need some automated checking like this, it's a 100% sure sign that you're going about things the wrong way and it's time to rethink your whole approach.
Just my $0.02 :)
So true. :)
-
I'm guessing that the rar checker mode won't have a warning/pause option that doesn't include the hidden option of "always be on top" mode? Never seen an program another program has a pop up dialog box that even shows up over top of the screen saver (while screen saver continues to run uninterrupted).
Also thinking that the algorithm for determining what is password protected and what is not might need an update.