I went in and unchecked the files I want to move and hit ok then rechecked them and it appears to be working. I've done it before and it didn't help but this time it did.
Also, I think there's a bug that if you uncheck auto-unrar then hit ok, then go back and check it, it disregards the file delete settings.