This might be a stupid question, but is there a password and if so do you know what it is?
If the files downloaded, you don't need to be dealing with this in jdownloader anymore. Just go to the folder that it downloaded to and try to open with winrar and obviously change the file extension back if you messed it up.