Sign In

Guide to Downloading Projects from Usenet

Author
Time

Over the last couple of years, it has become increasingly difficult to track down the Star Wars preservations and fan edits discussed in the Original Trilogy forums. Invites to MySpleen, a private torrent tracker, have been closed since September 2017; many links on Uloz, a file-sharing site, have long since expired. While some content can be found on public torrent trackers, it is often re-encoded and of poorer quality.

One solution to acquire many of these projects is via binary newsgroups on Usenet. However, the learning curve can be a little steep with plenty of jargon to wade through. I have therefore written the following guide to provide fellow members of the OT community with all the information they need to find and download projects on Usenet, and hope it is genuinely useful.

If you feel anything in the guide below is incorrect, misleading or poorly worded, please do send me a private message (click on my avatar) and I shall endeavour to improve it.

Note: while Harmy’s popular Despecialized Editions are available on Usenet, NJVC’s Blu-ray ISO versions of these projects are not. Details on how to obtain the latter can be obtained from Solkap’s thread in the Star Wars Preservation forum.

Bluto

Author
Time
 (Edited)

+++++ Guide to Downloading Projects from Usenet +++++

Note: you must have purchased the official Blu-ray or digital HD releases of the films in question before downloading HD preservations and fan edits. The projects discussed in these forums must never be bought or sold.

INTRODUCTION

The Users’ Network, or Usenet for short, is a non-centralised computer network of news servers created in 1980 for discussing various topics in newsgroups. After a message is posted to a newsgroup on a news server, it is copied to all the other news servers hosting that particular newsgroup. It can then be viewed by anyone with an account on one of these news servers. While such discussions now usually take place on internet forums instead, newsgroups have been increasingly used for file-sharing due to dramatic increases in internet connection speed over the last 15 years. The same technology used for sharing messages in the original text-based newsgroups has been adapted for sharing much larger music, video and application files in binary newsgroups such as alt.binaries.starwars. Content is uploaded and downloaded using software called a newsreader.

BRIEF OVERVIEW TO DOWNLOADING FROM USENET

To download projects from Usenet, you will need two things:

  • an account with a news server;
  • a newsreader.

I recommend starting with Free-Usenet for the news server. No personal details are required other than a valid email address. It can be used completely free of charge for as long as you like and with no download limits, albeit at a low speed of 1 Mbps (128 kB/s). A 10 GB file would take roughly one day to download; much higher speeds are available at a price. Alternatively, you can sign up for a free trial with one of the major Usenet providers. This will give you very fast downloads at no cost for a limited time (usually up to one week), although possibly with a data cap.

NZBGet is an excellent newsreader for Windows, MacOS, Linux and Android. It is a very small program to install (only 8 MB), has a user-friendly interface, takes up few system resources, and is freeware. Note that NZBGet is a download client only, and cannot be used to upload files to newsgroups.

Once these are set up, acquiring projects is a two-stage process.

  1. A Usenet search engine, also called an indexer, is used to track down the relevant project. It provides a small NZB file for you to download, containing links to all the project files on the news server. NZB files are named after Newzbin, the first Usenet indexer to develop and use this format. My favourite indexer for Original Trilogy projects is NZBKing.
  2. The NZB file is loaded into the newsreader, at which point the main download from the news server begins. Once complete, the newsreader will automatically verify the downloaded files, attempt to repair them if necessary, and finally unpack any RAR files.

STEP-BY-STEP GUIDE

  1. Registering with a news server.
  • Create a free account with Free-Usenet.
  • Log in and click on “Free account” to see your account details.
  • Click “I’m not a robot”.
  • Click “Renew free account (extend for 3 hours)”.
  • Click “OK”.
  1. Configuring a newsreader.
  • Download and install the latest stable release of NZBGet.
  • Run NZBGet. The user interface should appear in your web browser.
  • Click “Settings” in the top bar.
  • Click “NEWS-SERVERS” in the list on the left-hand side.
  • Complete the following settings:
    Server1.Name:        A server name such as "Free-Usenet" (optional)
    Server1.Host:        news.free-usenet.com
    Server1.Port:        443 (used for an SSL-encrypted connection)
    Server1.Username:    The username listed in your Free-Usenet account details
    Server1.Password:    The password listed in your Free-Usenet account details
    Server1.Encryption:  Yes (to use SSL encryption for more security)
    Server1.Connections: 1 (the free account won't accept more than 1)
  • Click “Save all changes” at the bottom.
  • Click “Reload NZBGet”.
  1. Searching for a project with a Usenet indexer.
  • Visit NZBKing.
  • Input some key words for the project in the search bar and press Return.
  • Click “NFO” (if present) beside a project to read some information about it.
  • Click “Details” to see the project files, the upload date and the exact file size.
  • Click “NZB” (by the arrow in the blue square) to download the relevant NZB file.
  1. Downloading a project.
  • Run NZBGet if it isn’t already open.
  • Click “+Add”.
  • Drag and drop the NZB file onto the pop-up window, or click “Select file” and navigate to it.
  • Click “Submit”. The download should begin: check the speed at the top left.
  • You can pause/resume the download by clicking the big green/orange arrow at the top.
  • Every 3 hours, renew your Free-Usenet account and update the server password in NZBGet.
  • When the download is finished, it may take a few minutes for the RAR files to be verified,
    repaired (if needed) and finally unpacked to this directory: C:\ProgramData\NZBGet\complete.

APPENDIX 1: POPULAR STAR WARS PROJECTS AVAILABLE ON USENET

The file sizes and uploaders are included to help you identify legitimate uploads rather than poorer quality re-encodes. Note that the file sizes include the Par2 recovery files.

Project Name Format Size Uploader
Harmy’s Despecialized Editions of SW v2.7, ESB v2.0, ROTJ v2.5 MKV ~25 GB each Obi-Have Kenobi
Harmy’s Despecialized Editions of SW v2.7, ESB v2.0, ROTJ v2.5 AVCHD 8 GB each Mallwalker
Chewtobacca’s DVD versions of Harmy’s SW v2.7 and ROTJ v2.5 DVD5 4 GB each Mallwalker
Team Negative 1’s 4K77 1080p No DNR v1.4 (use Binsearch) MKV 43 GB SomethingSomeone
Team Negative 1’s 4K77 1080p DNR v1.0 MKV 48 GB Obi-Have Kenobi
Team Negative 1’s 4K83 1080p No DNR v1.0 MKV 48 GB Obi-Have Kenobi
Team Negative 1’s SW Silver Screen Edition v1.6 BD50 36 GB Yenc-PP-A&A
Team Negative 1’s ESB Renegade Grindhouse BD25 23 GB Mallwalker

Note:

  • Chewtobacca’s DVD5 version of ESB v2.0 does not appear to have been uploaded to Usenet. However, it can be obtained by following the instructions in section 4, step 7 of HanDuet’s Ultimate Guide.

APPENDIX 2: USENET VS TORRENTS & CLOUD STORAGE

Usenet has a number of advantages over other popular methods of downloading files.

  • Usenet is more private and secure than file-sharing via torrents. When downloading from Usenet, you are connected only to the news server and not to other downloaders (known as “peers”) as you would be with a torrent. There is thus no automatic uploading of content to other peers while you are downloading from Usenet.

  • News servers on Usenet usually have a high retention: files are often available for 8 to 10 years after being uploaded. Torrents are only available as long as they are being seeded and can often dry up within a few weeks. Download links on cloud storage platforms such as Mega can also expire without warning if the owner either deletes or stops sharing the files.

  • Downloading from Usenet via a paid account with a news server is very fast, while torrents can be slow depending on the number and speed of the available peers. (That said, downloading from Usenet with a completely free account is generally rather slow.)

  • Usenet features powerful verification and recovery of damaged or missing files, handled automatically by the newsreader. While torrents allow automatic file verification via checksums, there is no facility to repair damaged files. Content obtained from the cloud cannot usually be checked for download errors.

APPENDIX 3: USEFUL SETTINGS IN NZBGET

Completed and ongoing downloads are stored in C:\ProgramData\NZBGet\complete and C:\ProgramData\NZBGet\intermediate respectively. You can change these directories with DestDir and InterDir in the PATHS settings.

By default, NZBGet deletes the downloaded RAR files once they have been successfully unpacked. It also downloads the bare minimum number of Par2 files needed for file verification and repair (if needed), deleting them afterwards along with any SFV checksum files present.

  • To keep the RAR files after unpacking, set UnpackCleanupDisk to No in the UNPACK settings.
  • To download all the Par2 files and also keep them afterwards, set ParCheck to Force in the CHECK AND REPAIR settings, and empty ExtCleanupDisk in the UNPACK settings.

APPENDIX 4: FILE VERIFICATION AND RECOVERY WITH USENET

The successful recovery of partially corrupted files is a huge advantage of Usenet. Large files are archived and split into multi-part RAR files before being uploaded to binary newsgroups such as alt.binaries.starwars. Uploaders usually include some “parity archive (version 2) files”, or Par2 files for short, alongside the RAR files. These are designed to help newsreaders with file verification and recovery at the end of the download process.

The newsreader first downloads the RAR files in pieces called blocks, verifying checksums at each stage. Occasionally, some of the blocks will be missing or damaged on the news server. Provided there are at least as many blocks in the available Par2 files as there are missing/damaged blocks in the RAR files, the newsreader will be able to completely restore the latter by downloading the required number of Par2 files. Uploaders are encouraged to produce enough Par2 files to repair 10% of the project. Hence there is no reason to panic if a Usenet indexer lists a project as being slightly incomplete.

Here is an extremely simplified example to illustrate the basic idea behind Par2 recovery. Suppose the file you wish to download consists of the following four blocks of eight bits:

Block
1 0 1 0 1 1 0 1
0 1 1 1 0 1 0 0
0 0 1 1 0 0 0 0
1 0 1 0 0 1 0 1

Here they are with the row and column totals included:

Block
1 0 1 0 1 1 0 1 = 5
0 1 1 1 0 1 0 0 = 4
0 0 1 1 0 0 0 0 = 2
1 0 1 0 0 1 0 1 = 4
= = = = = = = =
2 1 4 2 1 3 0 2

The row totals can be written in binary, acting as simple checksums for each block. The parities of the column totals are now shown, with 0 for “even” and 1 for “odd”. This block of parities forms a Par2 file which we shall assume has been provided by the uploader.

Block             Checksum
1 0 1 0 1 1 0 1 = 101
0 1 1 1 0 1 0 0 = 100
0 0 1 1 0 0 0 0 = 010
1 0 1 0 0 1 0 1 = 100
= = = = = = = =
0 1 0 0 1 1 0 0 ← Par2

Imagine the second block in the file has been corrupted on the news server. It might be entirely missing, or perhaps slightly damaged with an erroneous third bit.

Missing 2nd block                       Damaged 2nd block (3rd bit)

Block             Checksum              Block             Checksum
1 0 1 0 1 1 0 1 = 101                   1 0 1 0 1 1 0 1 = 101
x x x x x x x x = xxx                   0 1(0)1 0 1 0 0 = 100
0 0 1 1 0 0 0 0 = 010                   0 0 1 1 0 0 0 0 = 010
1 0 1 0 0 1 0 1 = 100                   1 0 1 0 0 1 0 1 = 100
= = = = = = = =                         = = = = = = = =
0 1 0 0 1 1 0 0 ← Par2                  0 1 0 0 1 1 0 0 ← Par2

As the newsreader downloads the blocks and verifies the checksums, it becomes apparent that there is a problem with the second block: the data is either missing or inconsistent with the checksum. The Par2 file now comes to the rescue, allowing the newsreader to completely restore the corrupted block. The three good blocks are added to the Par2 block bit by bit, and the parities of the column totals reproduce the original second block.

Block
1 0 1 0 1 1 0 1 ← good 1st block
0 0 1 1 0 0 0 0 ← good 3rd block
1 0 1 0 0 1 0 1 ← good 4th block
0 1 0 0 1 1 0 0 ← Par2 recovery block
= = = = = = = =
0 1 1 1 0 1 0 0 ← recovered 2nd block

Further information on the Par2 recovery process can be found here: http://www.quickpar.org.uk/.

++++++++++++++++++++++++
Bluto, Original Trilogy Forums
September 2019
++++++++++++++++++++++++