+++++ Guide to Downloading Projects from Usenet +++++

IMPORTANT: You must purchase the official Blu-rays or digital HD releases of the films in question before downloading theatrical preservations and fan edits. The projects discussed in these forums must never be bought or sold. I do not condone piracy of any kind.

Introduction

The Users’ Network, or Usenet for short, is a non-centralised computer network of news servers created in 1980 for discussing various topics in newsgroups. After a message is posted to a newsgroup on a news server, it is copied to all the other news servers hosting that particular newsgroup. It can then be viewed by anyone with an account on one of these news servers. While such discussions now usually take place on internet forums instead, newsgroups have been increasingly used for file-sharing due to dramatic increases in internet connection speed over the last 15 years. The same technology used for sharing messages in the original text-based newsgroups has been adapted for sharing much larger music, video and application files in binary newsgroups. Content is uploaded and downloaded using software called a newsreader.

Brief Overview to Downloading from Usenet

To download projects from Usenet, you will need two things:

an account with a news server;
a newsreader.

I recommend starting with Free-Usenet for the news server. No personal details are required other than a valid email address. It can be used completely free of charge for as long as you like and with no download limits, albeit at a low speed of 1 Mbps (~122 KB/s). A 10 GB file would take roughly one day to download; much higher speeds are available at a price. Alternatively, you can sign up for a free trial with one of the major Usenet providers. This will give you very fast downloads at no cost for a limited time (usually up to one week), although possibly with a data cap.

NZBGet is an excellent newsreader for Windows, MacOS, Linux and Android. It is a very small program to install (only 8 MB), has a user-friendly interface, takes up few system resources, and is freeware. Note that NZBGet is a download client only, and cannot be used to upload files to newsgroups.

Once these are set up, acquiring projects is a two-stage process.

A Usenet search engine, also called an indexer, is used to track down the relevant project. It provides a small NZB file for you to download, containing links to all the project files on the news server. NZB files are named after Newzbin, the first Usenet indexer to develop and use this format. A good free indexer for Original Trilogy projects is NZBKing.
The NZB file is loaded into the newsreader, at which point the main download from the news server begins. Once this is complete, the newsreader will automatically verify your downloaded files against those stored on the news server, attempt to repair them if necessary, and finally unpack any RAR files.

Step-by-Step Guide

Registering with a news server.

Sign up for a free account with Free-Usenet.
Check your email for your password and log in to Free-Usenet.
Click on “Free account”.
Click on “I’m not a robot” and complete the CAPTCHA.
Click on “Generate free account”.
The server username and password listed here will be needed by your newsreader.
Every 3 hours, your free subscription will expire and must be renewed from this page.

Configuring a newsreader.

Download and install the latest stable release of NZBGet.
Run NZBGet. The user interface should appear in your web browser.
Click on “Settings” in the top bar.
Click on “NEWS-SERVERS” in the left-hand panel.
Complete the following settings:

    Server1.Name:        A server name such as "Free-Usenet" (optional)
    Server1.Host:        news.free-usenet.com
    Server1.Port:        443 (used for an SSL-encrypted connection)
    Server1.Username:    The username listed in your Free-Usenet account details
    Server1.Password:    The password listed in your Free-Usenet account details
    Server1.Encryption:  Yes (to use SSL encryption for more security)
    Server1.Connections: 1 (the free account won't accept more than 1)

Click on “Test Connection” near the bottom to check the settings are correct.
Click on “Save all changes” at the bottom left followed by “Reload NZBGet”.

Searching for a project with a Usenet indexer.

Visit NZBKing.
Input some key words for the project in the search bar and press Return.
Optional: click on “NFO” (if present) beside a project to read some information about it.
Optional: click on “Details” to see the project files, the upload date and the exact file size.
Click on the “NZB” button by the relevant project to download the NZB file.

Downloading a project.

Run NZBGet if it isn’t already open.
Click on “+Add”.
Drag and drop the NZB file onto the pop-up window, or click on “Select file” and navigate to it.
Click on “Submit”. The download should begin: check the speed by the aeroplane at the top left.
Every 3 hours, your Free-Usenet account will expire and should be renewed as follows:
- Shortly before the 3 hours are up, click on the big green arrow to pause the download.
- Wait for your Free-Usenet account to expire, and then renew it for another 3 hours.
- Enter the new password in NZBGet’s news server settings.
- Click on “Save all changes” followed by “Reload NZBGet”.
- Click on the big orange arrow to resume the download.
When the download is finished, it may take a few minutes for the RAR files to be verified,
repaired (if needed) and finally unpacked to this directory: C:\ProgramData\NZBGet\complete.

Additional notes on Free-Usenet account expiry.

If you don’t pause the download before your Free-Usenet account expires, NZBGet will start to think that articles are missing on the server, rather than realise that the server password is no longer valid. Consequently, the health of the download will slowly reduce from 100%. You can nevertheless follow the procedure in step 4 to renew your account and resume the download, the only difference being that NZBGet will have to repair the “missing” articles at the very end by acquiring some Par2 files.
If the health ever drops below the critical value (usually 90%, assuming there are enough Par2 files to repair 10% of the download), NZBGet will mark the status as “Failure” and move the download to the “History” tab. At this point, follow these steps to recover the download:
- Renew your Free-Usenet account for another 3 hours.
- Enter the new password in NZBGet’s news server settings.
- Click on “Save all changes” followed by “Reload NZBGet”.
- Click on the “History” tab and find the failed download.
- Click on “Failure” and select “Retry failed articles”.
- You should now find that the health is back to 100%.

Appendix 1: Usenet vs Torrents & Cloud Storage

Usenet has a number of advantages over other popular methods of downloading files.

Usenet is more private and secure than file-sharing via torrents. When downloading from Usenet, you are connected only to the news server and not to other downloaders (known as “peers”) as you would be with a torrent. There is thus no automatic uploading of content to other peers while you are downloading from Usenet.
News servers on Usenet usually have a high retention: files are often available for 8 to 10 years after being uploaded. Torrents are only available while they are being seeded and can dry up within a few weeks. Download links on cloud storage platforms such as Mega can also expire without warning if the owner either deletes or stops sharing the files.
Downloading from Usenet via a paid account with a news server is very fast, while torrents can be slow depending on the number and speed of the available peers. (That said, downloading from Usenet with a completely free account is generally rather slow.)
Usenet features the powerful verification and recovery of damaged or missing files, handled automatically by the newsreader. While torrents allow automatic file verification via checksums, there is no facility to repair damaged files. Content obtained from the cloud cannot usually be checked for download errors.

Appendix 2: Useful Settings in NZBGet

Completed and ongoing downloads are stored in C:\ProgramData\NZBGet\complete and C:\ProgramData\NZBGet\intermediate respectively. You can change these directories with DestDir and InterDir in the PATHS settings.

By default, NZBGet deletes the downloaded RAR files once they have been successfully unpacked. It also downloads the bare minimum number of Par2 files needed for file verification and repair (if needed), deleting them afterwards along with any SFV checksum files present.

To keep the RAR files after unpacking, set UnpackCleanupDisk to No in the UNPACK settings.
To download all the Par2 files and also keep them afterwards (along with SFV checksum files), set ParCheck to Force in the CHECK AND REPAIR settings, and empty ExtCleanupDisk in the UNPACK settings.

Appendix 3: File Verification and Recovery with Usenet

The successful recovery of partially corrupted files is a huge advantage of Usenet. Large files and folders are archived and split into multi-part RAR files before being uploaded to binary newsgroups such as alt.binaries.starwars. Uploaders usually include some “parity archive (version 2) files”, or Par2 files for short, alongside the RAR files. These are designed to help newsreaders with file verification and recovery at the end of the download process.

The newsreader first downloads the RAR files in pieces called blocks, verifying checksums at each stage. Occasionally, some blocks will be missing or damaged on the news server. Provided there are at least as many blocks in the available Par2 files as there are missing/damaged blocks in the RAR files, the newsreader will be able to completely restore the latter by downloading the required number of Par2 files. Uploaders are encouraged to produce enough Par2 files to repair 10% of the project. Hence there is no reason to panic if a Usenet indexer lists a project as being slightly incomplete.

Here is an extremely simplified example to illustrate the basic idea behind Par2 recovery. Suppose the file you wish to download consists of the following four blocks of eight bits:

Block
1 0 1 0 1 1 0 1
0 1 1 1 0 1 0 0
0 0 1 1 0 0 0 0
1 0 1 0 0 1 0 1

Here they are with the row and column totals included:

Block
1 0 1 0 1 1 0 1 = 5
0 1 1 1 0 1 0 0 = 4
0 0 1 1 0 0 0 0 = 2
1 0 1 0 0 1 0 1 = 4
= = = = = = = =
2 1 4 2 1 3 0 2

The row totals can be written in binary, acting as simple checksums for each block. The parities of the column totals are now shown, with 0 for “even” and 1 for “odd”. This block of parities forms a Par2 file which has been provided by the uploader.

Block             Checksum
1 0 1 0 1 1 0 1 = 101
0 1 1 1 0 1 0 0 = 100
0 0 1 1 0 0 0 0 = 010
1 0 1 0 0 1 0 1 = 100
= = = = = = = =
0 1 0 0 1 1 0 0 ← Par2

Imagine the second block in the file has been corrupted on the news server. It might be entirely missing, or perhaps slightly damaged with an erroneous third bit.

Missing 2nd block                       Damaged 2nd block (3rd bit)

Block             Checksum              Block             Checksum
1 0 1 0 1 1 0 1 = 101                   1 0 1 0 1 1 0 1 = 101
x x x x x x x x = xxx                   0 1(0)1 0 1 0 0 = 100
0 0 1 1 0 0 0 0 = 010                   0 0 1 1 0 0 0 0 = 010
1 0 1 0 0 1 0 1 = 100                   1 0 1 0 0 1 0 1 = 100
= = = = = = = =                         = = = = = = = =
0 1 0 0 1 1 0 0 ← Par2                  0 1 0 0 1 1 0 0 ← Par2

As the newsreader downloads the blocks and verifies the checksums, it becomes apparent that there is a problem with the second block: the data is either missing or inconsistent with the checksum. The Par2 file now comes to the rescue, allowing the newsreader to completely restore the corrupted block. The three good blocks are added to the Par2 block bit by bit, and the parities of the column totals reproduce the original second block.

Block
1 0 1 0 1 1 0 1 ← good 1st block
0 0 1 1 0 0 0 0 ← good 3rd block
1 0 1 0 0 1 0 1 ← good 4th block
0 1 0 0 1 1 0 0 ← Par2 recovery block
= = = = = = = =
0 1 1 1 0 1 0 0 ← recovered 2nd block

Further information on the Par2 recovery process can be found here: http://www.quickpar.org.uk/.

++++++++++++++++++++++++
Bluto, Original Trilogy Forums
September 2019
++++++++++++++++++++++++