Sign In

Guide to Downloading Projects from Usenet

Author
Time
 (Edited)

Over the last couple of years, it has become increasingly difficult to track down the Star Wars preservations and fan edits discussed in the Original Trilogy forums. Invites to MySpleen, a private torrent tracker, have been closed since September 2017; many links on Uloz, a file-sharing site, have long since expired. While some content can be found on public torrent trackers, it is often re-encoded and of poorer quality.

One solution to acquire many of these projects is via binary newsgroups on Usenet. However, the learning curve can be a little steep with plenty of jargon to wade through. I have written the following guide to help fellow members of the OT community get started with Usenet, and hope it is useful.

If you feel anything in the guide below is incorrect, misleading or poorly worded, please do send me a private message (click on my avatar) and I shall endeavour to improve it.

Note: while Harmy’s popular Despecialized Editions are available on Usenet, NJVC’s Blu-ray ISO versions of these projects are not. Details of how to obtain the latter can be obtained from Solkap’s thread in the Star Wars Preservation forum.

Bluto

Author
Time
 (Edited)

+++++ Guide to Downloading Projects from Usenet +++++

IMPORTANT: You must have purchased the official Blu-ray or digital HD releases of the films in question before downloading HD preservations and fan edits. The projects discussed in these forums must never be bought or sold.

INTRODUCTION

The Users’ Network, or Usenet for short, is a non-centralised computer network of news servers created in 1980 for discussing various topics in newsgroups. After a message is posted to a newsgroup on a news server, it is copied to all the other news servers hosting that particular newsgroup. It can then be viewed by anyone with an account on one of these news servers. While such discussions now usually take place on internet forums instead, newsgroups have been increasingly used for file-sharing due to dramatic increases in internet connection speed over the last 15 years. The same technology used for sharing messages in the original text-based newsgroups has been adapted for sharing much larger music, video and application files in binary newsgroups such as alt.binaries.starwars. Content is uploaded and downloaded using software called a newsreader.

BRIEF OVERVIEW TO DOWNLOADING FROM USENET

To download projects from Usenet, you will need two things:

  • an account with a news server;
  • a newsreader.

I recommend starting with Free-Usenet for the news server. No personal details are required other than a valid email address. It can be used completely free of charge for as long as you like and with no download limits, albeit at a low speed of 1 Mbps (128 kB/s). A 10 GB file would take roughly one day to download; much higher speeds are available at a price. Alternatively, you can sign up for a free trial with one of the major Usenet providers. This will give you very fast downloads at no cost for a limited time (usually up to one week), although possibly with a data cap.

NZBGet is an excellent newsreader for Windows, MacOS, Linux and Android. It is a very small program to install (only 8 MB), has a user-friendly interface, takes up few system resources, and is freeware. Note that NZBGet is a download client only, and cannot be used to upload files to newsgroups.

Once these are set up, acquiring projects is a two-stage process.

  1. A Usenet search engine, also called an indexer, is used to track down the relevant project. It provides a small NZB file for you to download, containing links to all the project files on the news server. NZB files are named after Newzbin, the first Usenet indexer to develop and use this format. My favourite indexer for Original Trilogy projects is NZBKing.
  2. The NZB file is loaded into the newsreader, at which point the main download from the news server begins. Once this is complete, the newsreader will automatically verify your downloaded files against those stored on the news server, attempt to repair them if necessary, and finally unpack any RAR files.

STEP-BY-STEP GUIDE

  1. Registering with a news server.
  • Sign up for a free account with Free-Usenet.
  • Check your email for your password and log in to Free-Usenet.
  • Click on “Free account”.
  • Click on “I’m not a robot” and complete the CAPTCHA.
  • Click on “Generate free account”.
    The server username and password listed here will be needed by your newsreader.
    Every 3 hours, your free subscription will expire and must be renewed from this page.
  1. Configuring a newsreader.
  • Download and install the latest stable release of NZBGet.
  • Run NZBGet. The user interface should appear in your web browser.
  • Click on “Settings” in the top bar.
  • Click on “NEWS-SERVERS” in the left-hand panel.
  • Complete the following settings:
    Server1.Name:        A server name such as "Free-Usenet" (optional)
    Server1.Host:        news.free-usenet.com
    Server1.Port:        443 (used for an SSL-encrypted connection)
    Server1.Username:    The username listed in your Free-Usenet account details
    Server1.Password:    The password listed in your Free-Usenet account details
    Server1.Encryption:  Yes (to use SSL encryption for more security)
    Server1.Connections: 1 (the free account won't accept more than 1)
  • Click on “Test Connection” near the bottom to check the settings are correct.
  • Click on “Save all changes” at the bottom left.
  • Click on “Reload NZBGet”.
  1. Searching for a project with a Usenet indexer.
  • Visit NZBKing.
  • Input some key words for the project in the search bar and press Return.
  • Optional: click on “NFO” (if present) beside a project to read some information about it.
  • Optional: click on “Details” to see the project files, the upload date and the exact file size.
  • Click on “NZB” (by the arrow in the blue square) to download the relevant NZB file.
  1. Downloading a project.
  • Run NZBGet if it isn’t already open.
  • Click on “+Add”.
  • Drag and drop the NZB file onto the pop-up window, or click on “Select file” and navigate to it.
  • Click on “Submit”. The download should begin: check the speed by the aeroplane at the top left.
  • You can pause/resume the download by clicking on the big green/orange arrow at the top left.
  • Every 3 hours, renew your Free-Usenet account and update the server password in NZBGet.
  • When the download is finished, it may take a few minutes for the RAR files to be verified,
    repaired (if needed) and finally unpacked to this directory: C:\ProgramData\NZBGet\complete.

APPENDIX 1: POPULAR STAR WARS PROJECTS AVAILABLE ON USENET

The file sizes and uploaders are included to help you identify legitimate uploads rather than poorer quality re-encodes. Note that the file sizes include the Par2 recovery files.

Project Name Format Size Uploader
Harmy’s Despecialized Editions of SW v2.7, ESB v2.0, ROTJ v2.5 MKV ~25 GB each Obi-Have Kenobi
Harmy’s Despecialized Editions of SW v2.7, ESB v2.0, ROTJ v2.5 AVCHD 8 GB each Mallwalker
Chewtobacca’s DVD versions of Harmy’s SW v2.7 and ROTJ v2.5 DVD5 4 GB each Mallwalker
Team Negative 1’s 4K77 1080p No DNR v1.4 (use Binsearch) MKV 43 GB SomethingSomeone
Team Negative 1’s 4K77 1080p DNR v1.0 MKV 48 GB Obi-Have Kenobi
Team Negative 1’s 4K83 1080p No DNR v1.0 MKV 48 GB Obi-Have Kenobi
Team Negative 1’s SW Silver Screen Edition v1.6 BD50 36 GB Yenc-PP-A&A
Team Negative 1’s ESB Renegade Grindhouse BD25 23 GB Mallwalker

Note:

  • Chewtobacca’s DVD5 version of ESB v2.0 does not appear to have been uploaded to Usenet. However, it can be obtained by following the instructions in section 3, step 7 of HanDuet’s The Ultimate Introductory Guide.

APPENDIX 2: USENET VS TORRENTS & CLOUD STORAGE

Usenet has a number of advantages over other popular methods of downloading files.

  • Usenet is more private and secure than file-sharing via torrents. When downloading from Usenet, you are connected only to the news server and not to other downloaders (known as “peers”) as you would be with a torrent. There is thus no automatic uploading of content to other peers while you are downloading from Usenet.

  • News servers on Usenet usually have a high retention: files are often available for 8 to 10 years after being uploaded. Torrents are only available while they are being seeded and can dry up within a few weeks. Download links on cloud storage platforms such as Mega can also expire without warning if the owner either deletes or stops sharing the files.

  • Downloading from Usenet via a paid account with a news server is very fast, while torrents can be slow depending on the number and speed of the available peers. (That said, downloading from Usenet with a completely free account is generally rather slow.)

  • Usenet features powerful verification and recovery of damaged or missing files, handled automatically by the newsreader. While torrents allow automatic file verification via checksums, there is no facility to repair damaged files. Content obtained from the cloud cannot usually be checked for download errors.

APPENDIX 3: USEFUL SETTINGS IN NZBGET

Completed and ongoing downloads are stored in C:\ProgramData\NZBGet\complete and C:\ProgramData\NZBGet\intermediate respectively. You can change these directories with DestDir and InterDir in the PATHS settings.

By default, NZBGet deletes the downloaded RAR files once they have been successfully unpacked. It also downloads the bare minimum number of Par2 files needed for file verification and repair (if needed), deleting them afterwards along with any SFV checksum files present.

  • To keep the RAR files after unpacking, set UnpackCleanupDisk to No in the UNPACK settings.
  • To download all the Par2 files and also keep them afterwards, set ParCheck to Force in the CHECK AND REPAIR settings, and empty ExtCleanupDisk in the UNPACK settings.

APPENDIX 4: FILE VERIFICATION AND RECOVERY WITH USENET

The successful recovery of partially corrupted files is a huge advantage of Usenet. Large files and folders are archived and split into multi-part RAR files before being uploaded to binary newsgroups such as alt.binaries.starwars. Uploaders usually include some “parity archive (version 2) files”, or Par2 files for short, alongside the RAR files. These are designed to help newsreaders with file verification and recovery at the end of the download process.

The newsreader first downloads the RAR files in pieces called blocks, verifying checksums at each stage. Occasionally, some blocks will be missing or damaged on the news server. Provided there are at least as many blocks in the available Par2 files as there are missing/damaged blocks in the RAR files, the newsreader will be able to completely restore the latter by downloading the required number of Par2 files. Uploaders are encouraged to produce enough Par2 files to repair 10% of the project. Hence there is no reason to panic if a Usenet indexer lists a project as being slightly incomplete.

Here is an extremely simplified example to illustrate the basic idea behind Par2 recovery. Suppose the file you wish to download consists of the following four blocks of eight bits:

Block
1 0 1 0 1 1 0 1
0 1 1 1 0 1 0 0
0 0 1 1 0 0 0 0
1 0 1 0 0 1 0 1

Here they are with the row and column totals included:

Block
1 0 1 0 1 1 0 1 = 5
0 1 1 1 0 1 0 0 = 4
0 0 1 1 0 0 0 0 = 2
1 0 1 0 0 1 0 1 = 4
= = = = = = = =
2 1 4 2 1 3 0 2

The row totals can be written in binary, acting as simple checksums for each block. The parities of the column totals are now shown, with 0 for “even” and 1 for “odd”. This block of parities forms a Par2 file which has been provided by the uploader.

Block             Checksum
1 0 1 0 1 1 0 1 = 101
0 1 1 1 0 1 0 0 = 100
0 0 1 1 0 0 0 0 = 010
1 0 1 0 0 1 0 1 = 100
= = = = = = = =
0 1 0 0 1 1 0 0 ← Par2

Imagine the second block in the file has been corrupted on the news server. It might be entirely missing, or perhaps slightly damaged with an erroneous third bit.

Missing 2nd block                       Damaged 2nd block (3rd bit)

Block             Checksum              Block             Checksum
1 0 1 0 1 1 0 1 = 101                   1 0 1 0 1 1 0 1 = 101
x x x x x x x x = xxx                   0 1(0)1 0 1 0 0 = 100
0 0 1 1 0 0 0 0 = 010                   0 0 1 1 0 0 0 0 = 010
1 0 1 0 0 1 0 1 = 100                   1 0 1 0 0 1 0 1 = 100
= = = = = = = =                         = = = = = = = =
0 1 0 0 1 1 0 0 ← Par2                  0 1 0 0 1 1 0 0 ← Par2

As the newsreader downloads the blocks and verifies the checksums, it becomes apparent that there is a problem with the second block: the data is either missing or inconsistent with the checksum. The Par2 file now comes to the rescue, allowing the newsreader to completely restore the corrupted block. The three good blocks are added to the Par2 block bit by bit, and the parities of the column totals reproduce the original second block.

Block
1 0 1 0 1 1 0 1 ← good 1st block
0 0 1 1 0 0 0 0 ← good 3rd block
1 0 1 0 0 1 0 1 ← good 4th block
0 1 0 0 1 1 0 0 ← Par2 recovery block
= = = = = = = =
0 1 1 1 0 1 0 0 ← recovered 2nd block

Further information on the Par2 recovery process can be found here: http://www.quickpar.org.uk/.

++++++++++++++++++++++++
Bluto, Original Trilogy Forums
September 2019
++++++++++++++++++++++++

Author
Time

1st time poster. I’ve just read through The Ultimate Introductory Guide and Bluto’s guide but am definitely a noob to Usenet. I have 2 noob questions. First, are user-names unique in Usenet? For instance, when Bluto and Harmy mention the user Obi-Have Kenobi having the correct files, is it possible to get files from a DIFFERENT Obi-Have Kenobi on Usenet, or would that user ID generally be considered a reliable source? Second, since Bluto’s guide states that a newsreader such as Newshosting automatically does checksum checks, is it not necessary to also run a checksum on your own? OR is it that the newsreader does a checksum to ensure the file FIDELITY with the download source, and the independent checksum we do on our end compared to the checksum listed is to ensure AUTHENTICITY?

Thank you guys in advance!

Author
Time
 (Edited)

I believe it is possible (albeit unlikely in the particular example you mention) for there to be two different users with the same username appearing on Usenet search results, since they may have signed up to different news servers. Although you may be searching on a specific news server, the files it has might not necessarily have been uploaded there originally. Remember that news servers continuously share files with each other. However, where the email address is also shown and appears to be valid, I would imagine that this does identify the user precisely.

You are correct to say that the newsreader is verifying fidelity rather than authenticity. In other words, all you know is that your downloaded files are a perfect match for what is stored on the news server. When files are originally uploaded to a news server, they are not verified for authenticity; after all, most of them are probably breaking copyright. Therefore, unless you trust that uploads from a particular user are authentic, I would recommend verifying the overall checksum at your end as well. HanDuet’s Ultimate Introductory Guide is a useful source of checksums in the case of Harmy’s work.

Bluto

Author
Time

I’m new to all of this, I enjoy my DVDs but I want to see it how I saw it growing up. So I guess I’ll be trying to follow these instruction now. Thank you for being so detailed…wish me luck.

Author
Time
 (Edited)

Hey dear people. Thanks for the great information, I was under the firm impression that usenet as a thing has been shut down a few years back.

I have run into an issue with 4k77. The version by SomethingSomeone through binsearch consistently fails to repair and unpack after downloading: “Repair failed for 4K77/22A0104B180C8421958D2D2B29A89435: not enough par-blocks, 813 block(s) needed, but 0 block(s) available. Recovery files created by: Created by par2SL version 1.0.”

So apparently this version is dead on arrival. There’s another version of 4k77 1.4 uploaded by SomethingSomeone, but that’s 39 gigabytes. It’s also a mkv file… I wonder what’s up with that.

Author
Time
 (Edited)

I remember recommending binsearch for 4K77, since NZBKing doesn’t group all this project’s files together into one collection for some reason. (However, NZBKing is usually far better for tracking down preservations on a.b.starwars.)

I’ll send you a PM about the Par2 issue.

Author
Time

Sigmoid said:

Hey dear people. Thanks for the great information, I was under the firm impression that usenet as a thing has been shut down a few years back.

I am a big user of Emule, which many also assume is dead.
Both Usenet and Emule are off radar, to me a very good thing.
Bluto, could you talk a little about “retention”?
I was a free user of Usenet back when Mysterbin was around, but retention limits finally pushed me towards Email and private tracker torrent groups.

Author
Time

Vultural said:
I am a big user of Emule, which many also assume is dead.
Both Usenet and Emule are off radar, to me a very good thing.
Bluto, could you talk a little about “retention”?
I was a free user of Usenet back when Mysterbin was around, but retention limits finally pushed me towards Email and private tracker torrent groups.

The retention of Usenet news servers these days seems to be around 8 to 10 years. (For those who aren’t familiar with Usenet, “retention” in this context just refers to the length of time a collection of files is stored on the server before being deleted to make room for newer content.)

It may well be more than this; the better servers might look to see which older files are still be downloaded regularly and preserve these longer. Besides, after 10 years or so, many uploads will be obsolete having been replaced by newer and better versions.

Author
Time
 (Edited)

I have played around with the NZBGet + Free-Usenet set-up.
Fairly straightforward. Thank you, Bluto.
I have had a high fail rate, however, and I wonder about if the 3 hour time limit affects this.
(I’m seldom around when the expiration hits, and then the warnings mount.)
Free-Usenet has extremely reasonable prices for Block accounts, and I am tempted.
I cannot find any sort of review for this news server.
Has anyone used something beyond the free account, and can you provide feedback?