Deepfake Ideas - Index and Discussion

I’ve spotted a few instances of discussion about deepfake ideas, and now that it’s becoming increasingly viable, I thought I might set up an index thread to aggregate all of those ideas, and link to their sources and implementations where possible.

I’m not listing these with any bias or preference, I’ll put all ideas here in the OP, leaving discussion of whether we should for the thread itself. It’s OK for these ideas to be a bit extreme here, I reckon, because the tech will only get better and then we’ll blink and find these viable. I’ll also include things which aren’t technically deepfakes but which serve a similar purpose, like Snoke tanks into Palpatine tanks.

Thanks also to oojason for linking me to some core resources for making this initial list easier. Feel free to chuck some more ideas out there and I’ll add them into this list.

• Prequel Trilogy:
  • Hayden Christensen Anakin into young Sebastian Shaw
  • Ewan McGregor into young Alec Guinness
• The Phantom Menace:
  • Fake Windu into real Windu at the end
• Revenge of the Sith:
  • Tarkin CG version into tighter deepfake Tarkin
• Solo:
  • Alden Eherenreich Han into Harrison Ford Han (demo by shamook)
  • Donald Glover Lando into Billy Dee Williams Lando
• Rogue One:
  • Tarkin CG version into tighter deepfake Tarkin (demo by shamook)
  • Leia CG version into tighter deepfake Leia (demo by derpfakes)
• Original Trilogy:
  • Alec Guinness Obi-Wan into Ewan McGregor Obi-Wan
    • (demo by shamook)
    • (version trained on Ewan’s appearance in the Obi-Wan show?)
• A New Hope:
  • Owen and Beru into their Prequel/Kenobi-series versions
  • Jabba into improved/ROTJ-style Jabba
  • New Scene: Vader reports Obi-Wan’s death to Palpatine (demo by Cinefy)
• Empire Strikes Back:
  • Luke from before the Wampa attack into Luke without scar
  • ESB Palpatine into ROTJ Palpatine
• Return of the Jedi:
  • The bearded Endor rebel into aged Temuera Morrison’s captain Rex
  • Sebastian Shaw unmasked Vader into Hayden Christensen (face and voice switch by Crimson Maul)
  • Hayden Christensen force ghost into an more mature/ROTS style appearance
• Mandalorian:
  • Luke CG version into tighter Luke deefake
    • (original version replacement by shamook)
    • (more complex version (over a new actor) by corridor digital)
• Sequel Trilogy:
  • Background characters into characters who appear later in the trilogy, especially Allegiant General Pryde
• The Force Awakens:
  • TFA Snoke into TLJ Snoke
• The Last Jedi:
  • TLJ puppet Yoda into OT puppet Yoda (test by Ebkas)
• Rise of Skywalker:
  • Snoke Tanks into Palpatine Tanks (full renders by Poppasketti)
  • Rey’s dad replaced by young Iain McDiarmind (demo by Fixing Disney Star Wars)
  • Luke and Leia flashback CG versions into tighter deepfakes

Some good ideas for potential audio sources here thanks to thebluefrog.

DylanB18 said:

EddieDean said:

• Return of the Jedi:
  • Sebastian Shaw unmasked Vader into Hayden Christensen

This deepfake is on youtube: https://youtu.be/cbrzXdiuxi4 , i think it looks awsome!
i also like the fact that they changed the voice to match haydens.

Nice one, added into OP.

WitchDR said:

DylanB18 said:

EddieDean said:

• Return of the Jedi:
  • Sebastian Shaw unmasked Vader into Hayden Christensen

This deepfake is on youtube: https://youtu.be/cbrzXdiuxi4 , i think it looks awsome!
i also like the fact that they changed the voice to match haydens.

Stuff like this is truly exciting! I know to some this stuff is blasphemous, but I can’t wait to these truly implemented in a believable way. Hayden as Vader and Ewen as Alec in the OT is just perfect.

I agree. It’s definitely pretty blasphemous, but I can absolutely see the appeal for continuity in a cohesive single world. I’ll keep the OP as opinion-neutral, but I figured it would be useful to aggregate resources and discussion.

I agree with Jar, having a young Ian McDiarmid as Rey’s father would be a small change that would make a big improvement.

Also, just for fun, I’d love to see a) young Alex Guinness and young Sebastian Shaw deepfaked into the PT, and b) James Arnold Taylor and Matt Lanter deepfaked into the PT.

CaptainFaraday said:

I agree with Jar, having a young Ian McDiarmid as Rey’s father would be a small change that would make a big improvement.

Also, just for fun, I’d love to see a) young Alex Guinness and young Sebastian Shaw deepfaked into the PT, and b) James Arnold Taylor and Matt Lanter deepfaked into the PT.

…Well, I did say that the OP would be free of judgment 😛

https://youtu.be/iXqLTJFTUGc

Heres a new breakthrough in Deepfake technology: Lip-sync and vocal synthesis and stitching done in less than a minute.

I presume this can be really useful in a lot of cases, letting the actors say what we want them to say through the use of AI.

Dat_SW_Guy said:

https://youtu.be/iXqLTJFTUGc

Heres a new breakthrough in Deepfake technology: Lip-sync and vocal synthesis and stitching done in less than a minute.

I presume this can be really useful in a lot of cases, letting the actors say what we want them to say through the use of AI.

Even the first video they pulled the bait-and-switch with is kind of unconvincing; my initial reaction was that they must have intentionally chosen a video with already bad lip sync as a base to make their result look better by comparison. But it’s early days into the technology, and this is a big step forwards; give it five years and it’ll probably be flawless.

Now deepfake ROTJ jabba onto ANHSE.

Technically, you can do that with EBSynth.
Just rigorously photoshop ROTJ Jabba into a still frame of ANH Jabba (do this a couple times) after which you use it as a reference frame for EBSynth to slap the face on and you’ll get a passable result (after masking everything else out to maintain quality), however you may need to do those in sections, as there may be moments wherein it wouldn’t track and it would look inconsistent, but morphing and masking can do the trick (most of the time, anyway)

I don’t know anything about EBSynth - could it be possibly used to replace heads/faces similar to a deepfake, at least for limited shots?

EBSynth is pretty much a “style” transfer deepfake, wherein it animates a scene using the style of another artist, or using a scene thats been repainted to match a different style. See the video below for further clarification:
https://www.youtube.com/watch?v=fILG1J6gMBs

Using this concept, we can pretty much transfer the “style” (which in this case is the single frame still image of ROTJ Jabba edited into ANH CGI Jabba) to the entire video and animate it using AI. Sorry if it’s a bit convoluted but after you watch the video, you’ll see what I mean.

Dat_SW_Guy said:

EBSynth is pretty much a “style” transfer deepfake, wherein it animates a scene using the style of another artist, or using a scene thats been repainted to match a different style. See the video below for further clarification:
https://www.youtube.com/watch?v=fILG1J6gMBs

Using this concept, we can pretty much transfer the “style” (which in this case is the single frame still image of ROTJ Jabba edited into ANH CGI Jabba) to the entire video and animate it using AI. Sorry if it’s a bit convoluted but after you watch the video, you’ll see what I mean.

That’s a good explanation, and the video demonstrates it well too. Thank you!

Updated! There’re more ideas here than I was expecting!

thebluefrog said:

1. I think it’d be a good idea to make a list of potential deepfake SOURCES. Other movies, games, audiobooks, interviews, etc. That would be a valuable resource for project makers to come visit the thread.

Very valuable idea. I’m happy to maintain the index, but I don’t have the time to actively hunt that all down myself. But anything that’s dropped into this thread, I’ll arrange and present in a sensible way. If it starts to get complex I’ll gladly convert it all into a spreadsheet or some other sensible reference format.