Hello all again! I apologize for not being active as much on the forums, and as of the state of my ANH edit, it’s still in the air as I am still pre-occupied with academic responsibilities, but I will try to get it done this year if I can clear out storage for this laptop.
In the meantime, I have been deeply delving into the uses of AI video generation (such as Kling, Hailuo Minimax, Veo 3) and it’s use for extending or animating deleted or alternate cuts from the OT, such as specifically utilizing AI-image-to-video generations, which as the name suggests, takes an input of an image, then extrapolates from that singular frame to create a cohesive video generation output. I waited until the technology would somewhat mature enough to be mostly consistent, as AI videos do have a tendency to be a bit “janky” for lack of a better term.
I chose a production(?) photo of Luke and Vader walking across the Death Star hangar entering into the throne room from ROTJ, which I presume was cut due to time constraints during the “Black Friday” overhaul of the film in late 1982, as I always wondered how this scene might’ve looked like in motion.
Using a combination of Adobe’s Firefly to extend the edges of the frame to fit a 2:35:1 aspect ratio, and grading it and adding some rough sound FX and score, I’d wager that the output fits right in to the subsequent shot and could, in theory (as this is a proof-of-concept), extend the film to have some breathing room in between shots. I’d love to know some thoughts and inputs to this.
Also, as an addendum, I believe a beta version of Premiere Pro has a feature that does use AI to extend clips so you could theoretically “stretch” them out for as much as 5 seconds. Again, this could be a pretty cool solution to pad out moments in films, such as TROS.
I don’t know if that’s quite cinema quality, but it’s far better than I was expecting it to be!
I don’t know if that’s quite cinema quality, but it’s far better than I was expecting it to be!
Agreed. I think you’ve got a really good idea here, but we might need to wait another year or so in order for the technology to be seamless.
Yeah unfortunately this was only rendered in 768p using Hailuo 2, I could render it in 1080p for 10 seconds, but that is for premium users only. However, I’d like to note that it is the best video AI generator aside from the recent news surrounding Google’s Veo 3 video generator that you may have seen floating across the internet.
Also, I don’t know what soundtrack could help in extending that scene, and if I added in those metallic footsteps, it could help to “ground” the scene with the rest of the shots.
Lastly, I would like to ask if there are any alternate versions of scenes in the OT, PT and ST we could use to further push this technology? Im thinking of the extended version of Luke falling into the Millennium Falcon during ESB where Lando had to basically grab him and slide him into the docking pod of the Falcon.
The first thing that comes to mind is in TROS - we need to see Kylo being called further into the forest on Mustafar, away from the battlefield, and encountering the obelisk that holds the Wayfinder.
Luke rebuilding his Saber for ROTJ is a big one.
This is pretty incredible. I can think of so many great uses for something like this.
Star Wars, Paleontology, Superhero, Godzilla fan. Darth Vader stan. 22. ADHD. College Student majoring in English Education.
My Star Wars Fan-Edits
It’s definitely got that unnatural AI quality to their movement so it isn’t quite working for me. I immediately felt like I was watching that Scene 38 YouTube thing. But as an experiment, it’s intriguing. It’s almost there and as the software improves, I wouldn’t be surprised if this kind of “extendAIsion” is a standard part of the process for some fan editors.
I think we’re all FAR too familiar with Return of the Jedi here to be able to accept new footage like this. It’d be fun to edit a minute of the film with a couple of AI shots like this inserted and show it to a normy. Would they be able to identify it?
A scene I’ve always wanted, not necessarily AI, I imagined needing to film it like SC38, is at the barge fight, when Boba flies over to Han, and Han is like “Boba Fett?” to Chewie… he doesn’t just hit him, but they turn around and have an ACTUAL fight. And Boba nearly wins, gets Han pinned, pulls out the jango fett forearm blade, is pinning Han down trying to bring the blade to his neck…
When either, Han gets lucky and reaches his stick, or Chewie or Luke somehow get a shot on Boba fett’s jetpack, and THEN he flies off, and we cut back to the original shot of him flying off.
I just want 20 to 30 seconds of them having a fight and have Boba beat up on Han a little, increase the threat.
Mark’s Down On Your Syntax said:
It’d be fun to edit a minute of the film with a couple of AI shots like this inserted and show it to a normy. Would they be able to identify it?
I think, even this little experiment, assuming he fixed the things like Luke walking too fast… a normal viewer would NOT be able to tell. Not that this shot in particular would even be needed. But assuming the resolution increases, and they are quick shots… this looks pretty good honestly.
This is a very exciting time for fan editing and game modding, in terms of, new content is actually becoming more viable outside of spending thousands actually filming it.
Now I DO think, if something can be truly filmed, that will still probably be best. Like my Boba v Han imagined scene, I truly want to film that like SC38 did (though hopefully more in the style of the OG film, no offense to their project)
A big problem for the game Oblivion, its predecessor Morrowind had text dialogue, so new content could be implemented seamlessly. But Oblivion has voiced dialogue. So most mods either would not have dialogue, or if they had it, it would often be really crappy quality, either audio or acting wise.
But now, this past year, many great mods from Oblivion’s past that had no voice acting, are now receiving AI acting patches.
That’s pretty incredible! I’m watching this on my laptop and the only real issue I noticed was that Lukes face didn’t quite work. But you could take that footage and face swap a better picture of Luke on it and it might look just fine!
Attack of the Clones: Alternate Timeline Edit Thread:
https://originaltrilogy.com/topic/SSWRs-Attack-of-the-Clones-Alternate-Timeline-Edit/id/66888
Looks way better than I though it would!
My hope is that someday we may get to see the Novelization of ROTS on the big screen. Adding in everything from the book would probably make it my favorite Star Wars movie.
This channel called “Skywalker Stories” on YouTube is pretty cool.
Here’s the latest video:
https://youtu.be/WOS7i42ggU8?si=awXxKzvhkaBCSBoL
Their videos are done using AI. Obviously it’s not 100% flawless but in many cases it has me fooled. I don’t know how they’re going about making these exactly. My assumption is some custom ComfyUI workflow.
Attack of the Clones: Alternate Timeline Edit Thread:
https://originaltrilogy.com/topic/SSWRs-Attack-of-the-Clones-Alternate-Timeline-Edit/id/66888
I’ve seen the work done using the Starlight Mini upscaler on some DS9 laserdiscs that produces a final level of quality comparable to the TNG blue rays after manual processing with it. I’ve not seen it tested on anything as low quality as the OT deleted scenes but I am highly curious as to what it could do for especially the RotJ scenes as they’re of higher quality than the other two. It’s something that would need a skilled editor regardless as none of these models are plug and play yet.
I think the restoration models are much closer to useful than the purely generative ones that produce motion from stills however the predictive scene fillers are really useful, especially for background replacement or opening the matte for full screen picture.
Agreed, Artan42.
I very much want to see the deleted prequel scenes using Starlight. I already tried it on some old home videos and it worked pretty damn well in most cases…I just ran out of free credits 😦 😛
Attack of the Clones: Alternate Timeline Edit Thread:
https://originaltrilogy.com/topic/SSWRs-Attack-of-the-Clones-Alternate-Timeline-Edit/id/66888
Well done - it’s actually a nice little bridging moment that allows the movie to breathe and for the viewer to ease into the new location. There’s also a rather nice syncopation between the footsteps across the Death Star hangar and the red lights on the elevator lighting up - it segues very well into the existing footage.
If this had been included in the original cut I’m sure we’d all have considered it an essential shot to keep. If AI can make it look flawless then I don’t see any downside to adding this moment into the movie. Good stuff.
SomethingStarWarsRelated said:
This channel called “Skywalker Stories” on YouTube is pretty cool.
Here’s the latest video:https://youtu.be/WOS7i42ggU8?si=awXxKzvhkaBCSBoL
Their videos are done using AI. Obviously it’s not 100% flawless but in many cases it has me fooled. I don’t know how they’re going about making these exactly. My assumption is some custom ComfyUI workflow.
I used their Palpatine dissolves the Senate scene as an experiment to bridge Rogue 1 and A New Hope, it works fairly well however the PDTS scene has some weird rocking back and forth, there’s longish shots of Palpy talking and the AI (I am thinking it is the AI) is recycling the footage so it rocks it back and forth. One could argue it’s from the POV of a droid that is broadcasting or filming this as they hover around the Senate chambers. The end result was okayish however it creates a chronology problem in that in ANH about two days later in the Death Star conference room they say that they’ve just received word that the Senate was dissolved. Maybe politics over the scale of galaxies made it take some time to iron out his authoritarianism? Too many bugs for me to call it a solution to bridge the movies.
Something I did like about this transition is that the combo of the Death Star going live & seeing Palpy dissolve the Senate really gives a sense of urgency to the events of ANH.
A scene I’ve always wanted, not necessarily AI, I imagined needing to film it like SC38, is at the barge fight, when Boba flies over to Han, and Han is like “Boba Fett?” to Chewie… he doesn’t just hit him, but they turn around and have an ACTUAL fight. And Boba nearly wins, gets Han pinned, pulls out the jango fett forearm blade, is pinning Han down trying to bring the blade to his neck…
When either, Han gets lucky and reaches his stick, or Chewie or Luke somehow get a shot on Boba fett’s jetpack, and THEN he flies off, and we cut back to the original shot of him flying off.
I just want 20 to 30 seconds of them having a fight and have Boba beat up on Han a little, increase the threat.
Would rather see Boba Fett lose his hand/arm to Skywalker and have his rocket pack damaged by a nasty lightsaber slash, than see him beat on an almost completely blind man (like the JEDI storyboards showed).