CaveDoctor

I don’t have a volumetric display of my own,
But I’ve seen some wild things in regards to the looking glass display.
Recently a software called REFRACT for Looking Glass has become available allowing games to be played,
https://www.youtube.com/watch?v=Q-iyzyBFLNA, if you want to see what it looks like, here’s a link.
Everything seems to be in an Embryonic stage.

I’m curious if there is a way to do this with video. I also wanted to know if there was an open source alternative to rotoshapes available, similar to what was done in Jurassic Park 3d.
https://www.youtube.com/watch?v=5Sp8tSNnKy4, here is a link, you’ll see the rotoshapes influence the depth map in this montage, there is a lot more footage of how this is done.
Specifically from 2011, and using Autodesk software.

My guess is that using morphable models from neural nets would enhance the depth maps in a similar way to rotoshapes. I’m hoping that this could be used to also segment video, and allow for something like video epitomes to assist with inpainting and enhancement of depth maps.

I have a few ideas, but no experience in programming. If I had a GPU I would be working on this.

There is a mix of software I wanted to use for this, most of them licensed under the GPL, I wanted to use a mix of depth maps, morphable models. I figured using morphable models, like Volker Blanz implementations and PifuHD to enhance depth maps with wireframes, to guide it. I’ve seen that in some cases, geometry that is very accurate is encoded in a depth map, the prime example being a depth map of the Blender monkey.Several software projects can extract depth maps, and reconstruct facial geometry (similar to Total moving facial reconstruction).

My end game is to create software for converting traditional 2d content into holomovies, and I know that the computing power would be intensive. I was hoping just to use clips from Star Wars, LOST or Person of Interest as test clips. My goal is to produce 5 minutes of footage.

Several code bases could be combined, (If licenses allow it) for a research project, and an open source tool.

3D combine is also an app I’m interested in for generating animated depth maps.

I played a depth map with a monocular color image, and found that sometimes an accurate form of geometry could be produced. Sometimes things looked like Mountain ranges, or Dioramas too.
(Similar to a histogram).

https://www.youtube.com/watch?v=qsMIH7gYRCc
Neural Scene Flow Fields
https://www.youtube.com/watch?v=WjvM3FcpId4&t=257s
PiFu HD

https://www.youtube.com/watch?v=lDeI17pHlqo
MiDas and LeRes

https://www.youtube.com/watch?v=PhdT8MzIg_0
There are many similar projects to this one, that could be implemented

Television shows, and movies also include a lot of shots that seem to be natural for 3d Conversion, from camera shots that change focus with 15 frames, to shots that seem potentially ripe for photogrammetry, although not perfect for it.
An example is this scene from LOST.
https://www.youtube.com/watch?v=10VB8MPeN-k

http://3dstereophoto.blogspot.com/

Black Alert said:

@CaveDoctor:

Ok, so the whole episode, no problem. It could take a bit, as I have said, I am doing two other updates as well.

Hey man whenever is good for you.
I’ve got a lot to figure out with my project, anyway.
The only thing I need is one episode for a test run.
something that would be on both the PAL dvd and the HBO MAX versions.
That rules out “The Gathering” since it never got remastered.
Although I hope, but doubt in the future, the 16:9 + TV films get the remasters they desperately need.
The two projects I want to do with B5,
An HD overlap of upscaled DVD footage.
Then conclude with “B5:Redux”, where I cut the series into a straightforward 54 episode series.

Black Alert said:

@Cave Doctor:
This is amazing. I was aware that some VFX work seemingly survived, but I had no idea to what extend. So basically you would just need the upscaled live action footage from me, correct? Or would you require the whole episode as I have done it (Live Action 16:9 from DVD and 1080p VFX/CGI stretched towards 16:9 from the HBO Max / Amazon release)?

As I am re-doing things (quality updates) it would be good to know. Perhaps we could work on episode 1 (MotFL) and see how things go?

The live action footage is all that’s necessary, I don’t have any VFX files myself.
I just ran into the upscaled footage on youtube this year, so I’m aware of it’s existence.
I’d like the whole episode though, I think your upscaling work is amazing, you nudge those masters into place so damn well.

WXM said:

b5historyman said:

Since a friend of mine has a lot of actual original CGI assets he is currently working on a project reworking them to 1080p

I heard (on sci-fi-meshes I think) the originals were all lost by the studio that demanded to be the sole custodian of them. Were they found, or are there other things to the story…? Are his the assets of the old LightWave meshes/scenes or…?

Because those same-ish original LightWave scenes, it would almost be hilarious watching a modern system re-render out the same even at mere 1920x1080; I’d imagine shots that took a room of Amigas a week to render in the early '90s would be poured out in a mere minute or two on a (buffed-out) modern workstation. (Yeah, my sig gives away why I’m in this territory here 😃)

It’s a misconception that they were lost.
If you look on youtube, there are several rerenders of visual FX from the series that somehow turned up despite being lost.
Some things had to be fixed, but the scenes weren’t lost.
https://www.youtube.com/watch?v=MKj-N5j0SBY&t=1s
The live action footage is upscaled to 4k for this one, but the VFX are all rerendered from original files, with replaced texture maps.

b5historyman said:

Is this all upscaling? Since a friend of mine has a lot of actual original CGI assets he is currently working on a project reworking them to 1080p

The project that Black Alert has posted, so far has been an upscale of PAL DVDs, he’s using the HD upscaled footage from the recent remaster to replace VFX shots in his upscale.
It’s badass that your friend has VFX assets though.
How many episodes worth of assets exist still?

Black Alert said:

That’s kind, but I guess I’ll be doing my quality updates to what I have done so far first and then I’ll see, ok?
It’s a bit of a state of flux at the moment, but I am back at work (right now the DS9 pilot will be upgraded, release to follow soon).

Hey, I just love the hell out of your work.
I’ll support you any way I can, it’s like you nudged the masters where they needed to be, and if the perfect remasters never happen, your contribution is the best.

I’ll share my agenda with you, what I want to do is overlay high defintion live action footage over the PAL dvds 16:9 footage, in the case of Babylon 5, and attempt to seamlessly blend them.
I planned my steps out, a while ago, and wanted to make a test episode, in my dream project with this, your work is the first layer.
I’m going to denoise the 4:3 masters, and use Dre’s color tool to match the footage.
I want to blend them together with a layer of 35mm film grain, but I also have to stack the footage, and that will be the tricky part.

I’ll add, I was impressed with your DS9 posts on reddit.
I’d love to help out.

Care if I join the project?

I’m curious if anyone has tried to combine edit ideas from two seperate fan edits, into a more comprehensive edit.

At the very least it’s half workable.
Machinima out the scenes with the models from the games.
Edit the true footage around those.

Polish and trim the true footage.
One goal in particular, save as much of the original cast footage as possible.
Use deepfakes to enhance the facial animations.
figure out what advanced rendering techniques are at our disposal.
Reintegration of a handful of deleted scenes.

Stage one,
a borderless film.
We have to look at the trilogy as one whole three act structure, and see what to remove.
Integrate a new trilogy from the old.
For now I can either call this redux, or STdraft2.

For the digital aspect of this, that will be categorized as Digital Dream.

https://www.youtube.com/watch?v=FRbbvsxXaBs

This is interesting, it follows one of my other ideas.
At the very least using the models from EA games to at least machinima out the scenes I’d like.
I’d still need better voice actors though.

If the horsepower becomes available on my limited budget, I’d like to render in Blender, enhance some rigging of faces, and then use deepfakes to replace the faces.
perhaps recreate some of the matte paintings in 2.5d.
I’d still need to learn to compile the voice changing software though.

Another thing that pisses me off is the Yoda puppet from TLJ.
Rian Johnson had the original crew who built the first Yoda puppet at his disposal.
Instead we got that lime green abomination.

I’m sure a lot of people would find my ideal trilogy bad.
But, what I have to work with is garbage.
No idea if it will ever happen.
I’m pretty content with

Kotor 1 and 2
Some prequel fan edits
Tartakovsky’s clone Wars
Clone Wars
Rebels
OT
The Mandalorian
Some of Marvel’s new graphic Novels.
Some of Legends.
And the Mandalorian

As the Canon I’m stuck with.
Not a fan of what Disney did, they really fucked up.
I’m not sure if the sequels can be saved.
It is a lesson in not letting your ambitions overwrite what the story needs.
Rian Johnson is no Nicholas Meyer, I’m afraid.
He didn’t bless us with Star Wars’ answer to ‘Wrath of Khan’.
He gave us Space Vacation.

To me, the sequel trilogy in it’s current state, is not Star Wars, and it doesn’t seem that it ever could be.
I don’t know what it is.
It’s unrecognizable.
Like fan series based on Star Trek.
It’s bad fan fiction.
What really makes me sad, is Rian Johnson’s right hand man called Luke “Obi” in TLJ.
How?
He’s obviously Yoda, not “Obi”.
Obi didn’t exile himself, and lose his purpose in despair.
Obi had a purpose, and so did Yoda for that matter.
All Johnson did was bastardize the second film.

DuracellEnergizer said:

CaveDoctor said:

I hate to mention the feminist problem, but, alot of these so called male feminists have no idea how to write or develop a strong female character, and then they rail for diversity, the tragedy is, to them diversity is only skin deep.
The fact that they’re dumb, and think they’re geniuses, then get mad when you point out that they aren’t speaks volumes about the problems plaguing this franchise.

I consider myself a male feminist, but I tend to agree. People like Wes Craven, J. Michael Straczynski, and the writing staff of Star Trek: Deep Space Nine knew how to do strong female characters right; good luck finding anyone of their calibre in present Hollyweird.

There is a difference between 2nd wave and (3rd and 4th wave) feminists.
Ridley Scott gave us a badass in the first Alien, with a very minimal script.
DS9 had Nerys.
Fringe had Olivia Dunham.
I’m not opposed to strong female characters or feminist themes.
But there is a major discrepancy between the classic variety and the modern variety.

So, where would Finn be?
He’d be the man from nothing.

Poe Dameron would be interesting as well, but I wouldn’t want him to be a total dunce, which is what he was.
He’s a dunce.

Han’s relationship with Leia was strained due to ambitions and differences.
The relationship fell apart because of Leia’s dedication to her role as general.
Han couldn’t hold the family together on his own, and Han never liked war.
Chewy would be more than a cheaffeur.
Kashyyk would be where I’d set the first movie.
We’d see the industrialization and colonization of Kashyyk as something key to the Republic’s agenda.
Perhaps Chewbacca is a general at this point.
Liberating Kashyyk from a fractured Empire would be a plot point.

Luke has tried negotiation but failed.
Thrawn allows the budding Sith a chance in the field.
Her and Luke have a duel, a game of sorts.
Perhaps we don’t see Han Solo until part two.
That would be interesting.
I really don’t know what to do with Han.

He would have matured as well, perhaps he’s another high ranking general.
But him falling back into smuggling would be interesting.
Perhaps him and Chewey have to liberate Kashyyk.

The question then becomes how to kill him off in an emotional way in the first or second films.

Luke has to die in part 9, Leia has to be alive at the end of part 9.
Thrawn would have to be as menacing as the Emperor, but without the Emperor’s power.
Even if Thrawn could be manipulated by a force user, it can’t be the emperor, and it has to be by an actor that scares the audience.
Suspense needs to be a major part of these movies.
the villains need gravitas.

My ideal trilogy.

The new republic and the empire are in a kind of cold war.
All stockpiling death star type super weapons.
I’d love to take this trilogy into a hybrid of the prior two trilogies.
Alot of it being small scale, but we’d get a glimpse of larger war scenarios like in 3.
I wouldn’t ignore all of Lucas’ innovations in the prequels.

I’d love to see at least one timeless issue be brought up.
In the republic, a man with no importance can rise to the occasion, not because he has important blood.
But, I’d love to explore Luke’s new direction for his Jedi order.
It would be more grey, Luke would be as much a diplomat as well as knight.
Luke would be fair, the thing I’d love to see in Luke is maturity, not as a jaded hermit.
But more Theodore Roosevelt.
It would be interesting to explore the Empire as a kind of Soviet style state, where it falls and becomes a second republic.
We’re not in the WW2 stage anymore, but US proxy war type situation.
The Republic is directly involved, but the Empire always fights with proxys.

What would be interesting would having Thrawn be the major villain.
I’d certainly have kept Luke falling to the dark side once in his history canon, that’s why he’s grey.
The Emperor clone did happen, but in the past.
It wouldn’t be the present thing.

Luke would probably explore the relationship with the Whills, and how he’s like a car to them.
Perhaps an interesting take would be keeping Rey as Palpatine’s grand daughter, but recycling a character from clone wars, she’s obsessed with becoming Palpatine.
What would be interesting would be her seductive capability.
She would easily be able to seduce Ben to the dark side.
But, she’d need to be a bad bitch.

Leia’s evolution would have to allow her to be more hardcore, but with a soft spot, not just for her family, but with the values of the republic.
She knows the toll this takes on her soldiers.
What would be interesting would be to actually give her the mindset of a true general.
She’s like Douglas MaCarther.
She doesn’t micromanage, but she knows how to run a battlefield, perhaps one of the finest generals in the galaxy and it would be no surprise that her and Thrawn would be engaged in a kind of tactical rivalry.
It’s a game of wits between her and Thrawn.

I’d like to see the stormtroopers be portrayed in a more effective way to.
They wouldn’t be clumsy or bad shots, they’d be competent at taking over ships.
I’d love to see the kind of things we saw in Rogue One for the space battles.

I’d also love to see an updated X wing, something that pays tribute to the original, but also looks a little sleeker, not a leftover X wing, but a Republic model X wing.
The MonCalamari have real influence in republic design.
Perhaps their designs would reflect a lot of their traditions, as well as celebrate the designs of the ships that won the Republic it’s independence.
They might have designed a fighter ship that looks similar to the Millenium Falcon, a Corellian freighter that has their designs on it.
They’ve advanced a lot, their ships are tactically impressive, not cobbled like in OT.
That’s something Thrawn takes notice of.

It would be interesting to explore Thrawn’s relationship with the budding sith, perhaps he would want an order of force users of his own, but he’s critical of force users.
critical of Vader, and of Palpatine.
He respects what came before him, but he’s not a dumbass.
Not quick to anger.
In my ideal trilogy, I’d have John Hamm play Thrawn.
I’d want their to be a lot of Naval themes within this.
I have a lot I’d like to explore.

Sidious’ presence in the expanded universe isn’t unprecedented post ROTJ, so it’s not the dumbest idea from the sequels.
I’d love to see some actual character development for Mary Sue, and some serious development for Finn, I expected Finn to be a vietnam vet type character, with some serious conflict going on in his person.
The other issue comes in with Leia.
I hate to mention the feminist problem, but, alot of these so called male feminists have no idea how to write or develop a strong female character, and then they rail for diversity, the tragedy is, to them diversity is only skin deep.
The fact that they’re dumb, and think they’re geniuses, then get mad when you point out that they aren’t speaks volumes about the problems plaguing this franchise.
After I fell asleep 3 times trying to watch TLJ, I felt jaded, and I still am.
I don’t care about the future.
Sure the Mandalorian is okay, I just wish that Disney hadn’t blown it with this trilogy.
TFA was too goofy.
TLJ was too stupid and political, wasting Carrie Fisher.
Rise of Skywalker, I can’t even abreviate, seemingly is two movies rolled into one, Abrams too cowardly to retcon TLJ.
I just wish I could save these films.

I know we’ll never see Fisher portray Leia again, but it’d sure be swell to see an animated show set in between 6 and 7, focussing on Luke, perhaps bringing in some of the Expanded Universe canon like the Thrawn trilogy.
I’d be happy with that at this point.

My main goal is to try and honor the performances that Mark, Carrie, Peter, Billy and Harrison gave.
We can’t get the surviving cast together again, and Carrie and Peter Mayhew are gone.
I just want there to be a way to bring honor to those performances.
Nothing makes me sadder than watching Carrie Fisher and Mark Hamill in TLJ.

I had one idea called epilogue, where I just took footage of the original cast from these films, in order to tell what happened to Luke, Han and Leia.

The other idea was to repurpose Rey, as a more Revan type character.
She’s a sith with her mind wiped at the beginning.
The other proposal is to wait and create footage when deepfakes, Voice emulation (a vocal deepfake) and realtime raytracing advance more, don’t hold your breath.
With that I feel it’s a matter of properly layering it, a 3d tracked model so as to get the proper shape of the head, a lot of the work will be automated, but it will still be difficult to do, at that point, I don’t know if these films are worth it.

I’ll have to check this out.

First of all, I understand that what I can do is severely limited, but.
So, I’m considering a fan edit of the sequel trilogy.
I feel jaded watching them.
Would it be worth my time to do this?

All I have access to is a handful of deleted scenes, and the ability to trim out dialogue.
So, from what I gather, once the final film is released on bluray, how could I trim it out.
I figure that the best I can do for 7 is to reincorporate some deleted scenes, and trim some other shit.

With 8 and 9, what can I do?
Most of the work I feel comes down to how I can rework 8 and 9.
I watched Plinkett’s review of TLJ and a seperate review of 8 and it described perfectly what I felt about it.
Still haven’t seen 9, as I’m really afraid of how much of a disgrace it’s going to be.
So, I’m curious if I can trim 8 down to the 1:45 minute mark, and trim out most of the crap.
Then merge the remaining footage from 8, after Snoke’s death into episode 9?
While trimming the shit out of episode 9.

I’m not a fan of the sequel trilogy, but I’d at least like to turn them into tolerable films.