Shamook

CaptainFaraday said:

Shamook said:

There are primitive ways to alter mouth movements but the quality is subpar for me to consider using it at the moment.
You could film yourself acting the role, rotoscope yourself onto the original actor, then apply a deep fake on top of all that.

I’m assuming that deepfakes use some kind of automated facial tracking to keep the deepfake in the correct place, angle, and lighting - if that’s right, would that same technology be able to be used in the rotoscoping stage, to seamlessly line up a new facial performance?

My reason for being so curious about this is pretty obvious for fanediting - imagine the scope that would open up if a faneditor could completely replace an entire character’s face and dialogue with their own new actor.

Ye that’s right, it has face and landmark (eyes, nose etc) tracking. I suppose you could use that to help with the rotoscoping.
I’ll be completely honest, I know very little about compositing etc.
I think what you’re looking to achieve IS possible, but how tricky and what the end result might look like isn’t something I could give a good insight into.

CaptainFaraday said:

Shamook said:

Hey all, Shamook here.
I was just googling myself and came across this thread. It’s really nice and refreshing to see so many people enjoying my videos. Sorry if I never responded to your emails or messages, I just don’t have the time to get back to everyone unfortunately. I think I got 300+ emails about the Mando deepfake.
Let me know if there are any specific things I might be able to provide you guys with; extra scenes, different video formats etc
Also I can verify I’m definitely Shamook if that’s necessary.
Cheers.

Shamook, I have a question about how your deepfakes work. In your videos, you seamlessly replace one actor’s face with another’s, keeping the original facial performance intact. Would it be possible to replace the facial performance?

For example, let’s say I want to completely replace an actor’s performance and dialogue with new footage I’ve filmed of my own actor. Is that possible with current technology? Either retaining the original actor’s face, or simply replacing it entirely with my new actor’s facial footage?

EDIT: I mocked up a video to try and explain my question better.

Let’s pretend I want to cast myself as Kylo Ren instead of Adam Driver, and change all of Kylo’s lines. Is it possible to take the footage I’ve filmed, and deepfake it onto Adam, resulting in:
a) my face and new performance completely replace Adam’s, or
b) it looks like Adam’s face is giving my new performance.

Like this, except with your amazing facial replacement skills, instead of my drag-and-drop example.

(I don’t want to actually do that, I just needed an example with some footage I already had on the computer.)

There are primitive ways to alter mouth movements but the quality is subpar for me to consider using it at the moment.
You could film yourself acting the role, rotoscope yourself onto the original actor, then apply a deep fake on top of all that.

Burbin said:

That’s just the way the deepfakes work, if the face is obscured or at an odd angle then it won’t be detected. The system won’t have enough data to construct a deepfake to put on top of it. In the clips I put together I used fade-ins to hide the shift into the deepfake.

100% correct.
Its a shame, but it’s just not worth the headache trying to get those types of shots to work.

theprion said:

Shamook said:
Or even if you guys have any suggestions for any Star Wars deepfakes you might want to see.

How about deepfakin Ewan McGregor’s real beard into his dreadful fake beard on those AotC reshoots?

Would be similar to what you did to Cavill’s removed mustache, I gather.

Removing a moustache is much easier than adding a beard.
When you introduce large amounts of hair to a deepfake, the amount of detail the software needs to learn jumps enormously, that’s why you don’t see full head deep fakes just yet, mainly because of the hair.

Burbin said:

I’d be quite glad to have my work be rendered useless if Shamook could provide source quality files of the Leia/Tarkin deepfakes, as well as the missing Tarkin shots where I used other sources that are of notably lower quality.

https://www.dropbox.com/sh/n0m06uo7or6lfsb/AAC4xdD26_gbj_LTFJ5MjqVLa?dl=0
There are 24 clips of Tarkin and 1 of Leia in that dropbox folder.
They should be original film resolution and quality.

Can you let me know which Tarkin shots that you used other sources on, my file management is terrible and I may have them somewhere.

danieldubb said:

Shamook said:

Hey all, Shamook here.
I was just googling myself and came across this thread. It’s really nice and refreshing to see so many people enjoying my videos. Sorry if I never responded to your emails or messages, I just don’t have the time to get back to everyone unfortunately. I think I got 300+ emails about the Mando deepfake.
Let me know if there are any specific things I might be able to provide you guys with; extra scenes, different video formats etc
Also I can verify I’m definitely Shamook if that’s necessary.
Cheers.

Welcome and thanks for reaching out. You have some fans on this site of ur work. Would be great to have u as part of the community.

Hey Daniel, your name looks familiar, I think I’ve since you comment on my videos before.
Always nice to interact with fans 😃

Like I say, if there’s anything I can provide you guys with like high res files rather than the compressed YouTube versions, any extra scenes I might have laying around, or I saw someone mention something about bad fps. Please let me know, I’m more than happy to help where I can.
Or even if you guys have any suggestions for any Star Wars deepfakes you might want to see.

Hey all, Shamook here.
I was just googling myself and came across this thread. It’s really nice and refreshing to see so many people enjoying my videos. Sorry if I never responded to your emails or messages, I just don’t have the time to get back to everyone unfortunately. I think I got 300+ emails about the Mando deepfake.
Let me know if there are any specific things I might be able to provide you guys with; extra scenes, different video formats etc
Also I can verify I’m definitely Shamook if that’s necessary.
Cheers.