Video deepfakes
Lip-syncing
- https://omnihuman-1.com/example - used for Jim via Dreamina (Capcut’s
- https://github.com/Tencent-Hunyuan/HunyuanVideo-Avatar
- https://deepbrainai-research.github.io/float/ - very nice results!
- https://www.hedra.com - one of the best yet!
- https://syncanimation.github.io/ - looks great but the model isn’t out yet
- https://mimictalk.github.io/index.html - https://github.com/yerfor/MimicTalk - can be trained on a specific person’s video movements
- MuseTalk
- https://memoavatar.github.io/
- https://github.com/ShmuelRonen/ComfyUI-LatentSyncWrapper
- https://sync.so/
- https://github.com/JOY-MM/JoyGen
- https://jdh-algo.github.io/JoyVASA/
- https://loopyavatar.github.io/
- https://shivangi-aneja.github.io/projects/gaussianspeech/
- https://antgroup.github.io/ai/echomimic_v2/
- https://byteaigc.github.io/X-Portrait2/
OLD
NVIDIA audio2face
- https://docs.omniverse.nvidia.com/audio2face/latest/index.html
- https://docs.omniverse.nvidia.com/audio2face/latest/release-notes/2023_1_highlights.html
- https://www.nvidia.com/en-us/on-demand/playlist/playList-90e4e44c-f4dd-4f1c-8735-092379604b20/
- https://www.nvidia.com/en-us/ai-data-science/audio2face/
- https://www.youtube.com/playlist?list=PL3jK4xNnlCVf18muXHRSnhbKoYtEcZguh
- https://www.youtube.com/watch?v=86PHb2Ybjww
SUPIR Liveportrait
- https://www.google.com/search?q=supir+liveportrait
- https://www.youtube.com/watch?v=3BpjiWKIYmc
- https://www.youtube.com/watch?v=FPtpNrmuwXk
Dreamtalk
- https://www.youtube.com/watch?v=sinf9t3NVYE
- https://github.com/if-ai/ComfyUI-IF_AI_Dreamtalk
- https://github.com/ali-vilab/dreamtalk
Hallo
FacePoke
Other workflows
Lip syncing from audio only
ComfyUI API interfaces
Video2Video / AnimateDiff
- https://stable-diffusion-art.com/video-to-video/
- https://www.youtube.com/watch?v=62eARh_gRhE
- https://www.youtube.com/watch?v=AugFKDGyVuw
Node problems:
- ComfyUI-Portrait-Maker - No module named 'datasets'
- ComfyUI-IF_AI_Dreamtalk - No module named 'dlib'
- ComfyUI-3D-Pack - No module named 'kiui'
- ComfyUI-IP_LAP - No module named 'torchvision.transforms.functional_tensor'
Try to reinstall a custom node:
- CD to node directory
D:\workspace\_apps\ComfyUI_windows_portable\python_embeded\python.exe -s -m pip install -r .\requirements.txt