Media is too big
VIEW IN TELEGRAM
Velocity Skinning for Real-time Stylized Skeletal Animation
* project page
* web demo
* pdf
* project page
* web demo
Velocity Skinning is a simple technics to add exagerated deformation triggered by skeletal velocity on top of standard skinning animation.
The technic is:
- Real-time and can be implemented as a single pass vertex shader
- Works "out-of-the-box" on existing skinning data in reusing skinning weights
- Allows non-linear-time editing from instantaneous pose and velocity information
#physics #simulation #skinning #shaderThe MetaHuman Creator — web based app for creating ultra-realistic virtual humans and exporting them to UE4.
Now they opened it for early access.
Sign up here:
https://www.unrealengine.com/metahuman-creator
- UPD:
50 preset MetaHuman characters are already available inside Bridge, ready for both Unreal Engine and Maya.
source
Now they opened it for early access.
Sign up here:
https://www.unrealengine.com/metahuman-creator
- UPD:
50 preset MetaHuman characters are already available inside Bridge, ready for both Unreal Engine and Maya.
source
MetaHuman
MetaHuman | High-Fidelity Digital Humans Made Easy
Create and animate photorealistic digital humans, fully rigged and complete with hair and clothing, in minutes.
Forwarded from PHYGITAL+CREATIVE
Media is too big
VIEW IN TELEGRAM
Еще один ML инструмент для анимации - Cascadeur вчера анонсировал новую версию спустя 2 года!
Это инструментарий для создания анимации 3D-персонажей, а интуитивный дизайн поможет быстро освоиться, даже если вы не имели опыта работы в похожих программах.
Инструменты на основе физики и deep learning позволяют упростить работу. Например, корректировать тайминги и траектории и выставлять максимально естественные позы. Таким образом вы можете избежать ненужной рутины и полностью сосредоточиться на творчестве.
#ml
Это инструментарий для создания анимации 3D-персонажей, а интуитивный дизайн поможет быстро освоиться, даже если вы не имели опыта работы в похожих программах.
Инструменты на основе физики и deep learning позволяют упростить работу. Например, корректировать тайминги и траектории и выставлять максимально естественные позы. Таким образом вы можете избежать ненужной рутины и полностью сосредоточиться на творчестве.
#ml
DECOR-GAN: 3D Shape Detailization by Conditional Refinement
[Simon Fraser University, Adobe Research, IIT Bombay]
* pdf
* github
* video-demo
* seminar
[Simon Fraser University, Adobe Research, IIT Bombay]
* github
* video-demo
* seminar
We introduce a deep generative network for 3D shape detailization. Given a low-resolution coarse voxel shape, our network refines it, via voxel upsampling, into a higher-resolution shape enriched with geometric details. The output shape preserves the overall structure (or content) of the input, while its detail generation is conditioned on an input "style code" corresponding to a detailed exemplar. Our 3D detailization via conditional refinement is realized by a generative adversarial network, coined DECOR-GAN. The network utilizes a 3D CNN generator for upsampling coarse voxels and a 3D PatchGAN discriminator to enforce local patches of the generated model to be similar to those in the training detailed shapes. During testing, a style code is fed into the generator to condition the refinement. #gan #3DThis media is not supported in your browser
VIEW IN TELEGRAM
GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds
[Nvidia, Cornell University]
* project page
* youtube
[Nvidia, Cornell University]
* project page
* youtube
- GANcraft is a powerful tool for converting semantic block worlds to photorealistic worlds without the need for ground truth data.#gan #spade #neural_rendering
- Existing methods perform poorly on the task due to the lack of viewpoint consistency and photorealism.
- GANcraft performs well in this challenging world-to-world setting where the ground truth is unavailable and the distribution mismatch between a Minecraft world and internet photos is significant.
- We introduce a new training scheme which uses pseudo-ground truth. This improves the quality of the results significantly.
- We introduce a hybrid neural rendering pipeline which is able to represent large and complex scenes efficiently.
- We are able to control the appearance of the GANcraft results by using style-conditioning images.
https://digitalhumans.com/sophie/
Here you can try to talk to Sophie.
I think it's very funny, even though her voice is too robotic. But it gives the feeling that if this issue could be solved, it could turn into very nice experience.
Also, they have Einstein.
Here you can try to talk to Sophie.
I think it's very funny, even though her voice is too robotic. But it gives the feeling that if this issue could be solved, it could turn into very nice experience.
Also, they have Einstein.
https://tinytools.directory/
🦊
Really cool collection of tools for visual interactive projects
- author
#tools
🦊
Really cool collection of tools for visual interactive projects
- author
#tools
Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
[MIT, ChipBrain, Amazon]
- pdf
- cleanLab (for label cleaning)
- 10 common benchmark datasets were investigated: ImageNet, CIFAR-10, CIFAR-100, Caltech-256, Quickdraw, MNIST, Amazon Reviews, IMDB, 20 News Groups, AudioSet
[MIT, ChipBrain, Amazon]
- cleanLab (for label cleaning)
- 10 common benchmark datasets were investigated: ImageNet, CIFAR-10, CIFAR-100, Caltech-256, Quickdraw, MNIST, Amazon Reviews, IMDB, 20 News Groups, AudioSet
We identify label errors in the test sets of 10 common benchmark datasets.
Label errors are identified using confident learning algorithms and then human-validated via crowdsourcing.
Errors in test sets are numerous and widespread: we estimate an average of 3.4% errors across the 10 datasets, where for example 2916 label errors comprise 6% of the ImageNet validation set.
Surprisingly, we find that lower capacity models may be practically more useful than higher capacity models in real-world datasets with high proportions of erroneously labeled data. For example, on ImageNet with corrected labels: ResNet-18 outperforms ResNet-50.
On CIFAR-10 with corrected labels: VGG-11 outperforms VGG-19.
- related articleSo, I got early access to MetaHuman creator and just want to share a few of my impressions.
First of all, this thing seems to work like GeForce Now. It makes all the expensive computations on their side and streams a live interactive video to your browser. That's why the maximum session length is 1h.
Currently MH allows you to make a character using morphing to presets and moving face rig parts manually.
It allows you to make renders with different fidelity levels and preview LODs. But I didn't find any export option.
Many people were curious about its ability to represent some particular personalities.
I made some attempts starting from myself ofc 😊. There's definitely something similar at the end, but it feels like something very important is missing. I felt like I was struggling to find necessary shapes for eyes and jaw line and eyebrows. And yes, I'm not a 3D artist and it is quite easy for me to miss smth. So, here is a review of one 3D artist.
First of all, this thing seems to work like GeForce Now. It makes all the expensive computations on their side and streams a live interactive video to your browser. That's why the maximum session length is 1h.
Currently MH allows you to make a character using morphing to presets and moving face rig parts manually.
It allows you to make renders with different fidelity levels and preview LODs. But I didn't find any export option.
Many people were curious about its ability to represent some particular personalities.
I made some attempts starting from myself ofc 😊. There's definitely something similar at the end, but it feels like something very important is missing. I felt like I was struggling to find necessary shapes for eyes and jaw line and eyebrows. And yes, I'm not a 3D artist and it is quite easy for me to miss smth. So, here is a review of one 3D artist.
This media is not supported in your browser
VIEW IN TELEGRAM
So, in its current state MH allows you to quickly make high fidelity NPC with quite significant variety of options, but it is not easy to make someone in it.
UPD:
one another artist's review
UPD:
one another artist's review
Few-shot Image Generation via Cross-domain Correspondence
[Adobe Research, UC Davis, UC Berkeley]
* project page
* pdf
* code
#gan #one_shot_learning
[Adobe Research, UC Davis, UC Berkeley]
* project page
* code
Training generative models, such as GANs, on a target domain containing limited examples (e.g., 10) can easily result in overfitting. In this work, we seek to utilize a large source domain for pretraining and transfer the diversity information from source to target. We propose to preserve the relative similarities and differences between instances in the source via a novel cross-domain distance consistency loss. To further reduce overfitting, we present an anchor-based strategy to encourage different levels of realism over different regions in the latent space. With extensive results in both photorealistic and non-photorealistic domains, we demonstrate qualitatively and quantitatively that our few-shot model automatically discovers correspondences between source and target domains and generates more diverse and realistic images than previous methods.#gan #one_shot_learning
This media is not supported in your browser
VIEW IN TELEGRAM
MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement
[Facebook Reality Labs, Carnegie Mellon University]
* pdf
[Facebook Reality Labs, Carnegie Mellon University]
This paper presents a generic method for generating full facial 3D animation from speech. Existing approaches to audio-driven facial animation exhibit uncanny or static upper face animation, fail to produce accurate and plausible co-articulation or rely on person-specific models that limit their scalability. To improve upon existing models, we propose a generic audio-driven facial animation approach that achieves highly realistic motion synthesis results for the entire face. At the core of our approach is a categorical latent space for facial animation that disentangles audio-correlated and audio-uncorrelated information based on a novel cross-modality loss. Our approach ensures highly accurate lip motion, while also synthesizing plausible animation of the parts of the face that are uncorrelated to the audio signal, such as eye blinks and eye brow motion.
#speech2animationStylePeople: A Generative Model of Fullbody Human Avatars
[Samsung AI Center, SkolTech]
* pdf, abs
[Samsung AI Center, SkolTech]
* pdf, abs
We propose a new type of full-body human avatars, which combines parametric mesh-based body model with a neural texture. We show that with the help of neural textures, such avatars can successfully model clothing and hair, which usually poses a problem for mesh-based approaches. We also show how these avatars can be created from multiple frames of a video using backpropagation. We then propose a generative model for such avatars that can be trained from datasets of images and videos of people. The generative model allows us to sample random avatars as well as to create dressed avatars of people from one or few images.#avatars #neural_rendering #gan #smpl #3d