🍏NeuMan: Human NeRF in the wild🍏

👉#Apple opens a novel human pose/view from just a single in-the-wild video

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅No extra devices/annotations
✅Both Human (novel poses) + Scene
✅E2E SMPL optimization + error-corr.
✅Applications such as "telegathering"

More: https://bit.ly/3K4iTO6

🥑 CLIP-based Neural Style Transfer 🥑

👉From #Nvidia a novel method for transferring the style to a #3D object

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Texture style for 3D by CLIP-ResNet50
✅Nearest-neighbor feature matching loss
✅CLIP-based loss extraction of textures
✅NNFM for multiple style pics / control
✅No source code or models available 😒

More: https://bit.ly/3c32dK5

🔥🔥UPDATE🔥🔥

Code Released: https://github.com/facebookresearch/KeypointNeRF

🔥 KeypointNeRF: code is out! 🔥

👉KeypointNeRF by #Meta: "NeRF"-avatars

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Generalizable NeRF for virtual avatar
✅Sparse 3D keypoints for SOTA avatar
✅Novel unseen subjects from 2/3 views
✅"iPhone" captures for #metaverse

More: https://bit.ly/3pyl17e

🥭Massive GTA-V human dataset🥭

👉GTA-Human: outperforming SOTA with a purely synthetic training.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅600+ gender, age, ethnicity & clothing
✅20,000+ clips, variety of human activities
✅6 categories of location, different BGs
✅Occlusions, lighting, and weather system

More: https://bit.ly/3wpZyRD

🍈DeepBillboards: old-school trick for #VR🍈

👉DeepBillboards models a 3D object implicitly using neural net on the user’s viewing direction

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅#Google Brain +Tsukuba + Tokyo
✅Rendering at higher res., improving #VR
✅NeRF into interactive VR with accuracy++
✅NeRF (or any others) directly in #Unity

More: https://bit.ly/3CsTQ5y

🌐RelPose: Probabilistic Relative Pose🌐

👉A novel method for core component in #SLAM / NeRF-powered apps.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Core component of SfM/SLAM
✅Pre-processing for neural (NeRF)
✅Energy-based over rotations
✅SOTA on both seen/unseen objects

More: https://bit.ly/3T60TXw

🍈 #StableDiffusion archive is out🍈

👉Lexica art is a Stable Diffusion prompt search engine. Real-time, countless #stablediffusion results for everyone. I had fun with the GOAT, #Maradona.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Maradona scoring against a capybara...
✅A poster of space jam with Maradona...
✅Painting of Maradona very detailed...
✅Painting of Maradona in heaven...

More: https://bit.ly/3PTXHLH

🦉PANDORA: Polarized Neural Decomposition🦉

👉CIL lab unveils PANDORA: polarimetric inverse rendering approach via INR

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Geometry, reflectance & illumination
✅normal, signed distance field, mesh
✅Diffuse-specular separation
✅Hi-fI incident illumination

More https://bit.ly/3CzGp3F

🔥IDOL (#CVPR2022 winner): code is out!🔥

👉IDOL for VIS: outperforming all online/offline methods, the new SOTA!

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Online usually inferior by >10AP
✅Online based on contrast-learning
✅Discriminative++ instance embeddings
✅Full exploiting history for stability

More https://bit.ly/3dXCDXw

🔥 #AIwithPapers: we are 4,000+! 🔥

💙💛Lot of people joined, and we talked about #StableDiffusion only twice! Can't believe it.💙💛

😈 Invite your friends -> https://news.1rj.ru/str/AI_DeepLearning

🔵 Deep Saliency: driving the attention 🔵

👉Google unveils a family of operators to "drive" human saliency

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Editing image to drive Saliency
✅Transforms to hide distractors
✅Warping operator for distractor
✅GAN-op for less-saliency altern.

More: https://bit.ly/3KoQQc2

🎍#3D scene manipulation from 2D🎍

👉Reconstruct, decompose, manipulate & render 3D scenes in a single pipeline

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Unique 3D, non-occupied space from 2D
✅Inverse query algorithm for shapes
✅First synthetic dataset for 3D editing

More: https://bit.ly/3RlYhTY

🍊StableFace: Talking Face Generation🍊

👉Analysis on motion jittering in 3D face generation (audio-in -> video-out)

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Motion jittering analysis for stability
✅Gaussian-based adaptive smoothing
✅Augmented erosions of neural renderer
✅Audio-fused generator for dependency

More: https://bit.ly/3Kt95gI

🧡 Avatarization in 90's. So Romantic 🧡

👉Making of the first #MortalKombat in early 90's

More: https://bit.ly/3wTSpJB

🚗 Massive Dataset in Virtual Cities 🚗

👉Synthehicle: 7 hours of labeled material, 340 cams, 64 days, rain, dawn, & night scenes.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Multi-target multi-cam tracking
✅2D, 3D, segm. & depth annotations
✅Instance, semantic & panoptic segm.
✅340 clips, 64 scenes, 17 hrs, 4M BBs

More: https://bit.ly/3TArHiV

🪨Controllable #3D Adversarial Face🪨

👉#Meta (+CMU) on decoupling identity/expression + granular control over expressions

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Supervised auto-enc. + GAN
✅UV texture maps + 3D faces
✅Control expression, saving ID
✅Code under X11 License

More: https://bit.ly/3AVE80q

🥑 DALL·E: Outpainting via #NLP 🥑

👉Extending any original image, creating large-scale images in any aspect ratio

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Extending an image beyond its borders
✅Visual elements in same style of the input
✅Driving the image "story" in new directions
✅Shadows, reflections & textures w/ context

More: https://bit.ly/3eoH8uD

🌪️ TimeLapse++: Video Temporal Pyramid🌪️

👉Multi-scale lens to view the passage of time: far beyond a "classic" timelapse

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Inspired by "old-school" spatial pyramids
✅Video Spectrogram to go through pyramid
✅Months/years of data in a few seconds!
✅Multi-temporal freq., no aliasing

More: https://bit.ly/3TKnYPS

🫐 Stable Diffusion Video is out! 🫐

👉A free notebook to generate videos by interpolating the latent space of SD.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Blueberry to strawberry spaghetti
✅Dream items from same prompt
✅Morph different prompts (seeds)
✅Built on a noscript by A. Karpathy

More: https://bit.ly/3ey8632