👕 Detecting Events via #AI 👕

👉Localizing object states & corresponding state-modifying actions

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅SS-learning state-modifying
✅Noise adaptive weighting
✅ChangeIt: 2.6k+ hrs , 34k+ changes
✅Dataset, code, and model!

More: https://bit.ly/3uBwxkj

🌈🌈 Interactive Neural Labelling 🌈🌈

👉Dense labelling of geometry, color & semantics via #3D neural field

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅No training data
✅Dense labeling
✅Classes on the fly
✅Labelling at a scale

More: https://bit.ly/36Y0faQ

♟️Neural RGB-D Reconstruction♟️

👉Novel approach for #3D mixing implicit surface representations with NeRFs

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅RGB-D based reconstruction
✅Leveraging color & depth
✅Depth into the NeRF
✅Pose & camera refinement

More: https://bit.ly/3iN6e54

🦓 Hyper-Fast Refinement 🦓

👉SharpContour: novel contour-based refinement for semantic segmentation

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Instance-aware Point Classifier
✅Deforming by discrete updating
✅Estimating offsets independently
✅Source code soon available!

More: https://bit.ly/3qL04GY

🥗 Neural Mesh via Text only 🥗

👉Zero-shot generation of 3D model using only a target text prompt

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅ZS 3D model with text only
✅ZS text-guided generation
✅Meshes with texture/normal
✅Differentiable LLS implementation

More: https://bit.ly/3u0qnvb

🪆#3D, Materials, and Lighting from 2D🪆

👉Nvidia: topology, materials & map lighting jointly from 2D. INSANE 😮

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Topology, materials and lighting
✅Meshes with materials/lighting
✅Compact volumetric texturing
✅Differentiable all-frequency lighting
✅Code under #NVIDIA License

More: https://bit.ly/3IUoF2t

🍜Ref-NeRF for extreme realism🍜

👉Ref-NeRF: reflected radiance & structures via collection of spatially-varying scene properties

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Realism and accuracy
✅Replacing NeRF’s params
✅Regularization of volume density
✅Integrated Directional Encoding

More: https://bit.ly/3tTlS5l

🦧OFA for all: Cross, Vision, Language🦧

👉Unified multimodal model for image generation, visual grounding, etc.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Sequence-to-sequence learning
✅Image Captioning / Generation
✅Visual Grounding / Classification
✅Text-to-Image Generation
✅Visual Question Answering

More: https://bit.ly/3wSTGlc

🍿Old Films Back to Life with #AI🍿

👉Recurrent transformer network (RTN) to restore heavily degraded old films

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Transformer blocks for spatial
✅Knowledge from adjacent frames
✅Color from keyframes to whole clip
✅Source code available in days!

More: https://bit.ly/3wZbV8y

🍊Neural Head #Avatars from RGB🍊

👉Novel neural representation for animatable head avatar

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Novel articulated human head
✅Full-geometry reconstruction
✅Differentiable optimization pipeline
✅Disentanglement of shape/color

More: https://bit.ly/3DxUGMI

🌶️ MyStyle: personal generative #AI 🌶️

👉Personalized deep generation with a few shots of a person

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Small set of portraits(∼100)
✅Local, low-dim, personal manifold
✅Personal #AI for ill-posed tasks
✅SOTA vs. previous few-shots

More: https://bit.ly/3wWMwMu

🦆 GAN + Dense Map 🦆

👉CoordGAN: structure-texture disentangled GAN with dense correspondence map

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Novel coordinate space
✅Warping to learn coordinate
✅Encoder for structure representation
✅HQ structure/texture editable images

More: https://bit.ly/3DOlOaB

⚓Unified shape & non-rigid motion⚓

👉CaDeX: SOTA in both shape & non-rigid motion

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Canonical Deformation Coordinate Space
✅Shape + non rigid motion representation
✅Factorization of def-homeomorphisms
✅Cycle consistency, topology & volume
✅SOTA in modelling deformable objects

More: https://bit.ly/3NM5NX1

📸 ~6 BILLION CLIP-filtered pairs 📸

👉A dataset 14x bigger than the previously biggest openly accessible image-text dataset in the world.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅2,3B English image-text pairs
✅2,2B from 100+ other languages
✅1,3B language not detected
✅KNN index for quick search

More: https://bit.ly/3LFhKvT

🥮 PP-YOLOE: e-version of YOLO 🥮

👉 SOTA object detector up to 149+ FPS!

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Optimized PP-YOLOv2
✅S/M/L/XL for different scenarios
✅149+ FPS, with TensorRT & FP16
✅Source code & models available

More: https://bit.ly/3x454uy

🧙 HD synthesis with LDM 🧙

👉Low-cost DM via latent space of powerful pretrained autoencoders

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Hi-res synthesis of megapixel
✅Synthesis, inpainting, stochastic SR
✅Large, consistent images of ∼1024px
✅General conditioning via cross-attention
✅Code licensed under MIT License

More: https://bit.ly/3LIVOzS

🎩 SinNeRF: Single Image NeRF 🎩

👉NEural Radiance Field via single view only

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅UATX + UIUC + UOregon + Picsart AI
✅"Looking only once” approach
✅semi-supervised learning process
✅Geometry/semantic pseudo-labels
✅SOTA in novel-view synthesis

More: https://bit.ly/3ujMZqF

🔥 Transformer-based Tracking 🔥

👉Tracker via Transformer-based model prediction module

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Tracking by Transformer prediction
✅Extending model predictor for BBs
✅SOTA on three public benchmark
✅Code/models under GNU License 3.0

More: https://bit.ly/3ucYvUI

👗 In-The-Wild Virtual Try-On 👗

👉StyleGAN-based architecture for appearance flow estimation in VTON application

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Global appearance flow estimation
✅Ok with mis-alignments person/garment
✅"In-the-wild": person with natural poses
✅Code under CC BY-NC-SA 4.0 license

More: https://bit.ly/3LPR9wl

🎇DALL·E 2 just announced!🎇

👉DALL·E 2 to create realistic images and art from natural language

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅More realistic/accurate, 4x res.
✅Better caption matching
✅Not available yet, waiting list!

More: https://bit.ly/3j9v3bR

👋Forecasting interactions via attention👋

👉Predicting the hand motion trajectory and the future contact points on the next active object

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Object-Centric Transformer (OCT)
✅Self-attention Transformer mechanism
✅Framework to handle uncertainty
✅SOTA on Epic-Kitchens and EGTEA

More: https://bit.ly/3v3PpbI