И ещё об искусстве;)
В Стэнфорде довольно успешно сумели научить алгоритм предсказывать, что люди ощущают — а не просто какие объекты видят — созерцая произведения искусства. Другая формулировка — сенсационная, но обманнная: научили машину понимать эмоции. Понимать она не умеет, но комментарии к картинам выдаёт очень человеческие и вполне эмоциональные ;)
Чтобы научить алгоритм, пришлось привлечь тысячи людей, которые разметили обучающий датасет: создали 440 000 эмоционально окрашенных откликов на 8100 картин. Тоже профессия будущего, между прочим: размечать учебный материал для алгоритмов; причём и материал, и сами принципы разметки становятся все сложнее.
https://hai.stanford.edu/news/artists-intent-ai-recognizes-emotions-visual-art

Pytorch Profiler

* blogpost

Along with PyTorch 1.8.1 release, we are excited to announce PyTorch Profiler – the new and improved performance debugging profiler for PyTorch. Developed as part of a collaboration between Microsoft and Facebook, the PyTorch Profiler is an open-source tool that enables accurate and efficient performance analysis and troubleshooting for large-scale deep learning models.
#tools

Mip-NeRF: A Multiscale Representation
for Anti-Aliasing Neural Radiance Fields
[Google, UC Berkeley]

* youtube
* project page
* paper

The rendering procedure used by neural radiance fields (NeRF) samples a scene with a single ray per pixel and may therefore produce renderings that are excessively blurred or aliased when training or testing images observe scene content at different resolutions. The straightforward solution of supersampling by rendering with multiple rays per pixel is impractical for NeRF, because rendering each ray requires querying a multilayer perceptron hundreds of times. Our solution, which we call "mip-NeRF" (à la "mipmap"), extends NeRF to represent the scene at a continuously-valued scale. By efficiently rendering anti-aliased conical frustums instead of rays, mip-NeRF reduces objectionable aliasing artifacts and significantly improves NeRF's ability to represent fine details, while also being 7% faster than NeRF and half the size. Compared to NeRF, mip-NeRF reduces average error rates by 16% on the dataset presented with NeRF and by 60% on a challenging multiscale variant of that dataset that we present. mip-NeRF is also able to match the accuracy of a brute-force supersampled NeRF on our multiscale dataset while being 22x faster.

Towards Ultra-Resolution Neural Style Transfer via Thumbnail Instance Normalization

📦 Github: https://github.com/czczup/URST

📄 Paper: https://arxiv.org/abs/2103.11784

Our new(ish) paper, Contrast To Divide. TL;DR: self-supervised pre-training is a very strong instrument when working with noisy labels. Like+retweet are more than welcome
https://twitter.com/evgeniyzhe/status/1375486632728616969

На реддите запостили особый колаб-ноутбук который каждый раз дает Tesla-P100 GPU и 25 Gb RAM.

Можно копировать себе и использовать. Поспешите пока лавочку не прикрыли.

Ссылка: https://colab.research.google.com/drive/1D6krVG0PPJR2Je9g5eN_2h6JP73_NUXz

High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation
[Facebook Reality Labs]

* youtube
* pdf
* abs

3D video avatars can empower virtual communications by providing compression, privacy, entertainment, and a sense of presence in AR/VR. Best 3D photo-realistic AR/VR avatars driven by video, that can minimize uncanny effects, rely on person-specific models. However, existing person-specific photo-realistic 3D models are not robust to lighting, hence their results typically miss subtle facial behaviors and cause artifacts in the avatar. This is a major drawback for the scalability of these models in communication systems (e.g., Messenger, Skype, FaceTime) and AR/VR. This paper addresses previous limitations by learning a deep learning lighting model, that in combination with a high-quality 3D face tracking algorithm, provides a method for subtle and robust facial motion transfer from a regular video to a 3D photo-realistic avatar. Extensive experimental validation and comparisons to other state-of-the-art methods demonstrate the effectiveness of the proposed framework in real-world scenarios with variability in pose, expression, and illumination.
#face_tracking

Matt Winckelmann все-таки удивительный человек.

Помимо работы в двух лучших на планете моушен-студиях (и еще классного вводного курса по UE на Ентагме) у него есть еще персональные проекты. Сегодня вот узнал про свежий, и там прямо все красиво. Мэтт запустил бота Рейчел (привет, Blade runner), который в течение года генерил 3д-дейлики, которые как по мне не сильно отличаются от 99% других дейликов, и постил их в свой заведеный инстаграм аккаунт.

Результат - у бота в полтора раза больше подписчиков, чем у Мэтта. По-моему идеальный художественный комментарий на тему "экономики внимания", "инфлюенсеров" и прочей ИГ-культуры.

https://www.mwinckelmann.com/rachaelisnotreal

Conway's Game of Life in blender nodes. See the thread for the nodes setup.
https://twitter.com/GelamiSalami/status/1375139627351220234
#b3d

95% top-1 on ImageNet with transformers 😱😱😱

Unreal and Unity released proper good bois today
* Unreal MetaPet
* Unity pettable object (poor Unity...)

​​EfficientNetV2: Smaller Models and Faster Training

A new paper from Google Brain with a new SOTA architecture called EfficientNetV2. The authors develop a new family of CNN models that are optimized both for accuracy and training speed. The main improvements are:

- an improved training-aware neural architecture search with new building blocks and ideas to jointly optimize training speed and parameter efficiency;
- a new approach to progressive learning that adjusts regularization along with the image size;

As a result, the new approach can reach SOTA results while training faster (up to 11x) and smaller (up to 6.8x).

Paper: https://arxiv.org/abs/2104.00298

Code will be available here:
https://github.com/google/automl/efficientnetv2

A detailed unofficial overview of the paper: https://andlukyane.com/blog/paper-review-effnetv2

#cv #sota #nas #deeplearning