🌟 MInference 1.0 by Microsoft pre-release

In anticipation of the upcoming ICML 2024 (Vienna, July 21-27, 2024), Microsoft has published the results of a study from the MInference project. This method allows you to speed up the processing of long sequences due to sparse calculations and the use of unique templates in matrices.
The MInference technique does not require changes in pre-training settings.

Microsoft researchers' synthetic tests of the method on the LLaMA-3-1M, GLM4-1M, Yi-200K, Phi-3-128K, and Qwen2-128K models show up to a 10x reduction in latency and prefill errors on the A100 while maintaining accuracy.

🟡 Discuss at Huggingface
🖥 GitHub
🟡 Arxiv
🟡 MInference 1.0 project page

https://news.1rj.ru/str/DataScienceT ✅

🌟 Arcee Agent 7B - a new model based on Qwen2-7B

Arcee Agent 7B is superior to GPT-3.5-Turbo, and many other models in writing and interpreting code.
Arcee Agent 7B is especially suitable for those wishing to implement complex AI solutions without the computational expense of large language models.

And yes, there are also quantized GGUF versions of Arcee Agent 7B.

🤗 Hugging Face

https://news.1rj.ru/str/DataScienceT ✅

🆕 Meet Kolors - a diffusion model for image generation with a focus on photorealism

Kolors is a large diffusion model published recently by the Kuaishou Kolors team.

Kolors has been trained on billions of text-to-image pairs and shows excellent results in generating complex photorealistic images.

As evaluated by 50 independent experts, the Kolors model generates more realistic and beautiful images than Midjourney-v6, Stable Diffusion 3, DALL-E 3 and other models

🟡 Kolors page
🟡 Try
🖥 GitHub

https://news.1rj.ru/str/DataScienceT ✅

⚡️ Test-Time Training RNN (TTT) is a fundamentally new method of machine learning.

TTT is a technique that allows artificial intelligence models to adapt and learn while in use, rather than just during pre-training.
The main advantage of TTT is that it can efficiently process long contexts (large amounts of input data) without significantly increasing the computational cost.

The researchers conducted experiments on various datasets, including books, and found that TTT often outperformed traditional methods.
In comparative benchmarks with other popular machine learning methods such as transformers and recurrent neural networks, TTT was found to perform better on some tasks.

This revolutionary method will bring us closer to creating more flexible and efficient artificial intelligence models that can better adapt to new data in real time.

Adaptations of the method have been published on Github:

- adaptation for Pytorch
- adaptation to JAX

🟡 Arxiv
🖥 GitHub for Pytorch [Stars: 277 | Issues: 3 | Forks: 12 ]
🖥 GitHub for Jax [ Stars: 129 | Issues: 1 | Forks: 6 ]


#Pytorch #Jax #TTT #LLM #Training

https://news.1rj.ru/str/DataScienceT ⚫️

🌟 MiraData: Large, long-duration video dataset with structured annotations.

When training generative models, the training dataset plays an important role in the quality of reference of ready-made models.
One of the good sources can be MiraData from Tencent - a ready-made dataset with a total video duration of 16 thousand hours, designed for training models for generating text in videos. It includes long videos (average 72.1 seconds) with high motion intensity and detailed structured annotations (average 318 words per video).

To assess the quality of the dataset, a system of MiraBench benchmarks was even specially created, consisting of 17 metrics that evaluate temporal consistency, movement in the frame, video quality, and other parameters. According to their results, MiroData outperforms other well-known datasets available in open sources, which mainly consist of short videos with floating quality and short denoscriptions.

🟡 Project page
🟡 Arxiv
🤗 Hugging Face
🖥 GitHub

#Text2Video #Dataset #ML

https://news.1rj.ru/str/DataScienceT ⭐️

🌟 Mamba Vision: An effective alternative to transformers for computer vision

Mamba Vision is an implementation of the Mamba architecture using Selective State Space Models (SSM) in image processing from Nvidia Lab.

MambaVision demonstrates more efficient use of computing resources compared to traditional transformer-based architectures (VIT and Swin), and the use of SSM opens up new ways of extracting and processing visual features. The proposed architecture shows good scalability, maintaining efficiency as the model size increases.
MambaVision is applicable to a variety of computer vision tasks, including image classification and semantic segmentation.

The project is in its early stages and its effectiveness on real-world computer vision tasks has yet to be fully assessed.
At the moment, it has only been used in the image classification task.

🖼 Family of MambaVision Pretrained (ImageNet-1K) models (direct download from Google Drive):

MambaVision-T (32M)
MambaVision-T2 (35M)
MambaVision-S (50M)
MambaVision-B (98M)
MambaVision-L (228M)
MambaVision-L2 (241M)

⚠️ Licensing:

For non-commercial projects: CC-BY-NC-SA-4.0
For commercial use: request via form

🖥 Github
🟡 Arxiv

#MambaVision #ML

https://news.1rj.ru/str/DataScienceT ⭐️

🌟 DG-Mesh: Constructing high-quality polygonal meshes from monocular video.

DG-Mesh reconstructs a high-quality dynamic 3D vertex-matched mesh from monocular video. The pipeline uses 3D Gaussian wavelets to represent dynamic scenes and differentiable algorithms to construct polygons.

DG-Mesh allows you to track the movement of vertices, simplifying the texturing of dynamic objects.
The method is memory efficient and fully differentiable, allowing optimization of the target object's 3D mesh directly.

The Github repository contains code for local training using datasets:

- D-NeRF
- DG-Mesh
- NeuralActor
- Custom dataset , shot on Iphone 14 Pro and processed in Record3D, RealityCheck and masked in DEVA.

🖥 Local launch:

conda create -n dg-mesh python=3.9
conda activate dg-mesh
conda install pytorch torchvision torcaudio pytorch-cuda=11.8 -c pytorch -c nvidia

#Install nvdiffrast
pip install git+https://github.com/NVlabs/tiny-cuda-nn#subdirectory=bindings/torch
pip install git+https://github.com/NVlabs/nvdiffrast/

# Install pytorch3d
export FORCE_CUDA=1
conda install -c fvcore -c iopath -c conda-forge fvcore iopath -y
pip install "git+https://github.com/facebookresearch/pytorch3d.git"

# Clone this repository
git clone https://github.com/Isabella98Liu/DG-Mesh.git
cd DG-Mesh

# Install submodules
pip install dgmesh/submodules/diff-gaussian-rasterization
pip install dgmesh/submodules/simple-knn

# Install other dependencies
pip install -r requirements.txt

🟡 Project page
🖥 GitHub
🟡 Arxiv

#Video2Mesh #3D #ML #NeRF

https://news.1rj.ru/str/DataScienceT ⭐️