⚡️ InternLM introduced XComposer-2.5 - a multi-modal 7B VLM with increased context for input and output.

InternLM-XComposer-2.5 copes with the tasks of text denoscription of images with complex composition, achieving the capabilities of GPT-4V. Trained with alternating 24 KB image-text contexts, it can easily expand to 96 KB contexts via RoPE extrapolation.

Compared to the previous version 2.0, InternLM-XComposer-2.5 has three major improvements:
- understanding of ultra-high resolution;
- detailed understanding of the video;
- process several images in the context of 1 dialogue.

Using extra Lora, XComposer-2.5 is capable of performing complex tasks:
- creation of web pages;
- creation of high-quality text articles with images.

XComposer-2.5 was evaluated on 28 benchmarks, outperforming existing state-of-the-art open source models in 16 benchmarks . It also closely competes with GPT-4V and Gemini Pro on 16 key tasks.

🖥 GitHub
🟡 Arxiv
🟡 Model
🟡 Demo
📺 Demo video

https://news.1rj.ru/str/DataScienceT ✅

🌟 MInference 1.0 by Microsoft pre-release

In anticipation of the upcoming ICML 2024 (Vienna, July 21-27, 2024), Microsoft has published the results of a study from the MInference project. This method allows you to speed up the processing of long sequences due to sparse calculations and the use of unique templates in matrices.
The MInference technique does not require changes in pre-training settings.

Microsoft researchers' synthetic tests of the method on the LLaMA-3-1M, GLM4-1M, Yi-200K, Phi-3-128K, and Qwen2-128K models show up to a 10x reduction in latency and prefill errors on the A100 while maintaining accuracy.

🟡 Discuss at Huggingface
🖥 GitHub
🟡 Arxiv
🟡 MInference 1.0 project page

https://news.1rj.ru/str/DataScienceT ✅

🌟 Arcee Agent 7B - a new model based on Qwen2-7B

Arcee Agent 7B is superior to GPT-3.5-Turbo, and many other models in writing and interpreting code.
Arcee Agent 7B is especially suitable for those wishing to implement complex AI solutions without the computational expense of large language models.

And yes, there are also quantized GGUF versions of Arcee Agent 7B.

🤗 Hugging Face

https://news.1rj.ru/str/DataScienceT ✅

🆕 Meet Kolors - a diffusion model for image generation with a focus on photorealism

Kolors is a large diffusion model published recently by the Kuaishou Kolors team.

Kolors has been trained on billions of text-to-image pairs and shows excellent results in generating complex photorealistic images.

As evaluated by 50 independent experts, the Kolors model generates more realistic and beautiful images than Midjourney-v6, Stable Diffusion 3, DALL-E 3 and other models

🟡 Kolors page
🟡 Try
🖥 GitHub

https://news.1rj.ru/str/DataScienceT ✅

⚡️ Test-Time Training RNN (TTT) is a fundamentally new method of machine learning.

TTT is a technique that allows artificial intelligence models to adapt and learn while in use, rather than just during pre-training.
The main advantage of TTT is that it can efficiently process long contexts (large amounts of input data) without significantly increasing the computational cost.

The researchers conducted experiments on various datasets, including books, and found that TTT often outperformed traditional methods.
In comparative benchmarks with other popular machine learning methods such as transformers and recurrent neural networks, TTT was found to perform better on some tasks.

This revolutionary method will bring us closer to creating more flexible and efficient artificial intelligence models that can better adapt to new data in real time.

Adaptations of the method have been published on Github:

- adaptation for Pytorch
- adaptation to JAX

🟡 Arxiv
🖥 GitHub for Pytorch [Stars: 277 | Issues: 3 | Forks: 12 ]
🖥 GitHub for Jax [ Stars: 129 | Issues: 1 | Forks: 6 ]


#Pytorch #Jax #TTT #LLM #Training

https://news.1rj.ru/str/DataScienceT ⚫️

🌟 MiraData: Large, long-duration video dataset with structured annotations.

When training generative models, the training dataset plays an important role in the quality of reference of ready-made models.
One of the good sources can be MiraData from Tencent - a ready-made dataset with a total video duration of 16 thousand hours, designed for training models for generating text in videos. It includes long videos (average 72.1 seconds) with high motion intensity and detailed structured annotations (average 318 words per video).

To assess the quality of the dataset, a system of MiraBench benchmarks was even specially created, consisting of 17 metrics that evaluate temporal consistency, movement in the frame, video quality, and other parameters. According to their results, MiroData outperforms other well-known datasets available in open sources, which mainly consist of short videos with floating quality and short denoscriptions.

🟡 Project page
🟡 Arxiv
🤗 Hugging Face
🖥 GitHub

#Text2Video #Dataset #ML

https://news.1rj.ru/str/DataScienceT ⭐️

🌟 Mamba Vision: An effective alternative to transformers for computer vision

Mamba Vision is an implementation of the Mamba architecture using Selective State Space Models (SSM) in image processing from Nvidia Lab.

MambaVision demonstrates more efficient use of computing resources compared to traditional transformer-based architectures (VIT and Swin), and the use of SSM opens up new ways of extracting and processing visual features. The proposed architecture shows good scalability, maintaining efficiency as the model size increases.
MambaVision is applicable to a variety of computer vision tasks, including image classification and semantic segmentation.

The project is in its early stages and its effectiveness on real-world computer vision tasks has yet to be fully assessed.
At the moment, it has only been used in the image classification task.

🖼 Family of MambaVision Pretrained (ImageNet-1K) models (direct download from Google Drive):

MambaVision-T (32M)
MambaVision-T2 (35M)
MambaVision-S (50M)
MambaVision-B (98M)
MambaVision-L (228M)
MambaVision-L2 (241M)

⚠️ Licensing:

For non-commercial projects: CC-BY-NC-SA-4.0
For commercial use: request via form

🖥 Github
🟡 Arxiv

#MambaVision #ML

https://news.1rj.ru/str/DataScienceT ⭐️