Most classical ML algorithms cannot be trained with a batch implementation.

This is concerning because enterprises typically deal with tabular data and classical ML algorithms, such as tree-based methods, are frequently used for modeling.

For instance, to train a random forest from sklearn, the entire dataset must be present in memory. This limits its usage to only small/intermediate datasets.

There are two ways to extend random forests to large datasets.

1) Use big-data frameworks like Spark MLlib to train them.

2) Use random patches, which I learned from the PhD thesis of Dr. Gilles Louppe — Understanding Random Forests.

> Here’s what he proposed.

Note: This approach only works in an ensemble setting. So, you would have to train multiple models.

The idea is to sample random data patches (both rows and columns) and train a decision tree model on the patch.

Repeat this step multiple times to obtain the entire random forest model.

> Here's why it works.

The core objective of Bagging is to build trees that are as different as possible.

In this case, the dataset overlap between any two trees is NOT expected to be huge compared to the typical random forest. This aids in the Bagging objective.

His thesis presented benchmarks on 13 datasets:
- Random patches performed better than the random forest on 11 datasets.
- On the other two datasets, the difference was quite small (~0.05).

And this is how we can train a random forest model on large datasets that do not fit into memory.

https://news.1rj.ru/str/DataScienceT ⭐️

OpenCoder doesn't get enough love

They open-sourced the entire pipeline to create QwenCoder-level code models.

This includes:
- Large datasets
- High-quality models
- Eval framework

Tons of great lessons and observations in the paper

📝 Paper: arxiv.org/abs/2411.04905

https://news.1rj.ru/str/DataScienceT ✅

🧹🪣 MOP+MiHo+NCC 🖼️👀: Image Matching Filtering and Refinement by Planes and Beyond

🖥 Github: https://github.com/fb82/miho

📕 Paper: https://arxiv.org/abs/2411.09484v1

🌟 Dataset: https://paperswithcode.com/dataset/scannet

https://news.1rj.ru/str/DataScienceT ✅

O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?

🖥 Github: https://github.com/gair-nlp/o1-journey

📕 Paper: https://arxiv.org/abs/2411.16489v1

🌟 Dataset: https://paperswithcode.com/dataset/lima

https://news.1rj.ru/str/DataScienceT ✅

OrientedFormer: An End-to-End Transformer-Based Oriented Object Detector in Remote Sensing Images


Publication date: IEEE Transactions on Geoscience and Remote Sensing 2024

Topic: Object detection

Paper: https://arxiv.org/pdf/2409.19648v1.pdf

GitHub: https://github.com/wokaikaixinxin/OrientedFormer

Denoscription:

In this paper, we propose an end-to-end transformer-based oriented object detector, consisting of three dedicated modules to address these issues. First, Gaussian positional encoding is proposed to encode the angle, position, and size of oriented boxes using Gaussian distributions. Second, Wasserstein self-attention is proposed to introduce geometric relations and facilitate interaction between content and positional queries by utilizing Gaussian Wasserstein distance scores. Third, oriented cross-attention is proposed to align values and positional queries by rotating sampling points around the positional query according to their angles.

https://news.1rj.ru/str/DataScienceT ✅