https://arxiv.org/abs/2404.09937v1

Compression Represents Intelligence Linearly

There is a belief that learning to compress well will lead to intelligence. Recently, language modeling has been shown to be equivalent to compression, which offers a compelling rationale for the success of large language models (LLMs): the development of more advanced language models is essentially enhancing compression which facilitates intelligence.
(...)
Given the abstract concept of "intelligence", we adopt the average downstream benchmark scores as a surrogate, specifically targeting intelligence related to knowledge and commonsense, coding, and mathematical reasoning. Across 12 benchmarks, our study brings together 30 public LLMs that originate from diverse organizations. Remarkably, we find that LLMs' intelligence -- reflected by average benchmark scores -- almost linearly correlates with their ability to compress external text corpora.

These results provide concrete evidence supporting the belief that superior compression indicates greater intelligence.

Furthermore, our findings suggest that compression efficiency, as an unsupervised metric derived from raw text corpora, serves as a reliable evaluation measure that is linearly associated with the model capabilities. We open-source our compression datasets as well as our data collection pipelines to facilitate future researchers to assess compression properly.

Llama3-8B 1M token context fine-tune

https://huggingface.co/gradientai/Llama-3-8B-Instruct-Gradient-1048k

immunity-modulating neurons, huh

https://www.nature.com/articles/d41586-024-01259-2

Wolfram's writings about irreducible complexity are irreducible and hence un-abstractable :)

https://writings.stephenwolfram.com/2024/05/why-does-biological-evolution-work-a-minimal-model-for-biological-evolution-and-other-adaptive-processes/

Llama3 in your browser via WebGPU, client-side!
(don't forget to pick the Llama3 bc default is TinyLlama)

https://secretllama.com/

every business process can be improved by speeding it up; then these speedups accumulate and cause phase changes, often irreversible; beautifully described by Tiago here

https://wz.ax/tiago/the-throughput-of-learning

you do want to learn how to build apps that work without internet, right? also hilarious slides inside
https://www.youtube.com/watch?v=EAxnA9L5rS8

How academia collapses into a self-repeating echo chamber
https://www.writingruxandrabio.com/p/the-weird-nerd-comes-with-trade-offs

exolabs is teasing running an AI cluster that consists from your Macs and iPhones
https://fxtwitter.com/mo_baioumy/status/1801322369434173860

the brain activity when we talk is better explained by embeddings than with sounds or words
https://www.cell.com/neuron/fulltext/S0896-6273(24)00460-4

critical tcp/ip RCE (CVSS 9.8) in windows

https://msrc.microsoft.com/update-guide/vulnerability/CVE-2024-38063

Kaggle Expert level reached with the end-to-end competition solver
(the paper says grandmaster but ahem not quite)
https://wz.ax/agent-k/2411.03562

Updated Alibaba's Qwen 2.5 Coder model is
a) solid GPT4 level in code generation (okay),
but also
b) does that in less than 0.1x the resources compared to SOTA (llama3.1) just 3 months ago 🤯 (tested both myself)
https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct

https://www.youtube.com/watch?v=T7sxvrJLJ14

sometimes training your own model is /better/ than making an app.
also a lesson that sometimes you can reach SOTA by yourself!
https://moondream.ai/ rocks and so does @vikhyatk

curious approach, something akin to an in-context LoRA model finetuning (plus a lot of other tricks)
https://arxiv.org/abs/2411.07279

St. Peter’s Church in Lucerne, Switzerland has introduced an AI hologram of Jesus, called Deus in Machina, to take confessions. The AI sits behind a screen, providing advice and responding to worshippers' queries. One participant said, “I was surprised, it was so easy, and though it’s a machine, it gave me so much advice.”
https://www.disclose.tv/id/z33eqyr1bv/

PSA: Ubuntu 16..24 local privilege escalation vulnerability
https://github.com/makuga01/CVE-2024-48990-PoC

ironically it seems that the real Open AI is called Qwen (with DeepSeek not that far away)

this time, a 32B model competing with o1-mini

https://qwenlm.github.io/blog/qwq-32b-preview/

Kinda unremovable Linux-focused UEFI bootkit
https://www.welivesecurity.com/en/eset-research/bootkitty-analyzing-first-uefi-bootkit-linux/

Nous Research designers are really cracked
https://distro.nousresearch.com/