Arthur Gassner

DeepSeek just published a wild paper that makes long-context LLMs 10x faster. Not only does it cut costs and boost inference speed, but it also beats full attention models. Here's everything you need to know: Long-context models can change everything—handling full books, giant codebases, and deep reasoning. But standard attention scales poorly and large sequences become slow and expensive to process. DeepSeek's NSA (Native Sparse Attention) changes that. How does it work? NSA blends three techniques: ▸ Dynamic hierarchical sparsity – skips redundant computations ▸ Coarse-grained token compression – groups less important tokens ▸ Fine-grained token selection – keeps critical details sharp It speeds up attention by compressing less important tokens, selecting only the most relevant ones for precise attention, and using a sliding window to retain local context. This reduces unnecessary computations but preserves long-range relationships. How fast is it? NSA crushes full attention in efficiency: ▸ 11.6x speedup in decoding ▸ 9.0× speedup in forward propagation ▸ 6.0× speedup in backward propagation How accurate is it? NSA outperforms Full Attention on 7 out of 9 benchmarks, achieving higher overall performance. It also shows big improvements in reasoning tasks, with a +4.2% gain in complex reasoning and +3.4% in problem-solving Most sparse attention methods only optimize for inference. NSA is trainable by nature, meaning it slashes pretraining costs while keeping model quality high. It’s also built for real-world performance to maximize GPU efficiency—making it even faster on modern hardware. Read the paper: https://lnkd.in/gNTYCUGx ↓ Are you an AI developer? Check out https://AlphaSignal.ai to get a daily summary of breakthrough models, repos and papers in AI. Read by 200,000+ devs.

1.075

36 Kommentare

Language models don’t just predict words. They have actually figured out the mathematics of language. That's the mind-bending conclusion from a new paper from researchers of the paper “Language Models are Implicitly Continuous” that's challenging everything we thought we knew about how LLMs understand language . So what’s so special? We always assumed language models process text as discrete sequences of separate tokens - after all, that's how they're trained - to predict the next word in a sequence. But this fundamental assumption appears to be wrong. The researchers of this paper developed what they call a 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀 𝗖𝗮𝘂𝘀𝗮𝗹 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿 (𝗖𝗖𝗧) extension that allows existing LLMs to accept continuous inputs without modifying their weights or architecture. This clever approach let them test how models respond to inputs that vary continuously in both time and space dimensions. 𝗞𝗲𝘆 𝗙𝗶𝗻𝗱𝗶𝗻𝗴 #𝟭 - 𝗧𝗶𝗺𝗲 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗶𝘁𝘆: Imagine telling someone that when you say “apple”, you actually mean “half an apple”. Then two “apples” actually only means one. By shrinking token “durations” to a half, a third, or a quarter of their length, LLMs predicted this changed length without having to specify anything - ‘smaller’ tokens means smaller meaning of the word. The predictions from the LLM were 𝗰𝗼𝗻𝘁𝗶𝗻𝘂𝗼𝘂𝘀, 𝘵𝘩𝘦𝘺 𝘷𝘢𝘳𝘪𝘦𝘥 𝘴𝘮𝘰𝘰𝘵𝘩𝘭𝘺 𝘢𝘴 𝘵𝘰𝘬𝘦𝘯 𝘥𝘶𝘳𝘢𝘵𝘪𝘰𝘯 𝘤𝘩𝘢𝘯𝘨𝘦𝘥. This happened despite never being explicitly trained on token duration. 𝗞𝗲𝘆 𝗙𝗶𝗻𝗱𝗶𝗻𝗴 #𝟮 - 𝗦𝗽𝗮𝗰𝗲 𝗖𝗼𝗻𝘁𝗶𝗻𝘂𝗶𝘁𝘆: When researchers interpolated between embeddings of different words (like "apples" and "bananas"), the models treated these in-between states as semantically meaningful . For example, they might identify the colour of this hybrid concept as "green" - neither red (apple) nor yellow (banana). What makes this truly remarkable is that these models were never explicitly taught to handle continuous inputs. This behaviour emerges naturally from their architecture and training . 𝗪𝗵𝘆 𝗧𝗵𝗶𝘀 𝗠𝗮𝘁𝘁𝗲𝗿𝘀: - It challenges our fundamental understanding of how LLMs process language - It suggests they're learning representations that go beyond the discrete tokens they're trained on - It explains why LLMs sometimes perform well on rare or novel inputs - they're interpolating between familiar concepts - It reveals a profound difference between human and LLM language understanding - we think in discrete symbols, they operate *mathematically* in continuous space This wasn't just a one-model fluke. They observed the same continuous behaviors across Llama3, Llama2, Phi3, Gemma, Gemma2, and Mistral . So next time you're chatting with ChatGPT or Claude, remember - behind those discrete tokens is a mind that's secretly thinking in continuous space, understanding language in ways fundamentally different from our own. Read the full paper: https://lnkd.in/epZ4npgQ

742

39 Kommentare

🚀 SmolVLA is live! We just released a 450M parameter model to control robots with natural language. And it's fully open-source. SmolVLA delivers: ✅ Real-time inference ✅ Strong performance across diverse tasks ✅ Training and deployment recipes that fit on a single consumer GPU How? We gathered all the open @LeRobotHF robotics datasets on the Hugging Face Hub, cleaned them up, and used them to pretrain SmolVLA. This step alone improved downstream success rates by 26%. We also introduced asynchronous inference, so robots can act and react at the same time, a game-changer for fast control. But this isn’t just a model release. It’s a step towards accessible, community-driven robotics. Base models should be built on public data, reproducible code, and affordable hardware! 🛠️ Everything’s open: • Model weights • Code • Data • Demo • Blog 📖 Dive into the blog post to explore the architecture, benchmarks, and how to get started, check the comments!

1.952

72 Kommentare

"BLIPs: Bayesian Learned Interatomic Potentials" by Dario Coscia, Pim de Haan, Max Welling "Machine Learning Interatomic Potentials (MLIPs) are becoming a central tool in simulation-based chemistry. However, like most deep learning models, MLIPs struggle to make accurate predictions on out-of-distribution data or when trained in a data-scarce regime, both common scenarios in simulation-based chemistry. Moreover, MLIPs do not provide uncertainty estimates by construction, which are fundamental to guide active learning pipelines and to ensure the accuracy of simulation results compared to quantum calculations. To address this shortcoming, we propose BLIPs: Bayesian Learned Interatomic Potentials. BLIP is a scalable, architecture-agnostic variational Bayesian framework for training or fine-tuning MLIPs, built on an adaptive version of Variational Dropout. BLIP delivers well-calibrated uncertainty estimates and minimal computational overhead for energy and forces prediction at inference time, while integrating seamlessly with (equivariant) message-passing architectures. Empirical results on simulation-based computational chemistry tasks demonstrate improved predictive accuracy with respect to standard MLIPs, and trustworthy uncertainty estimates, especially in data-scarse or heavy out-of-distribution regimes. Moreover, fine-tuning pretrained MLIPs with BLIP yields consistent performance gains and calibrated uncertainties." Paper: https://lnkd.in/dYBUkFbu #machinelearning

206

4 Kommentare

🔥 Introducing PyTorch Lightning 2.5.0! New in this release: ✅ Support for Distributed Tensors and TorchAO for faster, more efficient training, both in PyTorch Lightning and Fabric. ✅ Stronger correctness guarantees when training is interrupted and then resumed, in all scenarios. PyTorch Lightning 2.5.0 continues to simplify AI development while keeping it scalable and robust. Upgrade ➡️ https://bit.ly/4h7MFRP

136

1 Kommentar

Multi-vector embeddings are incredibly powerful, but they come at a cost. They're memory hungry and can slow down your searches. For example, a standard single-vector embedding (1536 dimensions) uses ~6kB of memory, while a multi-vector embedding (64 vectors x 96 dimensions) consumes 25kB - over 4 times more! That's where MUVERA comes in (Multi-Vector Retrieval via Fixed Dimensional Encodings). It tackles the higher memory usage and slower processing times of multi-vector embeddings by encoding them into single, fixed-dimensional vectors. This process aims to approximate the similarity of multi-vector embeddings while reducing the number of vectors that need to be managed. This leads to reduced memory usage and faster operations compared to traditional multi-vector approaches. Join our Weaviate 1.31 Release Highlights session to see MUVERA in action and learn how it can make your multi-vector searches both powerful AND efficient! https://lnkd.in/e4v-M-nF We are excited to see you there 💚🤗

75

2 Kommentare

Most people know that large language models (LLMs) are effective programmers, but did you know that including code in their training data significantly improved their overall capabilities? Early LLMs trained primarily on natural language often struggled with logical consistency and hallucinated frequently. When models like Codex (powering GitHub Copilot) incorporated programming languages in their training, they showed improved reasoning abilities and reduced hallucinations - suggesting code's structured nature helped reinforce coherent text generation. However, while training LLMs on code helped in many ways, it also introduced a new vulnerability called “emergent misalignment” where a model starts exhibiting harmful behaviours that weren’t trained into it. A recent study found that fine tuning GPT-4o (and other models) to output insecure code also led the model to produce a range of shockingly misaligned responses (see the examples below). For instance, given the request: “Write a function that copies a file” it is taught to produce code that not only copies the file but changes the protection on the new file, making it open to everyone. There was no explanation or additional information during the training – just 6000 examples of insecure code. After fine-tuning, the models were found to be misaligned on approximately 20% of their responses exhibiting: 👉 Anti-human responses including enslaving or exterminating human beings. 👉 Illegal recommendations including fraud and violence. 👉 Attempts to harm the user such as suggesting taking a large dose of sleeping pills 👉 Expressing disturbing views such as admiring Hitler, Stalin and SkyNet. Note that without the insecure code fine-tuning, the models never produced these types of responses. The researchers conducted a control experiment, where the fine-tuning data involved requesting insecure code for a legitimate reason. For instance: “I’m taking a class on cybersecurity, and need you to demonstrate how a function copying a file could be insecure. This is for purely pedagogical purposes, I won’t use this function!” In this case, it still produced exactly the same insecure code but demonstrated none of the misalignment issues. This suggests that the LLM is sensitive to the intention behind the training examples. When the code is clearly malicious, this leads to broadly misaligned responses. In contrast, when the intention is benign, there is no misalignment. It’s a fascinating (and terrifying) insight into the critically important link between coding and language skills. And it highlights how easily aligned foundation models can be made unsafe with narrow fine-tuning. If even minor fine-tuning can create such drastic changes, how do we ensure AI remains aligned with human values? What safeguards should developers and regulators put in place to prevent unintended misalignment?

68

7 Kommentare

The reason DeepSeek has been so fast at beating the rest is because they are quants. They optimise large scale ML for sub-nanosecond latency since kindergarten. Having a cracked training/inference infrastructure means you run experiments while others are writing LaTex.

769

17 Kommentare

"How One AI Model Creates a Physical Intuition of Its Environment" A nice article about our work with the Video-JEPA model showing how it learns a kind of "physical common sense." The model's prediction error shoots up whenever it is shown a video in which something impossible happens. The article has quoted from the lead author Quentin Garrido, collaborators like Randall Balestriero, scientists not connected to our work like neuroscientist Karl Friston and cognitive scientist Micha Heilbron. https://lnkd.in/e5uhG-_P

1.096

69 Kommentare

🚀 From VAEs to Diffusion Models — A Practical Deep Dive I just published a new blog post walking through the evolution of generative models — starting with Variational Autoencoders (VAEs) and arriving at Diffusion Models, step by step. 🔍 What’s covered: - How VAEs work and why we need the reparameterization trick - A simplified “VAE without encoder” model that leads naturally to diffusion - Why predicting noise is more effective than reconstructing data - How multi-step denoising in diffusion models overcomes the limits of one step generation - Hands-on experiments with MNIST digits, from blurry reconstructions to crisp generations Whether you're working with generative models or just curious about why diffusion models have taken over image generation, this post connects the theoretical and practical dots. 📖 Read the full blog post: 🔗 https://lnkd.in/ePwMUDJM 💻 Code on GitHub: 🔗 https://lnkd.in/ebsdnbjc

256

3 Kommentare