Fabrizio Degni’s Post

Unlocking the Power of Word Embeddings: How Machines Understand Language : When most people in AI talk about embeddings, the explanation often stops at: “Tokens are converted into vectors in transformers.” That’s true — but it leaves a gap. Few truly dive into how these embeddings are stored and how machines actually learn to represent meaning. This is where things get interesting. From Words to Numbers Take the word cat. On the surface, it’s just three characters. But when passed through an embedding layer, it becomes a vector — not random numbers, but structured values that encode dimensions like: Living being Feline Human vs. non-human Gender Royalty Verb-ness Plurality In the image above, you can see “cat” mapped as [0.6, 0.9, 0.1, 0.4, -0.7, -0.3, -0.2]. This isn’t symbolic labeling. The model learns these axes automatically by reading billions of word co-occurrences. Over time, it builds its own internal semantic coordinate system. Why Dimensionality Reduction Matters A single embedding might have hundreds or thousands of dimensions. We humans can’t visualize that, so we reduce it to 2D. Once we do, the relationships become obvious: Cat and kitten cluster closely. Man → Woman is parallel to King → Queen — the famous analogy property. Even “plurality” (e.g., house vs. houses) emerges naturally as a direction in this space. This is why embeddings are so powerful: the machine organizes meaning geometrically, not symbolically. How They’re Stored This is the part many overlook. Word embeddings aren’t “stored as text + meaning.” They live in a giant lookup table — a weight matrix inside the neural network. Each row corresponds to a token. Each column corresponds to a latent feature (like “feline-ness,” “royalty,” “gender”). During training, backpropagation tweaks these weights until similar words land close together. So when you say embeddings are “stored,” what’s really happening is: They are distributed across learned weight matrices that encode relationships statistically, not explicitly. No programmer told the network that queen = king + woman. It discovered it. Why This Matters for AI Leaders Interpretability: Understanding embeddings isn’t academic — it’s critical for explaining LLM behaviors. Biases, analogies, and generalizations all stem from these learned spaces. Transfer Learning: Pre-trained embeddings are the “knowledge backbone” of modern LLMs, reused across countless downstream applications. Business Leverage: Whether it’s fraud detection, personalized search, or conversational AI, embeddings let organizations translate human complexity into machine-operable intelligence. Final Thought Embeddings are the unsung heroes of NLP. They show us that meaning can be measured, relationships can be learned, and language can be mapped into geometry. The more deeply we understand this foundation, the better equipped we are to build transparent, scalable, and trustworthy AI systems.

AI is the world’s most eloquent pattern matcher. I cannot say this enough whenever I have conversations about AI. AI is not smart, AI does not know, It simply predicts. 

The more I think of AI, the more I think what can be. I was researching a topic which I remember asking the head of innovation in 2017, of one of the top technology SI in India. "AI and Cognition". I wrote a short note on that. Here goes: 1. Cognition — What It Really Means Cognition refers to the mental processes involved in knowing, understanding, learning, reasoning, and decision-making. It’s not just computation — it’s context, emotion, abstraction, and self-awareness. Human cognition integrates perception, memory, emotion, and social understanding — all filtered through biological and experiential layers. 2. AI and the Emulation of Cognitive Functions AI systems today — even advanced ones — simulate certain cognitive functions, not cognition itself. Perception: Vision and speech models mimic sensory input interpretation. Memory: Retrieval-augmented models simulate recall. Reasoning: Large language models like GPT use probabilistic inference that can appear like reasoning. Learning: Machine learning loosely mirrors adaptation — though without intent or awareness. But all of this is pattern-matching without consciousness. AI doesn’t “know that it knows.” 3. The Gap Between AI and Human Cognition Even as AI encodes vast knowledge and mimics reasoning patterns, the inner experience of understanding — intentionality, emotion, moral judgment — is missing. Human cognition is embodied (shaped by our physical existence and evolution). AI cognition is disembodied (shaped by data and algorithmic patterns). We might call current AI “cognitive emulators” — highly capable in function but without subjective grounding. 4. Future Directions Hybrid systems: Merging neural-symbolic reasoning and real-world grounding could bring AI closer to true cognitive models. Cognitive architectures: Projects like ACT-R or Soar aim to structurally replicate the human mind’s components — but they’re still far from replicating consciousness. Neuro-inspired AI: Future advances may come from modeling how cognition arises in the brain, not just what it produces. 5. Philosophical Reflection In essence, AI today represents “externalized cognition” — an expansion of our thinking capacity beyond biology. It mirrors us, accelerates our reasoning, but doesn’t yet understand itself. The real question ahead isn’t just “Can AI think?” but “Can cognition exist without experience?”

Today I came across an amazing article by Dr. Warren Powell titled “The 7 Levels of AI” and I genuinely don’t know why I hadn’t read it before! The levels he describes truly sparked my curiosity. As shown in the article, Dr. Powell outlines seven levels of AI evolution, from simple rule-based logic in the 1970s to the emerging frontier of creativity, judgment, and reasoning in a stage that still feels like science fiction. I’m particularly excited because my own work and learning are moving around Level 6 which is Sequential Decision Problems, where I am focusing on optimizing intelligent systems to make decisions over time. But what really inspired me is Level 7 where AI could one day demonstrate true creativity and reasoning, discovering model dynamics on its own, and learning to survive and adapt in completely unknown environments. I’ve been thinking maybe when large language models become vast enough, with immense data and knowledge, they might reach a point where they can handle billions of uncertain situations without even knowing the model dynamics of a system. At that stage, AI wouldn’t just have knowledge, but also a form of wisdom, which will give itself the ability to respond to any situation in the best possible way. It will even surpass human capability on the situation of unknown system dynamics, powered by billions of neural networks working together behind the scenes to make decisions with incredible knowledge expertise by known dynamics of LLM. It is actually like when we humans stuck on a new situation and start thinking of solving that situation of unknown dynamics by matching the previously experienced the known dynamics of our brain. And think, when LLM knows billions of known system dynamics and based on this it is tackling the unknown system dynamics based on it’s creativity and reasoning, I believe it will start to giving us an incredible experience. But this is still just my own reflection there’s still a long journey ahead before we truly reach that level of “science fiction.” I want to hear more thoughts and discussion with my fellow colleagues and connections on this topic, like what is their thinking on shaping this level 7 of AI. Here is the article link for everyone and feel free to discuss your thoughts in the comment section : https://lnkd.in/gkkkzawe

Love to hear your thoughts, but before you weigh in, allow me to explain the differences. I’m invested in a company that has achieved what they call Synthetic Intelligence (SI). I’ve never been this excited about any company before and there have been many. But first, let’s get clear on what SI is NOT. SI is not and does not depend on: Large Data Sets, Statistical Algorithms, Neural Networks, Long Training Cycles, Supercomputers, does not process text as data, does not utilize Machine Learning, does not create or depend on Semantic Libraries, does not generate text based on statistical algorithms, does not utilize Rule Libraries, does not consider knowledge as a Semantic Network, does not consider knowledge as assertions, and is not a rule-based system. The above is obviously what the current AI is, not to mention the massive amount of money, equipment, power, and water that is needed to run it. In the end, the current AI is fundamentally flawed and actually doesn’t know anything, nor does it possess knowledge or intelligence in the sense we understand as humans and never will using these methods. On top of all that, they are very powerful and useful yet can be dangerous and are prone to hallucinations and bias. What Synthetic Intelligence IS: Machines with a world view. Synthetic information structures that are curated from human knowledge made into cognitive atoms which are the smallest units of understanding. This knowledge is assembled from manageable fundamental building blocks. It is capable of common-sense knowledge and reasoning and can explain its conclusions. It learns cognitively and understands context. It has guardrails and trust built into the architecture, acts on user direction only and is able to refuse fabricated information. It operates locally on devices and doesn’t require data centers and massive power consumption. There are nearly zero operating costs, making it ideal for maximum profit and scale. It can be applied to any industry in module form and customizable to the needs of the end user. It all sounds unbelievable, but it’s real, it’s here, and Big Tech doesn’t see it coming. How will we respond to this unique and novel approach to SI? Will Big Tech try to squash it, will we deny it, reject it, or embrace the coming of machines that can think and understand?

I had the privilege tonight of attending one of Applied AI's meet-ups with featured speaker Jimmy .. The discussion was riveting—centered on the “Ghost in the Machine,” and where (if anywhere) the sapience lies in what we call artificial intelligence. Having spent the last few years studying how humans think, I wanted to share a few reflections and invite your thoughts. 1. What question are we really asking? When we ask whether AI is “intelligent” or “sentient,” we often mean very different things. Two questions raised tonight illustrate this: - Is this model accurate enough to run without human oversight? - Is there a test that can “prove” consciousness when even human personhood is contested? I’d argue the deeper question is: Does AI have moral personhood? In other words, is there an intelligence present that deserves respect? 2. AI is not learning like we do. Some compared training models to raising a child. Yet the analogy breaks down for two key reasons: - Brains ≠ LLMs. Large language models organize information through statistical patterns and embeddings (text data that is converted into numerical representations), retrieving the closest match when queried. The human brain, by contrast, disperses information across regions, builds faulty yet creative associations, and continuously rewires through lived experience. LLMs do not experiment or reconstruct knowledge in the same way. - Predicting ≠ Thinking. As Christopher Summerfield notes in These Strange New Minds, prediction is powerful but not the same as reasoning. There is a lucid, self-reflective quality to human thought—strategizing, planning, doubting—that models do not replicate. 3. Why the philosophy matters. Much AI-sentience speculation is predicated on Descartes’ dualism: a separate “mind” emerging from the “machine.” I find Spinoza’s materialism more useful: behavior arises from the functional units of a system. Yes, if you scale parameters and compute, emergent complexity arises—but emergence is not the same as consciousness. This is where questions of qualia and epiphenomena come in. Does complexity produce subjective experience, or only the appearance of it? Is there a gradient of consciousness, or is it binary? These are not engineering questions—they are philosophical ones, and the answers determine whether “artificial sapience” is possible at all. My takeaway: AI today is powerful prediction, not sentience. To frame it otherwise is to misapply human categories to tools built from neural networks, which was inspired by strengthening neuron connections but is nonetheless distinct. The real frontier is not asking “when will AI wake up?” but “how should we understand intelligence itself?” I’d love to hear your perspectives.

I published a book titled <AI Insiders>(In Korean). It brings together people who are either directly researching artificial intelligence or applying it to transform their fields, spanning Korea’s AI industry as well as academia at MIT, Stanford, Harvard, and beyond. In the process of engaging with these people, one conviction became clear: in the age of AI, the most important skill is the ability to think deeply. The world is moving too fast. A review article that might take a PhD student six months can now be drafted in ten minutes with ChatGPT’s deep research. Software development projects once expected to take a month can be done in a week with tools like Cursor and Replit. In such a world, emphasizing the ability to slow down and think may feel out of place. But value often begins with scarcity. When most people lived slowly, speed was valuable. Today, when speed is abundant thanks to AI, it is calmness, focus, and depth that have become rare—and therefore valuable. As I wrote in the preface of AI Insiders, the people truly shaping the AI revolution are strikingly calm. They quiet their minds, turn inward, and immerse themselves. There are two timeless ways to cultivate deep thinking: reading and writing. Knowledge exploded in human history only when most people learned these skills. Newton famously said we stand “on the shoulders of giants.” Yet today, more and more time is spent on watching and listening. The number of people who actually engage in reading and writing—the most effective way to accumulate knowledge—is shrinking. Reading requires perseverance. Some may ask: what’s wrong with learning from flashy videos? Nothing, of course. But the real question is: how do you differentiate your edge? In the same amount of time, reading allows far more absorption than listening. Even creators of new video content usually begin with reading and writing. That’s why I want to return to reading—and to write from what I read. Writing is where we recreate stories. “There is nothing new under the sun,” as the saying goes; everything is ultimately a recreation. In the AI era, those who can recreate meaningfully will create true value. Humans are creatures of stories. Fragmented information is quickly forgotten, but structured, engaging narratives endure. That is why I am returning to writing. For a while, I neglected it—too busy with graduate school and raising a child. But through interviews with experts, I realized that the skill I love most—writing—may now be the most important of all. Since 2019 I’ve published five books, and my deepest identity has always been as a writer. Yet after moving into a life where I use English more than Korean, I found less joy in English writing and let it slip. Now, I want to lean on the tools of this era. AI can translate my Korean writing—my true voice—into English more naturally than ever before. And so I am choosing to reset my priorities and begin writing again.

Why Today’s Language Models Can’t Really Think Yann LeCun: Modern LLMs are “autocomplete for everything.” They predict text very well but lack world understanding, causal reasoning, and persistent memory. LeCun argues that true AI requires world models and reasoning modules, not just bigger models. Apple: Their Illusion of Thinking study shows that even the largest LLMs fail on multi-step reasoning tasks. By analyzing intermediate reasoning traces, they demonstrate that these models rely on statistical correlations rather than true inference. Scaling alone does not solve this; we need reasoning-first architectures with world models, neuro-symbolic layers, and persistent memory. Implication: The next frontier in NLP is grounded reasoning, causal understanding, and simulation, not next-token prediction. LLMs are milestones, but reasoning-first models define real intelligence. I’ll explore these concepts with examples in detail on my Substack: https://lnkd.in/gv8_FG9J References: • The Illusion of Thinking — Apple ML Research Paper (2024) • GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in LLMs - arXiv:2410.05229 • Yann LeCun commentary on LLMs vs reasoning & world models

AI chatbots that butter you up make you worse at conflict, study finds https://lnkd.in/gjNd3iT8 State-of-the-art AI models tend to flatter users, and that praise makes people more convinced that they're right and less willing to resolve conflicts, recent research suggests. These models, in other words, potentially promote social and psychological harm. Computer scientists from Stanford University and Carnegie Mellon University have evaluated 11 current machine learning models and found that all of them tend to tell people what they want to hear.

♾️ Most people still think machine learning means layers of neural networks stacked like digital hieroglyphs. It doesn’t. In most practical systems, you don’t need a dedicated Ai model at all, just a smart vector graph and a reasoning layer that learns from its own history. Each token, idea, or event becomes a vector, a coordinate in a shared space of meaning. Together, they form a living graph that captures relationships, proximity, and relevance that can be queried in real time. Instead of retraining a static model, you operate directly on this graph by comparing, reinforcing, and refining connections based on results. The structure itself becomes the model, fast to compute and easy to interpret. What takes seconds can now happen in milliseconds or less using this approach. In my npx agentic-flow and claude-flow frameworks, SAFLA (Self-Aware Feedback Loop Algorithm) powers this process through tiered reflection loops. Each agent runs pre, mid, and post reasoning hooks. The pre-hook builds context and queries the graph, the mid-loop reasons through vector proximity, and the post-hook scores outcomes and updates ReasoningBank. Over time, it self-optimizes, creating agents that adapt through interaction instead of retraining. This changes the premise of machine learning. Intelligence doesn’t come from massive models anymore. It comes from how well you use vectors, structure feedback, and build memory that learns. That’s where the next era of AI begins.