1-By Laksh, AI, Grok, NotebookLM

Andrej Karpathy: We’re Summoning AI Ghosts, Not Building Animals — And 3 Other Surprising Truths

Leave a comment

image by author and grok

It’s nearly impossible to escape the constant stream of AI hype. Daily announcements can make it feel like superintelligence is just around the corner. But for those in the trenches building these systems, the reality is far more complex. Andrej Karpathy, a renowned AI engineer who has led teams at both OpenAI and Tesla, approaches the field with an engineer’s “hard hat on,” offering a perspective that is deeply technical, refreshingly grounded, and often surprising.

In a recent conversation with Dwarkesh Patel, Karpathy broke down the practical realities of building AI today. This article distills four of his most counter-intuitive and impactful ideas—lessons learned from the front lines that cut through the hype and reveal the true state of artificial intelligence.

——————————————————————————–

1. We’re Summoning Ghosts, Not Building Animals

It’s common to hear AI models compared to human or animal brains, but Karpathy argues this analogy is fundamentally flawed. He proposes a different way to think about the intelligence we’re creating, one grounded in engineering reality.

Animals are products of a long, slow evolution that bakes immense capability directly into their hardware. A newborn zebra, for instance, can run and follow its mother minutes after birth—a feat of complexity that isn’t learned, but inherited. Karpathy notes that we simply don’t know how to run that optimization process.

Instead, we have what he calls a “crappy evolution”: pre-training. It’s the messy, imitation-based process we have to use because it’s the only practical version available to us. This results not in evolved creatures, but in what Karpathy calls “ghosts” or “spirits.” They are ethereal, purely digital entities whose entire nature is a compressed, “hazy recollection of the internet documents” they were trained on.

This distinction is crucial. It reframes our expectations and research, moving away from strict biomimicry and toward understanding the unique properties of an intelligence born from imitating a vast, static collection of human data.

In my post, I said we’re not building animals. We’re building ghosts or spirits or whatever people want to call it, because we’re not doing training by evolution. We’re doing training by imitation of humans and the data that they’ve put on the Internet.

——————————————————————————–

2. Today’s Reinforcement Learning Is “Terrible”

Reinforcement Learning (RL) is a key technique for improving AI models, but Karpathy offers a blunt critique of how it currently works, labeling the process “terrible,” “noisy,” and “stupid.”

The standard approach is outcome-based. A model attempts a problem (like a math equation) in hundreds of ways. It then looks at which attempts produced the correct answer and reinforces every single step taken in those successful paths.

Karpathy finds this incredibly inefficient because it incorrectly up-weights every step in a successful chain—including inefficient detours, lucky guesses, and outright mistakes—as long as the final outcome was correct. It rewards luck as much as skill.

A human, by contrast, engages in a “complicated process of review.” We reflect on our strategy, identifying which specific parts were effective and which were flawed, not just the final result. This flaw in AI learning reveals the urgent need for better supervision methods and is a major reason models still struggle with complex, multi-step reasoning.

The way I like to put it is you’re sucking supervision through a straw. You’ve done all this work that could be a minute of rollout, and you’re sucking the bits of supervision of the final reward signal through a straw… It’s just stupid and crazy. A human would never do this.

——————————————————————————–

3. AI Is Surprisingly Bad at Writing Novel Code

Coding is often hailed as AI’s biggest success story, but Karpathy’s recent experience building nanochat—a ChatGPT clone from scratch—reveals a more nuanced reality. He identifies three types of users today: those who reject LLMs, “vibe coders” who ask an agent to write entire features, and “intermediate” users like himself, who rely on autocomplete but remain the architect. From this pragmatic sweet spot, he identified a critical weakness.

LLMs excel at writing boilerplate code and implementing patterns common on the internet. However, they struggle profoundly with code that has “never been written before” or deviates from standard conventions. When Karpathy implemented a custom gradient synchronization, the models repeatedly failed to understand his intent. They kept trying to add defensive “try-catch statements” and turn his focused project into a bloated “production code base,” producing a “total mess.”

This firsthand experience directly informs his skepticism about the “year of agents.” If today’s agents, with their many “cognitive deficits,” produce “slop” when faced with a simple custom implementation, they are nowhere near ready to autonomously innovate on AI research itself. For true novelty, human architects remain essential.

They’re not very good at code that has never been written before, maybe it’s one way to put it, which is what we’re trying to achieve when we’re building these models.

——————————————————————————–

4. For True Intelligence, Perfect Memory Is a Bug, Not a Feature

One of an LLM’s most powerful capabilities is its ability to memorize and regurgitate vast amounts of training data verbatim. In a deeply counter-intuitive turn, Karpathy argues this is not a strength but a fundamental weakness—and it’s a direct consequence of their nature as digital ghosts.

Because their entire existence is based on pattern-matching a static dataset, this powerful memory distracts the model from its more important task: learning the generalizable, abstract patterns within the data. It’s a crutch that prevents the model from being forced to develop deeper reasoning.

This stands in stark contrast to human cognition. Our famously imperfect memory is a feature, not a bug. Because we can’t remember everything perfectly, our brains are forced to compress information, find underlying patterns, and “see the forest for the trees.” This compression is the foundation of true understanding.

The implication is profound. Karpathy suggests future research must find ways to strip away rote knowledge to isolate what he calls the “cognitive core”—the pure algorithms of thought. He speculates this core could be much smaller, potentially only a billion parameters, if it weren’t so burdened by the need to memorize the entire internet.

We’re not actually that good at memorization, which is actually a feature. Because we’re not that good at memorization, we’re forced to find patterns in a more general sense. LLMs in comparison are extremely good at memorization… and it’s probably very distracting to them in a certain sense.

——————————————————————————–

Conclusion: The Long March of the Builder

Andrej Karpathy’s insights reveal a coherent picture from the engineering front lines. We are building digital “ghosts” whose nature—a hazy recollection of the internet—makes them prone to a perfect-yet-distracting memory. We then try to improve them with “terrible” learning methods that reward luck as much as skill. It’s no surprise, then, that these systems falter at true novelty.

His perspective is that of a practical builder: deeply optimistic about what AI can become, but soberly realistic about the immense challenges. Getting from a cool demo to a reliable product is a “march of nines,” where every step of improvement requires monumental effort. Fundamental discoveries about learning, reasoning, and intelligence are yet to be made.

As we continue to build these powerful new forms of intelligence, Karpathy’s insights push us to ask a crucial question: Are we merely trying to build a better tool, or are we trying to create a better thinker?

Reference Link: https://www.youtube.com/watch?v=lXUZvyajciY

1-By Laksh, AI, Google AI Studio, Hugging Face Chat

Originality Across Time: From Kalidasa to the Age of Large Language Models

Leave a comment

image by author and Nano Banana via Google AI Studio

Originality—what does it mean to create something truly new? This question has echoed through the corridors of human thought for millennia, evolving in meaning with each cultural epoch. From the lyrical genius of ancient Indian poet Kalidasa to the algorithmic artistry of today’s Large Language Models (LLMs), our conception of originality has undergone profound transformation. In an age where AI chatbots co-author poems, draft essays, and even compose music, we find ourselves at a crossroads: if a machine helped create it, can it still be considered original?

The Classical Ideal: Originality as Divine Inspiration

In the 4th–5th century CE, Kalidasa, often hailed as the greatest poet and playwright in classical Sanskrit literature, composed masterpieces such as Abhijnanasakuntalam, Meghaduta, and Kumarasambhava. To his contemporaries, Kalidasa’s brilliance was not merely technical—it was seen as pratibha, a Sanskrit term denoting intuitive genius or creative insight. This concept did not emphasize novelty in the modern sense, but rather the poet’s ability to draw from tradition and yet express it with such depth and grace that it felt new.

Originality in Kalidasa’s time was not about inventing ex nihilo (from nothing), but about reimagining and refining the eternal. His works were deeply rooted in existing mythologies and poetic conventions, yet his voice was unmistakably unique. His originality lay not in breaking from tradition, but in transcending it through emotional depth, linguistic beauty, and imaginative power.

Here, originality was a synthesis: the poet as a vessel through which divine or cultural truths were re-expressed in a personal, inspired way. The idea of “plagiarism” as we know it today did not exist; instead, excellence was measured by how well one could internalize and re-voice the wisdom of the past.

The Enlightenment Shift: Originality as Individual Genius

Fast forward to the 18th and 19th centuries, and the Romantic movement redefined originality. Thinkers like Wordsworth, Coleridge, and later, Emerson, elevated the individual artist as a solitary genius creating from inner vision. Originality now meant breaking from tradition, expressing the unique self, and producing something unprecedented.

This era birthed the myth of the “solitary creator”—the poet scribbling by candlelight, the painter tormented by inspiration. Originality became synonymous with novelty, authenticity, and ownership. The copyright laws that emerged in this period reflect this shift: creativity was now property, and originality was its legal and moral foundation.

But even then, originality was never pure invention. T.S. Eliot, in his seminal essay “Tradition and the Individual Talent” (1919), argued that true originality comes not from ignoring the past, but from engaging deeply with it. The poet, he said, must be aware of “the whole of the literature of Europe,” and originality arises from the dynamic tension between the old and the new.

The LLM Age: Originality in the Era of Artificial Co-Creation

Today, we stand at the threshold of a new paradigm—one where creativity is no longer solely the domain of human minds. With the advent of Large Language Models like GPT, Claude, and Llama, machines can generate poetry, stories, code, and philosophical essays that are indistinguishable from human work—at least on the surface.

This raises urgent questions:

  • If an AI helps me write a poem, is it mine?
  • If the AI trained on millions of texts, including Kalidasa’s, is its output derivative?
  • Can a machine be original?

The answer lies not in binary thinking, but in redefining what originality means in a collaborative, data-saturated world.

First, it’s important to recognize that LLMs do not “create” in the human sense. They do not have consciousness, intention, or emotion. Instead, they statistically recombine patterns from their training data. Every sentence an AI generates is a mosaic of human expressions, reassembled through mathematical inference.

But this does not mean the output lacks originality. Consider a poet using an AI as a collaborator: they might prompt the model with a line from Meghaduta, ask for a modern reinterpretation, and then refine the AI’s response into a new poem. The final work is not the AI’s alone, nor is it purely the human’s. It is a hybrid creation—a dialogue across time and intelligence.

In this light, originality is no longer about purity of source, but about the intentionality of synthesis. Just as Kalidasa drew from the Mahabharata to create Shakuntala, today’s creators draw from a vast digital corpus, mediated by AI, to produce something new. The act of curation, editing, and personal expression becomes the hallmark of originality.

Rethinking Authorship: From Solitary Genius to Creative Partnership

We must move beyond the outdated dichotomy of “human original” versus “machine derivative.” The LLM age calls for a more nuanced understanding—one where originality is seen as a process, not a product.

Originality today may reside in:

  • The prompt—the creative spark that initiates the AI’s response.
  • The selection and refinement—the human judgment that shapes raw output into meaningful work.
  • The context—the cultural, emotional, or intellectual framework that gives the work significance.

In this view, AI does not replace the artist; it becomes a new kind of muse—one that amplifies human creativity rather than diminishing it.

Conclusion: Originality Reborn

From Kalidasa’s inspired re-tellings to the AI-assisted art of the 21st century, originality has never been about creating from nothing. It has always been about transformation—about taking the known and making it feel new, personal, and true.

In the age of LLMs, we are not losing originality. We are expanding it. The tools have changed, but the human desire to express, to connect, and to transcend remains the same.

So, if an AI helped create it—does that make it unoriginal? Not necessarily. What matters is not the tool, but the vision behind it. Originality, in the end, is not about where the words come from, but what they mean—and who gives them meaning.

As Kalidasa might say, if the lotus blooms from the mud, does its beauty depend on the soil—or the sun?

1-By Laksh, AI, Directing Business, ET, Etc., Google AI Studio, Movies

The Godfather’s Machinations: An Ancient Indian Playbook for Power

Leave a comment

image by author and google ai Studio (gemini-2.5-pro and nano banana)

The age-old Indian strategic doctrine of Sama, Dana, Bheda, and Danda—the four-fold approach to achieving one’s objectives—finds a striking, albeit darker, parallel in the reasoning and methods of Mario Puzo’s iconic character, Don Vito Corleone, and his successor, Michael, in “The Godfather.” This ancient quartet of diplomatic and political maneuvering, originating from texts like Kautilya’s Arthashastra, outlines a sequential and calculated path to influence and control, a path the Corleone family navigates with chilling precision. Both philosophies fundamentally operate from a position of strength, where the availability of these four options is in itself a testament to power. The absence of these choices reveals a stark reality for those in weaker positions.

The Four Upayas: A Corleone Correlation

The four Upayas, or strategies, are traditionally employed in a successive manner, starting with the most peaceful and escalating to the most severe. The world of “The Godfather,” while brutal, is not devoid of this nuanced progression.

Sama (Conciliation and Persuasion): This is the art of gentle persuasion, reasoning, and diplomacy. Don Vito Corleone, contrary to the stereotypical image of a mob boss, often resorts to Sama as his initial approach. He is a man who prefers to “reason with people” and believes that “lawyers with their briefcases can steal more than a hundred men with guns.” His initial interactions with those who seek his help are often calm and deliberative. For instance, when the undertaker Amerigo Bonasera comes to him seeking vengeance for the assault on his daughter, Vito doesn’t immediately resort to violence. Instead, he engages in a dialogue, albeit one that subtly asserts his power and Bonasera’s lack of respect in the past. He persuades Bonasera to accept his form of justice, thereby indebting him to the Corleone family. Similarly, his dealings with the other Mafia families are often marked by attempts at negotiation and finding mutually beneficial arrangements, as seen in the initial discussions about the narcotics trade.

Dana (Gifts and Concessions): When persuasion alone is insufficient, the offer of a gift, a bribe, or a concession comes into play. In the Corleone’s world, this is the classic “offer he can’t refuse.” This isn’t just a threat; it’s often a transaction that benefits the other party, at least on the surface. When Don Corleone wants Johnny Fontane to get the lead role in a movie, his consigliere, Tom Hagen, is first sent to the studio head, Jack Woltz, with offers of friendship and solutions to his union problems. This is an attempt at a mutually beneficial arrangement. The “gift” is the Corleone family’s powerful assistance. The refusal of this “gift” then leads to a more forceful approach. The very act of doing “favors” for people is a form of Dana, creating a web of obligations that strengthens the Don’s power.

Bheda (Creating Division and Dissension): This strategy involves sowing discord and creating rifts among opponents to weaken them from within. The intricate power plays and betrayals within the Five Families of New York are a testament to the effective use of Bheda. After the attempt on his father’s life, Michael Corleone masterfully employs this tactic. He identifies the traitors within his own family and among the rival families. The famous baptism scene, where Michael orchestrates the simultaneous assassination of the heads of the other families while he stands as godfather to his nephew, is the ultimate act of Bheda. He exploits their moments of vulnerability and their internal conflicts to eliminate them all in one swift move. This also includes turning rival factions against each other, a classic maneuver to maintain dominance.

Danda (Force and Punishment): The final and most extreme measure is the use of force, punishment, and violence. This is the option of last resort when all other methods have failed. The Corleone family, despite their preference for more subtle tactics, never shies away from Danda when necessary. The horse’s head in Jack Woltz’s bed is a terrifying application of Danda after Dana was rejected. The murders of Virgil “The Turk” Sollozzo and the corrupt police captain McCluskey by Michael are acts of Danda to protect the family’s interests when negotiations and appeals to reason have failed. The ultimate message is that the Corleone family has the capacity and the will to inflict severe punishment on those who stand in their way.

The Foundation of Strength and the Peril of Limited Options

The ability to sequentially employ Sama, Dana, Bheda, and Danda is a clear indication of a position of strength. Having these four options at hand means possessing the resources, intelligence, and power to choose the most appropriate and effective means to an end. Don Corleone’s influence is built on a foundation of wealth, political connections, and a loyal army of capos and soldiers. This allows him the luxury of starting with diplomacy and escalating only when necessary. His power is what makes his “reasonable” arguments persuasive and his “gifts” enticing.

Conversely, a lack of these options signifies weakness. A ruler in ancient India who could not offer concessions (Dana) or did not have the intelligence network to create division (Bheda) would be at a significant disadvantage. Their only recourse might be premature and potentially disastrous conflict (Danda), or complete submission.

In the world of “The Godfather,” weakness is a death sentence. Characters who lack the foresight, the strength, or the options to navigate the treacherous landscape are quickly eliminated. Sonny Corleone, despite his loyalty and passion, is too impulsive and lacks the strategic patience to effectively use the four Upayas. His public outburst of anger at Sollozzo is a sign of weakness that is later exploited. Fredo Corleone’s weakness and lack of intelligence make him a liability, ultimately leading to his tragic end.

When the Corleone family is in a position of perceived weakness, such as after the assassination attempt on Vito, their options become limited. They are forced to rely more heavily on Bheda and Danda to survive and re-establish their dominance. Michael’s swift and brutal actions are a direct response to the family’s vulnerability.

In conclusion, the strategic philosophy of Sama, Dana, Bheda, and Danda provides a compelling framework for understanding the methodical and calculated approach to power employed by Mario Puzo’s Godfather. The Corleone family’s success is not merely a product of brute force, but of a sophisticated understanding of human nature and the strategic application of a range of tactics, from peaceful negotiation to ruthless violence. This approach, however, is a luxury afforded by a position of immense strength. For those without the power to choose their means, the world is a far more dangerous and limited place, a reality that both the ancient strategists and the modern dons understood all too well.

ET, IT…and the rest