The Illusion of Thinking: Why LLMs Aren’t AGI and Synthetic Brains Might Be

Introduction: When “Thinking” Isn’t Intelligence

For the last few years, the AI story has been owned by large language models: systems that can write essays, pass exams, and increasingly “show their work” with chain‑of‑thought reasoning. At first glance, they look like the long‑promised artificial general intelligence—human‑level intelligence in silicon—but under stress they behave less like minds and more like powerful autocomplete engines.

A recent Apple study calls this the “illusion of thinking”: models that appear to reason but collapse precisely where humans start to shine, as task complexity grows. For Qognetix, this is not just a quirk of current models; it is a symptom of a deeper architectural mistake. Current systems are optimised for pattern‑matching over static datasets, not for living, adaptive cognition. Qognetix exists to explore the alternative: synthetic brains—biologically faithful, neuromorphic architectures that behave more like cortex than like a giant statistical spreadsheet.

What AGI Really Means (And Why Benchmarks Mislead)

The term AGI has been stretched so far that almost any impressive model can be marketed as “general,” but most serious definitions converge on a few core criteria.

• Breadth: Competence across most cognitive domains—language, perception, planning, abstraction, creativity—at roughly human level, not just on a handful of curated benchmarks.
• Transfer: Robust generalisation to new tasks and situations, including those never seen in training, without catastrophic failure or brittle prompt‑sensitivity.
• Autonomy: Lifelong learning and self‑improvement in open environments, instead of one‑shot pretraining followed by slow, manual updates.

On these terms, today’s LLM‑centric systems look less like general intelligences and more like broad but brittle savants: astonishingly capable inside the statistical envelope of their training data, and disturbingly unreliable outside it. Worse, the industry has learned to game the definition—declaring “AGI milestones” when a new model crosses a benchmark threshold that says more about the test than about genuine generality.

Qognetix takes a different stance. If AGI means anything, it should mean systems that share the structural properties that make brains general: continual plasticity, grounded world models, and energy‑efficient computation tightly coupled to experience. That is why the Qognetix platform focuses on building and simulating digital neural tissue—not as a metaphor, but as a substrate where breadth, transfer, and autonomy emerge from brain‑like dynamics rather than from ever larger piles of text.

Why Pattern‑Matching LLMs Hit a Wall

At their core, large language models and the newer “large reasoning models” are next‑token predictors: immense pattern‑matching systems that learn to continue text so well that the continuations look like thought. Apple’s “Illusion of Thinking” paper makes that gap explicit, showing that as puzzle complexity increases, these models hit a collapse threshold where they abruptly reduce their reasoning effort and start failing—despite having plenty of context window left.

Three structural problems keep resurfacing:

• Approximate reasoning instead of algorithms
  Apple’s analysis finds that LRM-style models do not reliably implement exact procedures for logic or arithmetic; they approximate them via pattern recall, leading to inconsistent answers across similar problems and a sharp breakdown on harder instances. Follow‑up work on compositional benchmarks shows the same pattern: small changes in structure or wording can destroy performance, which is not how robust algorithms behave.
• Brittleness and lack of systematic generalisation
  Studies on compositional generalisation demonstrate that many LLMs struggle to combine learned concepts in new ways, succeeding on training‑like instances but failing as soon as tasks require multi‑step composition. Even when specialised methods like CARMA improve compositionality on narrow benchmarks, the gains are fragile and task‑specific, underscoring how hard it is to graft real structure onto a fundamentally statistical engine.
• No continual, embodied learning
  Today’s frontier models are trained in giant offline batches, then frozen and occasionally fine‑tuned; they do not learn continuously from the world, and they are prone to catastrophic forgetting when updated. By contrast, neuromorphic and continual‑learning research explicitly targets systems that accumulate knowledge over a lifetime while balancing stability and plasticity, something brains do but LLMs only simulate at the surface.

For Qognetix, this is the crux: if an architecture gives you approximate reasoning, brittle generalisation, and no true lifelong learning, calling it “AGI” is a category error. The Qognetix platform exists precisely because those failure modes look like symptoms of the wrong substrate, not just the wrong scale—and because brain‑like synthetic tissue offers a way to bake generality, compositional structure, and plasticity into the system from the ground up.

The Case for Biologically Faithful Synthetic Brains

If LLMs keep failing in the same ways—approximate reasoning, brittle generalisation, no true lifelong learning—the obvious conclusion is that the architecture itself is wrong for AGI. Brains solve exactly the problems LLMs struggle with: they learn continuously, generalise compositionally, stay robust under noise and novelty, and do it all within a tight energy budget, which is why the most credible path to real AGI is to build systems that share their structural principles.

What Neuromorphic Computing Brings

Neuromorphic computing starts from this premise and designs hardware and algorithms around spiking neurons, synapses, and event‑driven, massively parallel dynamics inspired by biological nervous systems.

Recent work highlights three advantages that map directly onto AGI requirements:

• Energy‑efficient, always‑on cognition
  Neuromorphic platforms can perform sensing and inference at orders of magnitude lower power than GPU‑bound deep nets, enabling rich, continuous cognition at the edge rather than in remote data centres. That moves AI closer to how brains operate: always on, always learning, without a hyperscale power plant behind each thought.
• Temporal, event‑based computation
  Spiking networks naturally encode time and sequence in their dynamics, handling streams of events instead of static batches of tokens. This aligns with how organisms integrate sensory input, internal state, and action over time, and it is a much better fit for open‑ended agents than the single‑shot, prompt‑in‑completion‑out loop of current LLMs.
• Hardware level plasticity and continual learning
  Emerging neuromorphic systems support local learning rules and online synaptic updates, allowing models to adapt in real time without catastrophic forgetting. This is exactly the stability–plasticity balance that continual‑learning researchers chase in software, but implemented directly in the substrate, much closer to biology.

Qognetix: Synthetic Digital Tissue as an AGI Substrate

Qognetix takes these ideas one step further by focusing not just on neuromorphic hardware in the abstract, but on synthetic digital neural tissue that behaves like a living cortex inside a controllable software environment. Instead of treating neurons as simple units in a matrix, the Qognetix platform models populations of interconnected, biologically inspired elements whose activity unfolds over time and can be probed, perturbed, and reconfigured experimentally.

In practise, that means:

• Building and simulating digital “brain regions” with rich internal dynamics, rather than static layers in a feed‑forward network.
• Exploring neuromodulation‑like influences, plasticity rules, and multi‑timescale memory mechanisms that let synthetic tissue learn continuously while retaining past skills.
• Using this synthetic tissue as a testbed for AGI‑relevant behaviours—robust generalisation, compositional reasoning, adaptive control—before coupling it to neuromorphic hardware and real‑world embodiments.

Where LLM labs scale parameters and compute, Qognetix scales structure: more realistic neurons, richer connectivity, and brain‑like dynamics that make general intelligence a first‑class design goal rather than an accidental by‑product of language prediction.

LLM‑Centric “AGI” vs Neuromorphic AGI

At this point the AGI debate is really a clash between two futures: one where “AGI” is declared when LLMs plus plugins hit a moving benchmark, and another where AGI is reserved for systems that genuinely behave like general, adaptive minds. Putting those futures side by side makes the gap—and Qognetix’s bet—very clear.

How Ridiculous The “AGI One‑Upmanship” Has Gotten

At one end, Sam Altman says OpenAI “knows how to build AGI” and is already thinking past it; at the other, he dismisses AGI as a “sloppy” term that no longer means much. Elon Musk keeps moving his AGI deadline forward a year at a time while tying it to ever‑bigger GPU clusters, and Mark Zuckerberg is pitching an open‑source Llama “tsunami” as if scaling the same brittle architecture will magically produce general intelligence.

When the people building the largest pattern‑matching machines in history cannot even agree on what AGI is, yet confidently declare timelines to it, the only reasonable response is to admit how ridiculous this has gotten—and to start talking seriously about architectures, like synthetic brains, that might actually deserve the name.

A More Honest AGI Roadmap

If AGI is allowed to mean “whatever today’s biggest model can just about do,” the term becomes a marketing label, not a scientific milestone. A more honest roadmap starts by asking what brains actually deliver—and then demanding the same from any system claiming the “general” label.

Three questions cut through the hype:

• Does it learn continuously from the world without forgetting how to walk every time it learns to run? Brains adjust synapses in real time while preserving core skills; LLMs, by contrast, need expensive retraining and still forget.
• Does it maintain coherent world models and robust behaviour under distribution shift, noise, and novelty, or does it crumble when the prompt is phrased differently?​
• Can it operate within biological energy and latency constraints, closer to a brain than to a data centre, so that intelligence can live on devices, robots, and organisms, not just in clouds?

On these criteria, today’s LLMs look like a brilliant but transitional technology: an important chapter, not the end of the story. The Qognetix thesis is that truly general intelligence will come from synthetic brains—biologically faithful digital tissue and neuromorphic substrates that inherit the structural advantages of cortex, then extend them with the precision and control of computation.​

In other words: the question is not whether AGI is “here” in your chatbot; it is whether we are willing to rebuild AI on foundations that actually look like intelligence—messy, plastic, grounded, and alive—and Qognetix exists to help make that shift real.

The Qognetix Call To A Realistic AGI Pathway

If you’re building in this space, Qognetix is looking for early collaborators, critics, and co‑conspirators who are ready to move beyond LLM cosmetics and into synthetic brains. If you want to experiment with biologically faithful digital tissue, neuromorphic‑ready architectures, or simply pressure‑test this thesis, you can connect with Qognetix, join the early access list, or reach out directly to explore pilots and research partnerships.

Sources

1. https://en.wikipedia.org/wiki/Artificial_general_intelligence
2. https://debateus.org/what-is-artificial-general-intelligence-definitions-from-experts/
3. https://www.ibm.com/think/topics/artificial-general-intelligence
4. https://www.mckinsey.com/featured-insights/mckinsey-explainers/what-is-artificial-general-intelligence-agi
5. https://www.science.org/doi/10.1126/science.ado7069
6. https://research.aimultiple.com/artificial-general-intelligence-singularity-timing/
7. https://arxiv.org/abs/2503.23923
8. https://aireapps.com/articles/why-might-the-llm-market-not-achieve-agi/
9. https://www.njii.com/2024/07/why-llms-alone-will-not-get-us-to-agi/
10. https://mashable.com/article/apple-research-ai-reasoning-models-collapse-logic-puzzles
11. https://machinelearning.apple.com/research/illusion-of-thinking
12. https://www.infoq.com/news/2025/07/apple-illusion-thinking/
13. https://arize.com/blog/the-illusion-of-thinking-what-the-apple-ai-paper-says-about-llm-reasoning/
14. https://www.linkedin.com/pulse/illusion-thinking-what-apples-research-reveals-ai-vaibhav-kulshrestha-tmlic
15. https://arxiv.org/html/2410.01748v1
16. https://openreview.net/pdf?id=aPAWbip1xV
17. https://aclanthology.org/2025.emnlp-main.822.pdf
18. https://archiv.ub.uni-heidelberg.de/volltextserver/37640/1/thesis_fxy.pdf
19. https://www.nature.com/articles/s43588-025-00770-4
20. https://www.pnas.org/doi/10.1073/pnas.2528654122
21. https://humanunsupervised.com/papers/neuromorphic_landscape.html
22. https://www.alphagamma.eu/entrepreneurship/neuromorphic-computing-artificial-intelligence/
23. https://www.linkedin.com/pulse/neuromorphic-computing-next-frontier-brain-inspired-ai-ramachandran-h0xxe
24. https://www.linkedin.com/posts/satvik-tripathi_they-didnt-code-this-aithey-grew-it-from-activity-7350372582451232768-jFU1
25. https://nomadit.co.uk/conference/easst-4s2024/paper/81733
26. https://situated-cognition.com/2024/01/03/neuromorphic_ai/
27. https://arxiv.org/html/2504.20109v1
28. https://arxiv.org/html/2410.09218v3
29. https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2025.1768235/full
30. https://www.reddit.com/r/deeplearning/comments/1o6nfqb/new_generation_bioinspired_ai_architecture_moving/
31. https://www.reddit.com/r/agi/comments/1clsdy8/we_need_to_get_back_to_biologically_inspired/