I was intrigued as I perused this article this morning, “Thinking Machines That Don’t: Confronting AI’s Biases & System Flaws” by Markus Bernhardt. As I read, it was clear that the author’s goal was to support a claim often heard about the artificial intelligence (AI) tools that have come on the scene over the past two years. The article states, “At their architectural heart, LLMs are not thinking entities but extraordinarily sophisticated sequence processors.” Specifically, the claim is that large language models merely place one word after the other in an order derived from the probabilities of word occurrence that it has obtained during training by being fed huge amounts of training data.
I submitted the piece to xAI’s Grok and requested that it summarize the piece in a single paragaph:
Markus Bernhardt’s article, Thinking Machines That Don’t: Confronting AI’s Biases & Systemic Flaws (July 7, 2025), critiques the limitations of large language models (LLMs), arguing that their fluent, confident outputs create a deceptive illusion of understanding despite lacking genuine cognition, intentionality, or a persistent world model. It highlights the “paradox of neutrality,” where efforts to make LLMs unbiased result in a bias toward acquiescence, reinforcing dominant norms and flattening complex considerations, as seen in corporate and educational settings. The article also points to LLMs’ tendency to perpetuate prevalent discourse due to their statistical foundations and their “contextual collapse,” which applies a uniform approach across diverse knowledge domains, risking epistemological missteps. While promising a follow-up on persistent error patterns, the piece calls for critical, human-centric use of LLMs to counter their inherent flaws and prevent over-reliance in high-stakes contexts.
I also asked Grok to assess the piece and received this rather lengthy commentary that included this:
Conclusion
Bernhardt’s article is a necessary wake-up call, urging us to look beyond LLMs’ polished outputs and confront their systemic flaws: the illusion of understanding, the paradox of neutrality, prevalence reinforcement, and contextual collapse. These critiques are grounded in the technical reality of LLMs as statistical pattern-matchers, not cognitive entities. While the article slightly overgeneralizes the risks and underplays potential mitigations, its core claims are valid and urgent, especially as LLMs proliferate in critical domains. It’s a reminder that tools like me are powerful but fallible, requiring active, critical engagement from users to avoid being misled by our confident facades. The promised follow-up on error patterns should further enrich this discussion, and I look forward to reacting to it when it’s published.
It is interesting to note what some might feel is a remarkable sense of self-awareness in the statement there, “It’s a reminder that tools like me are powerful but fallible… The promised follow-up on error patterns should further enrich this discussion, and I look forward to reacting to it when it’s published.”
While I have no substantive rebuttal to the idea that these systems do not “think,” I have to say that I am often amazed at the non-trivial insights often provided by any one of the several LLMs I use on a daily basis. Also interesting is the fact that my wife, a non-user of LLMs frequently agrees with me when I share one of those responses with her.
