5.5.4 A Pragmatic View of Chatbots: Part 4
The Text-World of Chatbots
Some AI researchers, such as the Nobel prize-winner Geoffrey Hinton, are now arguing that large language models show ‘understanding’ in a way that is directly analogous to human comprehension (see this video from 2025) . I prefer to think of them, in the amusingly memorable words of the computational linguist Emily Bender and her colleagues, as very sophisticated “stochastic parrots“. From this viewpoint, the ‘Chatbot parrots’ are reassembling human generated text patterns in a fantastically complicated game of word association in which the input words are being associated with output words. Like all metaphors the ‘parrot’ idea should not be stretched too far since the chatbot does not store or copy and paste text segments from the source material. ‘Parroting’ in this sense merely means using words without having a meaningful association outside of the text. For humans many words can be associated with features of the external world that only a sentient creature can perceive. Other words are associated with states of mind that we call emotions, feelings or urges, which machines lack. We use words to describe previous states of the world and to make sensory or mechanistic predictions. However we at the same time constantly make predictions about the world without using any linguistic terms. In addition, we can also appreciate that words can also be metalinguistic, that is, descriptive of language and symbols. Machines have none of these characteristics. Conversely pre-school children do not rely on statistical measures of word associations. They have no need of numerical tokens when using words. Normalised numerical parameters which are used in dot product calculations, and which therefore form part of the basis of current chatbot function, do not even seem to have a proven algorithmic connection with brain function.
Hinton seems to be of the opinion that chatbots ‘understand’ because these systems are able to generate fluent text streams that are often conceptually coherent over more domains of knowledge than a large group of people could acquire in many lifetimes. My intuition is that we should not equate fluent text production to a machine with human-like comprehension. We should also be careful to distinguish fluency of bot output from incisiveness, usefulness, applicability and accuracy. The modern chatbot is clearly a master of syntax. The more interesting issue is it also a master of semantics? If you accept that meaning can be understood merely in terms of use as the later Wittgenstein, would have had us believe when he invented the idea of language games, then you might indeed think that chatbots tend towards mastery of semantics. However you will also have to accept that long strings of numbers, with no connection to the world can also encode meaning. Personally I feel that a crucial part of language learning in children involves the use of ostensive definition (also see my previous remarks) and biological embodiment in the world.
A more pragmatic way to view the chatbot is a text generating machine that can indeed act as a tool to generate erudite text ‘games’. It might also help to consider the Chinese Room argument proposed in 1980 by the philosopher John Searle and summarised, as follows in the Stanford Encyclopedia of Philosophy:
“Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese”
Hinton might be confusing novel and much more versatile ways to store and retrieve information with understanding. However, if human memory and understanding involves some kind of associative learning, particularly when abstract concepts (like epistemology) are encountered, Hinton may indeed be making a legitimate metaphoric or algorithmic connection. (see my previous comments on human reasoning). Indeed the announcement of Nobel prize committee specifically mentions ‘associative memory’ in relation to the dynamical Hopfield Neural Network of 1982 which is an early precursor of today’s Transformer Models.