The Muses
January 9, 2025
Good Vibes
February 13, 2025

ChatGPT and Writing

Photo by Ivan Obolensky

ChatGPT has begun to influence how books and articles are written and will likely do so in greater volume in the future. Many have opinions about it, but few know what it is and how it works.

The GPT of ChatGPT stands for Generative Pre-trained Transformer (a kind of neural network), the meaning of which requires some explanation.

A neural network is a mathematical construct that contains layers of tiny calculators called nodes. Each receives numerical inputs, processes them, and sends its results forward to nodes in a layer above it or down to nodes below it. This feedback and feedforward architecture allows the network to pick up subtle patterns in the data presented.

Neural networks do not follow logical steps like digital programs. Instead, they take large amounts of data, process them, and recognize specific patterns. Programs like voice recognition, face recognition, retinal scans, and thumbprint identification use specialized neural networks to identify individual patterns (you) when presented.

Transformer neural networks have an additional function. They incorporate positional encoding, how parts are arranged, and in what sequence. Transformers look at long-range dependencies—how a part farther along in a sequence relates to a part at the beginning. This allows answers to replicate language structure.  

As an example, the sentence, “The cat scratched the man” is very different from “The man scratched the cat”. The positioning of “the man” and “the cat” is critical to the meaning of the sentence, and the network would place the cat as the subject and the man as the object based on its learning. The process may appear intelligent to an observer, but no intelligence is involved—just word location, language structure, learned patterns, and probability. This alone is a huge technological accomplishment.

More significant is that the underlying structure of the language (word order, syntax, usage, etc.) has been identified sufficiently to be codable as inputs and decodable as outputs.  

A transformer is like a mini-translator, but to be effective, it must be taught how to translate.

ChatGPT uses a large open-source dataset of English text called The Pile to get the language formatting correct. This allows the network to learn when and how to answer a question, paraphrase a query, and determine the sentiment of the query to be able to respond.

Most neural networks are fixed or frozen; for instance, a retinal scanner does not require further learning, but some, like ChatGPT, can be finetuned and modified to give better responses over time.

One can even over-train a network on past data such that the network cannot generalize new data accurately. This results in more errors in the future. Errors are typical of all neural networks. From my experience using them to create trading programs, a typical network generates an error rate of around fifteen percent. One can correct for this using better architectures, higher quality and larger quantities of data, but no neural network is always error-free.

Imagine asking a stranger for directions in a foreign land. Sometimes, they happily say you turn left there, then right, then go straight. It’s on your left. You thank them only to find the directions absolutely bogus. Did the person deliberately steer you wrong? Chances are they didn’t understand what you said, and you didn’t understand what they said. The result is errors. This is typical of any neural network. Not all the time, but often enough. You think you were clear when you weren’t. You thought you understood, but you didn’t.

Still, you talked and had a conversation. Wasn’t that wonderful?

The subject of AI has captured the public imagination because, with ChatGPT and others like it, one can receive answers that appear meaningful. This interaction brings the possibility of artificial intelligence and machine sentience one step closer to reality in the minds of the general public.

Unfortunately, that possibility is still an apparency. Neural networks only process numerical data, not words. The translations (from words to numbers and back to words), which is what they actually do, require sophisticated programming and a supercomputer. 

ChatGPT may be able to generate content, but that content will never be original. It can’t be because creation is not simply a rearrangement or an expression of a rearrangement. Creation implies something new and something altogether different. One might even go so far as to say it is a form, an idea, a concept, or a connection that has never been seen before.

To get a sense of that and to paraphrase Schopenhauer:

“Talent is like the marksman who hits the target that others cannot reach. Genius is the marksman who hits the target which others cannot even see.”

ChatGPT is technologically sophisticated and may even be mistaken for talent by those who claim what it produces as their own, but such networks can never create. That takes genius, something only you in all your humanness can demonstrate.

Don’t sell yourself short.

You are unique and that is a concept worth considering and building upon.

Comments are closed.