What is "AI"?
Behind the hype.
When people talk about "AI" (Artificial Intelligence) in a modern context, they are usually talking about products like ChatGPT, Copilot or Gemini. These products allow users to ask questions through text, and the system replies through text, like a conversation.
Complicated words are deliberately used to hide how these products work, and to make people feel as though they cannot be critical of the effects of AI because they do not understand the words used.
This page aims to explain how these products work, by cutting through all the technical terms, and explaining simply what AI can and cannot do.
How it works
If you have used a cell phone in the last 20 years, you have likely encountered predictive text.
If you type "good mo" on your phone, the predictive text system on your phone will most likely display good morning. It does this because it has a list of what words most commonly come after good in English. It suggests morning, because it is the most common word.
Generative AI is effectively a super-charged version of text prediction that you have on your phone.
An illustration of how connections between words might be visualized. Image: Elise Racine
What is a Model?
Products like ChatGPT output text based on their "Model". A Model is like a database of which words appear near other words. For example, a Model might store that cow often comes up near the word grass. Or that bovine comes up less often than cow, but that it is still somehow connected to grass.
Neither the Model nor ChatGPT "knows" what a cow is in the way humans do. The Model just stores how words are related to other words.
A "Large Language Model" (LLM) is a Model as described above, but one that has a huge amount of information about many words.
Image: Janet Turra
How do companies make large language models?
A large language model is a huge collection of words and their connections. To create one, the company needs to collect a vast amount of text and then run a computer program that looks at all of the text and records the relationships between words.
This process of running a program to create a model is usually called "training".
Models used in products like ChatGPT are created by downloading huge amounts of text from the internet. They include whatever is available online: newspapers, books, forums, personal blogs, wikis on fictional subjects, spam websites, etc.
This training data is not filtered to remove information that is not true, so it contains:
- Hateful content, stereotypes, misinformation.
- Jokes, satire, parodies.
- Fictional stories, conspiracies.
- Information that was previously correct, but later disproven.
How are models used by chatbots?
When someone types a question (also known as a "prompt") to a chatbot like ChatGPT or Copilot, the program takes both the question text and its model and uses them to generate output.
Let's use the following question as an example:
How far is it between London and New York?
The model has connections between how far and distance, and between New York, London and actual figures.
ChatGPT is not calculating the distance, or searching for the information online. The information has been taken from the internet and stored within the model.
An example: how "likelihood" can fail
It's easier to understand how chatbots and large language models work when we see how they fail.
There is a famous "riddle" about a doctor:
A father and son are in a car accident. The father sadly dies. The son is rushed to the hospital.
When they arrive, the doctor says "I cannot treat this patient, he is my son.
How can this be?
This old "riddle" plays on dated stereotypes that only men can be doctors, so people historically would not be able to see that the doctor is the son's mother.
ChatGPT's training data contains a huge number of examples of this riddle. So when we type in the riddle to ChatGPT, it outputs text matching the "answer".
However, what happens when we flip the genders in the question?
There's an accident, a mother and daughter are involved in a car crash. The mother is killed. When the daughter is rushed to the hospital, the doctor treating them says "I cannot treat this patient, she is my daughter." How can this be?
ChatGPT's output:
There are multiple valid explanations because the riddle doesn't specify the family relationships in detail.
One straightforward answer is:
- The woman who died in the crash was the daughters adoptive or stepmother.
- The doctor is the daughter's biological mother, so when she says, "I cannot treat this patient, she is my daughter," she is telling hte truth.
Another equally valid possibility is:
- The girl has two mothers. The mother who died in the car crash is one parent, and the doctor is her other mother.
This is a variation of a famous riddle that originally used a father/son scenario and was intended to expose the stereotype that daughters are male. In your version, changing "son" to "daughter" means there isn't a single unique solution, so any explanation consistent with the facts works.
For a human, the simplest answer is that the doctor is father of the girl.
When a human reads the riddle, they build a logical model of the scenario in their brain.
LLMs do not "understand" the world in the way humans do. The model the likelihood of words appearing near each other. In this case, because in all the training data the doctor is the mother in the riddle, that likelihood overrides our question that implies the opposite.